Description
Large-language-model inference can use ROCm acceleration through the vLLM serving engine. This variant targets AMD GPU systems that need high-throughput model serving and memory-efficient batching.
It may expose an inference server and consume substantial GPU memory. Validate ROCm compatibility, restrict service access, and protect prompts or outputs that may contain private data.