# Hardware Support **URL:** https://discuss.vllm.ai/c/hardware-support/5.md [Latest](https://discuss.vllm.ai/latest.md) · [Categories](https://discuss.vllm.ai/categories.md) --- ## [About the Hardware Support category](https://discuss.vllm.ai/t/about-the-hardware-support-category/62)
**Author:** [@youkaichao](https://discuss.vllm.ai/u/youkaichao)\ **Replies:** 0
--- ## [How to Look at the metrics/performance in our Blackwell 6000 seetup](https://discuss.vllm.ai/t/how-to-look-at-the-metrics-performance-in-our-blackwell-6000-seetup/3083)
**Author:** [@bcodermicro](https://discuss.vllm.ai/u/bcodermicro)\ **Replies:** 1\ **Last updated:** [October 23, 2026, 9:32pm UTC](https://discuss.vllm.ai/t/how-to-look-at-the-metrics-performance-in-our-blackwell-6000-seetup/3083 "2026-10-23T21:32:19Z")
Hello there we are using blackwell 6000 on Dell Servers and we are using openshift and k6 to do performance testing.. we have a grafana dashboard set up and using a 28B gemma model. quantized 8bit. What we are running i… --- ## [Vllm-ascend 支持 Bidirectional KV Transfer特性吗，KV双向传输](https://discuss.vllm.ai/t/vllm-ascend-bidirectional-kv-transfer-kv/3068)
**Author:** [@zxTsai](https://discuss.vllm.ai/u/zxTsai)\ **Replies:** 1\ **Last updated:** [October 20, 2026, 9:17am UTC](https://discuss.vllm.ai/t/vllm-ascend-bidirectional-kv-transfer-kv/3068 "2026-10-20T09:17:56Z")
vllm-ascend 支持 Bidirectional KV Transfer特性吗，KV双向传输 --- ## [Title: OutOfResources: shared memory (98304 \> 65536) on Turing (SM75) — global attention layers with head\_dim=512](https://discuss.vllm.ai/t/title-outofresources-shared-memory-98304-65536-on-turing-sm75-global-attention-layers-with-head-dim-512/2990)
**Author:** [@GentleHarbor25](https://discuss.vllm.ai/u/GentleHarbor25)\ **Replies:** 2\ **Last updated:** [September 20, 2026, 7:13am UTC](https://discuss.vllm.ai/t/title-outofresources-shared-memory-98304-65536-on-turing-sm75-global-attention-layers-with-head-dim-512/2990 "2026-09-20T07:13:02Z")
Title: OutOfResources: shared memory (98304 \> 65536) on Turing (SM75) — global attention layers with head\_dim=512 Summary I am unable to serve this checkpoint on NVIDIA Turing (compute capability 7.5) GPUs. vLLM fails d… --- ## [Is this normal for vllm serve output waiting 1 hour b´before startup](https://discuss.vllm.ai/t/is-this-normal-for-vllm-serve-output-waiting-1-hour-b-before-startup/2946)
**Author:** [@edib-isic](https://discuss.vllm.ai/u/edib-isic)\ **Replies:** 7\ **Last updated:** [September 9, 2026, 7:29am UTC](https://discuss.vllm.ai/t/is-this-normal-for-vllm-serve-output-waiting-1-hour-b-before-startup/2946 "2026-09-09T07:29:58Z")
i just started vllm serve is this the normal output of vLLM is it stuck because cannot reach it on localhost:8000 no INFO Server Started or serving at?? is this ok output?? everything seems ok $ vllm serve unsloth/… --- ## [请问一下最新版的vllm-ascend是否支持GLM-5.3 flash](https://discuss.vllm.ai/t/vllm-ascend-glm-5-3-flash/2931)
**Author:** [@IceWater](https://discuss.vllm.ai/u/IceWater)\ **Replies:** 1\ **Last updated:** [August 28, 2026, 8:40am UTC](https://discuss.vllm.ai/t/vllm-ascend-glm-5-3-flash/2931 "2026-08-28T08:40:12Z")
请问一下最新版的vllm-ascend是否支持GLM-5.3 flash和deepseek v4-flash 0731 --- ## [GFX1030 Support PR](https://discuss.vllm.ai/t/gfx1030-support-pr/2930)
**Author:** [@BlivionIaG](https://discuss.vllm.ai/u/BlivionIaG)\ **Replies:** 1\ **Last updated:** [August 27, 2026, 10:07pm UTC](https://discuss.vllm.ai/t/gfx1030-support-pr/2930 "2026-08-27T22:07:49Z")
Hello there, I couldn’t find a space or place to ask about where to discuss the gfx1030 pr feat(rocm): enable gfx1030 (RDNA2) as a recognized RDNA arch by BlivionIaG · Pull Request #52391 · vllm-project/vllm · GitHub R… --- ## [Can Qwen 3.5 models (397B or 9B) run on a single TPU v5e-8 node?](https://discuss.vllm.ai/t/can-qwen-3-5-models-397b-or-9b-run-on-a-single-tpu-v5e-8-node/2876)
**Author:** [@arjuntech-07](https://discuss.vllm.ai/u/arjuntech-07)\ **Replies:** 3\ **Last updated:** [August 15, 2026, 9:37am UTC](https://discuss.vllm.ai/t/can-qwen-3-5-models-397b-or-9b-run-on-a-single-tpu-v5e-8-node/2876 "2026-08-15T09:37:48Z")
Hi everyone, I was reviewing the tpu-inference README and noticed that TPU v5e is listed as a recommended architecture. I am trying to figure out if I can run Qwen 3.5 models on a single TPU v5e-8 node. Looking at the … --- ## [ROCm/vLLM issues compared to CUDA/vLLM](https://discuss.vllm.ai/t/rocm-vllm-issues-compared-to-cuda-vllm/2859)
**Author:** [@DMKode](https://discuss.vllm.ai/u/DMKode)\ **Replies:** 3\ **Last updated:** [August 11, 2026, 7:10pm UTC](https://discuss.vllm.ai/t/rocm-vllm-issues-compared-to-cuda-vllm/2859 "2026-08-11T19:10:17Z")
Hello, I am developing an app that uses LLMs and needs vLLM to run. To be honest I am not so deeply technical so I am using AI for support and during our testing we found out that performance our app is much worse on RO… --- ## [Support for Nvidia A2 Tensor Core GPU, stuck on shm\_broadcast, unsure how to debug](https://discuss.vllm.ai/t/support-for-nvidia-a2-tensor-core-gpu-stuck-on-shm-broadcast-unsure-how-to-debug/2828)
**Author:** [@lkldev](https://discuss.vllm.ai/u/lkldev)\ **Replies:** 2\ **Last updated:** [August 6, 2026, 11:03am UTC](https://discuss.vllm.ai/t/support-for-nvidia-a2-tensor-core-gpu-stuck-on-shm-broadcast-unsure-how-to-debug/2828 "2026-08-06T11:03:48Z")
Hi, Recently I am testing out running vLLM on multiple Nvidia A2 Tensor Core GPUs, but most of the models that I tested with are stuck at shm\_broadcast.py, No available shared memory broadcast block found in 60 seconds.… --- ## [Issue when running vllm on the cpu](https://discuss.vllm.ai/t/issue-when-running-vllm-on-the-cpu/2856)
**Author:** [@dozed](https://discuss.vllm.ai/u/dozed)\ **Replies:** 1\ **Last updated:** [August 4, 2026, 5:26pm UTC](https://discuss.vllm.ai/t/issue-when-running-vllm-on-the-cpu/2856 "2026-08-04T17:26:36Z")
Hi, when running vllm on the cpu via uv run vllm serve Qwen/Qwen2.5-7B-Instruct I get the following error: ERROR 08-04 18:59:00 \[v1/executor/multiproc\_executor.py:901\] WorkerProc failed to start. ERROR 08-04 18:59:00 \[v… --- ## [Performance Tuning for L40s with Qwen36-35B](https://discuss.vllm.ai/t/performance-tuning-for-l40s-with-qwen36-35b/2848)
**Author:** [@Gulensah](https://discuss.vllm.ai/u/Gulensah)\ **Replies:** 1\ **Last updated:** [July 29, 2026, 2:26pm UTC](https://discuss.vllm.ai/t/performance-tuning-for-l40s-with-qwen36-35b/2848 "2026-07-29T14:26:45Z")
Dear all, I have a VM, underlined with Proxmox and 2 x Nvidia L40s. I passthrough the cards to quest VM which is Ubuntu. According to the documentation offical guide (vLLM Parallelism and Scaling), if their is no NVLINK… --- ## [Running vLLM on Intel Arc iGPU](https://discuss.vllm.ai/t/running-vllm-on-intel-arc-igpu/2835)
**Author:** [@ekaakurniawan](https://discuss.vllm.ai/u/ekaakurniawan)\ **Replies:** 1\ **Last updated:** [July 26, 2026, 6:13am UTC](https://discuss.vllm.ai/t/running-vllm-on-intel-arc-igpu/2835 "2026-07-26T06:13:58Z")
The vLLM memory allocation for discrete and integrated GPUs needs to be planed differently. For dGPU, the default GPU memory utilization is set to 0.8 (80%) of the total GPU memory. This may not suitable for iGPU in whic… --- ## [Vllm-ascend会不会支持gemma4多模态功能](https://discuss.vllm.ai/t/vllm-ascend-gemma4/2830)
**Author:** [@Dexterai](https://discuss.vllm.ai/u/Dexterai)\ **Replies:** 1\ **Last updated:** [July 24, 2026, 2:56am UTC](https://discuss.vllm.ai/t/vllm-ascend-gemma4/2830 "2026-07-24T02:56:07Z")
gemma4的多模态需要attention算子支持新的mask方式 --- ## [INTEL XPU: Dual Arc B60Pro -\> Single Arc B70Pro](https://discuss.vllm.ai/t/intel-xpu-dual-arc-b60pro-single-arc-b70pro/2770)
**Author:** [@Huehnerbrust](https://discuss.vllm.ai/u/Huehnerbrust)\ **Replies:** 1\ **Last updated:** [July 11, 2026, 2:55pm UTC](https://discuss.vllm.ai/t/intel-xpu-dual-arc-b60pro-single-arc-b70pro/2770 "2026-07-11T14:55:33Z")
Hi all, im am currently running a dual arc b60 pro setup. I got this for testing, and since I knew that I had large documents to process I went for the 48GB VRAM of that setup. I am serving with VLLM to an openwebui. M… --- ## [Making best use of varying GPU generations](https://discuss.vllm.ai/t/making-best-use-of-varying-gpu-generations/396)
**Author:** [@chris](https://discuss.vllm.ai/u/chris)\ **Replies:** 4\ **Last updated:** [July 4, 2026, 7:57pm UTC](https://discuss.vllm.ai/t/making-best-use-of-varying-gpu-generations/396 "2026-07-04T19:57:15Z")
I have three NVIDIA CUDA-capable GPUs, and that is where the good-sounding part ends. First I have an RTX5070ti with its mere 16gb vRAM but it’s at least DDR7 and a 256-bit memory bus. (yeah, oops - the 5090 w/32gb vra… --- ## [Win11 wsl2 在 AMD RYZEN AI MAX+ 395](https://discuss.vllm.ai/t/win11-wsl2-amd-ryzen-ai-max-395/2752)
**Author:** [@kuolung1](https://discuss.vllm.ai/u/kuolung1)\ **Replies:** 1\ **Last updated:** [July 3, 2026, 1:10am UTC](https://discuss.vllm.ai/t/win11-wsl2-amd-ryzen-ai-max-395/2752 "2026-07-03T01:10:23Z")
在以上的環境中， 無法正常跑 DeepSeek ocr 2 我想掛 ROCm 使用8060s GPU --- ## [Vllm-ascend怎么支持responses](https://discuss.vllm.ai/t/vllm-ascend-responses/2666)
**Author:** [@h171505612](https://discuss.vllm.ai/u/h171505612)\ **Replies:** 1\ **Last updated:** [May 27, 2026, 1:52pm UTC](https://discuss.vllm.ai/t/vllm-ascend-responses/2666 "2026-05-27T13:52:31Z")
目前codex硬性要求用responses，我在Atlas 800T A2上部署了qwen3.6-27b模型 （vllm-ascendv0.19）正常api调用已经可以使用，但是responses会报错，这个怎么解决 --- ## [VGPU on podman "No CUDA GPUs are available"](https://discuss.vllm.ai/t/vgpu-on-podman-no-cuda-gpus-are-available/2653)
**Author:** [@jerome](https://discuss.vllm.ai/u/jerome)\ **Replies:** 0\ **Last updated:** [May 23, 2026, 6:19am UTC](https://discuss.vllm.ai/t/vgpu-on-podman-no-cuda-gpus-are-available/2653 "2026-05-23T06:19:29Z")
I have an NVIDIA L40S with: Driver 580.95.05 + CUDA 13.0 nvidia-smi works perfectly Torch sees the GPU without any problems (torch.cuda.is\_available() == True) However, all modern inference engines fail with the erro… --- ## [最新开源的Qwen3.6的moe模型，vllm-ascend支持吗？](https://discuss.vllm.ai/t/qwen3-6-moe-vllm-ascend/2561)
**Author:** [@huangcheng1x](https://discuss.vllm.ai/u/huangcheng1x)\ **Replies:** 1\ **Last updated:** [April 17, 2026, 6:13am UTC](https://discuss.vllm.ai/t/qwen3-6-moe-vllm-ascend/2561 "2026-04-17T06:13:52Z")
如题，是否用最新的18.cr1版本是否可以？或者还有更新的版本？ --- ## [vLLM hangs during worker initialization on Blackwell PCIe GPUs unless --disable-custom-all-reduce is used](https://discuss.vllm.ai/t/vllm-hangs-during-worker-initialization-on-blackwell-pcie-gpus-unless-disable-custom-all-reduce-is-used/2540)
**Author:** [@zhouyulin11929](https://discuss.vllm.ai/u/zhouyulin11929)\ **Replies:** 1\ **Last updated:** [April 11, 2026, 2:33am UTC](https://discuss.vllm.ai/t/vllm-hangs-during-worker-initialization-on-blackwell-pcie-gpus-unless-disable-custom-all-reduce-is-used/2540 "2026-04-11T02:33:01Z")
Description When deploying a large model with tensor parallelism on a multi-GPU server, vLLM hangs during worker initialization. The logs repeatedly show: \[v1/engine/utils.py:1000\] Waiting for 1 local, 0 remote core en… --- ## [\# SM120 (RTX PRO 6000) NVFP4 MoE Performance Report -- Qwen3.5-397B](https://discuss.vllm.ai/t/sm120-rtx-pro-6000-nvfp4-moe-performance-report-qwen3-5-397b/2536)
**Author:** [@brandonmmusic-max](https://discuss.vllm.ai/u/brandonmmusic-max)\ **Replies:** 1\ **Last updated:** [April 11, 2026, 2:15am UTC](https://discuss.vllm.ai/t/sm120-rtx-pro-6000-nvfp4-moe-performance-report-qwen3-5-397b/2536 "2026-04-11T02:15:49Z")
\# SM120 (RTX PRO 6000) NVFP4 MoE Performance Report – Qwen3.5-397B ## TL;DR Best sustained single-user decode on \`nvidia/Qwen3.5-397B-A17B-NVFP4\` with 4x RTX PRO 6000 (SM120): \*\*50.5 tok/s\*\* using Marlin W4A16 fallback… --- ## [Vllm启动时，日志卡在nccl相关部分，不继续往下](https://discuss.vllm.ai/t/vllm-nccl/1445)
**Author:** [@XiaoDouGeGe](https://discuss.vllm.ai/u/XiaoDouGeGe)\ **Replies:** 16\ **Last updated:** [April 8, 2026, 10:28am UTC](https://discuss.vllm.ai/t/vllm-nccl/1445 "2026-04-08T10:28:53Z")
使用4张L20启动vllm模型服务 nvidia-smi Wed Aug 20 16:45:37 2025 ±----------------------------------------------------------------------------------------+ | NVIDIA-SMI 570.172.08 Driver Version: 570.172.08 CUDA … --- ## [SM120 (RTX PRO 4000): 6.5x throughput gain and v0.18.1 regression findings](https://discuss.vllm.ai/t/sm120-rtx-pro-4000-6-5x-throughput-gain-and-v0-18-1-regression-findings/2525)
**Author:** [@Guiss-Guiss](https://discuss.vllm.ai/u/Guiss-Guiss)\ **Replies:** 1\ **Last updated:** [April 3, 2026, 7:33pm UTC](https://discuss.vllm.ai/t/sm120-rtx-pro-4000-6-5x-throughput-gain-and-v0-18-1-regression-findings/2525 "2026-04-03T19:33:48Z")
I spent a full day optimizing vLLM on 2x RTX PRO 4000 Blackwell (SM120, 24GB GDDR7 each) and want to share what I found because some of this was really painful to discover. Setup: driver 580.126.20, CUDA 13.0, Ubuntu 24… --- ## [Mixed GPU support?](https://discuss.vllm.ai/t/mixed-gpu-support/2506)
**Author:** [@kstine](https://discuss.vllm.ai/u/kstine)\ **Replies:** 1\ **Last updated:** [March 31, 2026, 6:22pm UTC](https://discuss.vllm.ai/t/mixed-gpu-support/2506 "2026-03-31T18:22:23Z")
Is it possible to run inference across GPUs from multiple vendors? We have GPUs from multiple different vendors, but all with 32GB of VRAM. Being able to pool them together would be ideal for running larger models. --- ## [MoE config on GH200](https://discuss.vllm.ai/t/moe-config-on-gh200/1718)
**Author:** [@tomschelsen](https://discuss.vllm.ai/u/tomschelsen)\ **Replies:** 9\ **Last updated:** [February 4, 2026, 10:02am UTC](https://discuss.vllm.ai/t/moe-config-on-gh200/1718 "2026-02-04T10:02:05Z")
Running Qwen/Qwen3-Coder-30B-A3B-Instruct on an Nvidia GH200, I get the following warning : WARNING 10-10 07:30:42 \[fused\_moe.py:798\] Using default MoE config. Performance might be sub-optimal! Config file not found at… --- ## [Running NVFP4 Nemotron model on Win11/WSL RTX 5080 + 5070 Ti](https://discuss.vllm.ai/t/running-nvfp4-nemotron-model-on-win11-wsl-rtx-5080-5070-ti/2310)
**Author:** [@trevor](https://discuss.vllm.ai/u/trevor)\ **Replies:** 2\ **Last updated:** [February 2, 2026, 4:00pm UTC](https://discuss.vllm.ai/t/running-nvfp4-nemotron-model-on-win11-wsl-rtx-5080-5070-ti/2310 "2026-02-02T16:00:39Z")
Hello, I am trying to run the NVIDIA Nemotron 3 Nano NVFP4 model across 2x Blackwell GPUs installed in my system. The model appears to be only about 20 GB in size, so it ought to work with my dual-GPU configuration with … --- ## [Vllm-ascend跑量化qwen2.5\_7b问题](https://discuss.vllm.ai/t/vllm-ascend-qwen2-5-7b/2300)
**Author:** [@lihao458](https://discuss.vllm.ai/u/lihao458)\ **Replies:** 1\ **Last updated:** [February 2, 2026, 6:31am UTC](https://discuss.vllm.ai/t/vllm-ascend-qwen2-5-7b/2300 "2026-02-02T06:31:09Z")
qwen2.5 -7b量化模型，用vllm-ascend跑报错KeyError: 'model.embed\_tokens.weight'。在h800+vllm上跑正常，配置如下 llm\_model = LLM( model="xxxx/quantized\_model\_awq\_quant4", quantization="awq", dtype="float16", tensor\_parallel\_size… --- ## [vLLM on RTX5090: Working GPU setup with torch 2.9.0 cu128](https://discuss.vllm.ai/t/vllm-on-rtx5090-working-gpu-setup-with-torch-2-9-0-cu128/1492)
**Author:** [@KieranBicheno](https://discuss.vllm.ai/u/KieranBicheno)\ **Replies:** 18\ **Last updated:** [January 13, 2026, 7:47am UTC](https://discuss.vllm.ai/t/vllm-on-rtx5090-working-gpu-setup-with-torch-2-9-0-cu128/1492 "2026-01-13T07:47:35Z")
vLLM on RTX 5090 Blackwell: A Technical Deep Dive Hi folks, I spent most of today trying to get vLLM running with PyTorch 2.9.0 and it looks like the most recent build takes care of a lot of errors. There are so many wa… --- ## [Support for RTX 6000 Blackwell 96GB card](https://discuss.vllm.ai/t/support-for-rtx-6000-blackwell-96gb-card/1707)
**Author:** [@ThommyB](https://discuss.vllm.ai/u/ThommyB)\ **Replies:** 5\ **Last updated:** [January 5, 2026, 6:02pm UTC](https://discuss.vllm.ai/t/support-for-rtx-6000-blackwell-96gb-card/1707 "2026-01-05T18:02:18Z")
I ran a RTX4500 Nvidia Ada Type card with 24 GB vRAM. The memory is small but I could load several 7B or 8B models with FP8 or int4. Since I like to run several LLMs in on card I ordered a RTX 6000 Blackwell with 96 GB v… [Next page](https://discuss.vllm.ai/c/hardware-support/5.md?page=1)