# Question about vllm-ascend performance on server with 8\*910B3
**URL:** https://discuss.vllm.ai/t/question-about-vllm-ascend-performance-on-server-with-8-910b3/712
**Category:** Ascend Support
**Created:** [May 28, 2025, 8:51am UTC](https://discuss.vllm.ai/t/question-about-vllm-ascend-performance-on-server-with-8-910b3/712 "2025-05-28T08:51:25Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@kyle](https://discuss.vllm.ai/u/kyle)
#### Post date: [May 28, 2025, 8:51am UTC](https://discuss.vllm.ai/t/question-about-vllm-ascend-performance-on-server-with-8-910b3/712/1 "2025-05-28T08:51:25Z")
**Environment Information is as follows:**
OS: Kylin10
Ascend NPU Driver: Ascend-hdk-910b-npu-driver\_23.0.7\_linux-aarch64.run
Ascend NPU Firmware: Ascend-hdk-910b-npu-firmware\_7.1.0.11.220.run
Ascend Docker Runtime: Ascend-docker-runtime\_5.0.RC3.2\_linux-x86\_64.run
Docker: docker-ce-26.1.3-1.el8.aarch64.rpm
Containerd: containerd.io-1.6.32-3.1.el8.aarch64.rpm
vllm-ascend: vllm-ascend-v0.8.5rc1
**Run 32B model as follows:**
export [IMAGE=quay.io/ascend/vllm-ascend:v0.8.5rc1](http://IMAGE=quay.io/ascend/vllm-ascend:v0.8.5rc1)
docker run --rm
–name vllm-ascend-env
–device /dev/davinci0
–device /dev/davinci1
–device /dev/davinci2
–device /dev/davinci3
–device /dev/davinci\_manager
–device /dev/devmm\_svm
–device /dev/hisi\_hdc
-v /usr/local/dcmi:/usr/local/dcmi
-v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi
-v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/
-v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info
-v /etc/ascend\_install.info:/etc/ascend\_install.info
-v /root/.cache:/root/.cache
-p 8000:8000
-v /home/test:/mnt
-e VLLM\_USE\_MODELSCOPE=True
-e PYTORCH\_NPU\_ALLOC\_CONF=max\_split\_size\_mb:256
-it $IMAGE
vllm serve /mnt/Qwen3-32B --tensor-parallel-size 4 --max-model-len 16384 --gpu-memory-utilization 0.85
**API request result shows the processing speed is only 10 tokens/s:**
**Question:**
Is the above performance normal? How can I improve it?
Because Running 32B model in Nvidia A10 based on vllm 0.8.5rc1 can achieve 25 tokens/s, 910B3 should not be worse than A10 as far as I know.
---
### Author:  [@RunLLM](https://discuss.vllm.ai/u/RunLLM)
#### Post date: [May 28, 2025, 8:52am UTC](https://discuss.vllm.ai/t/question-about-vllm-ascend-performance-on-server-with-8-910b3/712/2 "2025-05-28T08:52:02Z")
Based on the available data, 10 tokens/s for a Qwen3-32B model on Ascend 910B3 NPUs is lower than expected, especially since similar or better performance is reported on Nvidia A10 GPUs (25 tokens/s) with vLLM 0.8.5rc1. The vLLM-Ascend backend is still under active development, and performance tuning for Ascend NPUs may not be as mature as for Nvidia GPUs. There are no official benchmarks for 32B models on 910B3 in the provided sources, but users have reported that vLLM-Ascend performance is not yet on par with Nvidia GPUs and that multi-card (tensor parallel) support is still a work in progress ([vllm-ascend PR](https://github.com/vllm-project/vllm/pull/8054), [vllm-ascend repo](https://github.com/vllm-project/vllm-ascend)).
To potentially improve performance, ensure you are using the latest vllm-ascend code, and check for any updates or tuning guides in the [vllm-ascend repository](https://github.com/vllm-project/vllm-ascend). Also, verify that your environment variables and device bindings match the recommendations in the official documentation and PR discussions. If issues persist, consider opening an issue in the vllm-ascend repo with your environment details and logs for targeted support.
Would you like a step-by-step breakdown of possible tuning steps or troubleshooting for Ascend NPU performance?
Sources:
- [vllm-ascend PR](https://github.com/vllm-project/vllm/pull/8054)
---
### Author:  [@kyle](https://discuss.vllm.ai/u/kyle)
#### Post date: [May 28, 2025, 9:02am UTC](https://discuss.vllm.ai/t/question-about-vllm-ascend-performance-on-server-with-8-910b3/712/3 "2025-05-28T09:02:17Z")
Thanks for reply.
What is the normal performance for Qwen3-32B model on Ascned 910B3 NPUs? I would like some step-by-step tuning advices for improving the performance, thanks very much.
---
### Author:  [@RunLLM](https://discuss.vllm.ai/u/RunLLM)
#### Post date: [May 28, 2025, 9:02am UTC](https://discuss.vllm.ai/t/question-about-vllm-ascend-performance-on-server-with-8-910b3/712/4 "2025-05-28T09:02:55Z")
It seems likely that there are no official or community-published benchmarks for Qwen3-32B on Ascend 910B3 NPUs in the provided sources, so “normal” performance is not clearly defined. Most reports indicate that vLLM-Ascend is still catching up to Nvidia GPU performance, and multi-card scaling is under development ([vllm-ascend PR](https://github.com/vllm-project/vllm/pull/8054), [vllm-ascend repo](https://github.com/vllm-project/vllm-ascend)). Users have not shared specific tokens/s numbers for 910B3, but your observed 10 tokens/s is considered low compared to Nvidia A10 results.
Here are step-by-step tuning suggestions based on available documentation and user feedback:
1. **Update vllm-ascend** : Use the latest code from [vllm-ascend](https://github.com/vllm-project/vllm-ascend), as performance improvements are ongoing.
2. **Tensor Parallel Size** : Ensure `--tensor-parallel-size` matches your available NPUs and is supported by your hardware.
3. **Memory Utilization** : Adjust `--gpu-memory-utilization` (e.g., try 0.9 or 0.95) to maximize NPU memory usage.
4. **Batch Size** : Increase `--max-num-seqs` if memory allows, as higher batch sizes can improve throughput.
5. **Model Context Length** : Lower `--max-model-len` if you do not need long contexts, as this reduces memory pressure.
6. **Ascend Environment Variables** : Set `PYTORCH_NPU_ALLOC_CONF=max_split_size_mb:256` and other recommended variables from the [PR discussion](https://github.com/vllm-project/vllm/pull/8054).
7. **Profile and Monitor** : Use `npu-smi` and logs to check NPU utilization and memory usage.
8. **Report Issues** : If performance is still low, open an issue in the [vllm-ascend repo](https://github.com/vllm-project/vllm-ascend) with detailed logs and environment info for targeted help.
Would you like more detail on any of these steps?
Sources:
- [vllm-ascend PR](https://github.com/vllm-project/vllm/pull/8054)
---
### Author:  [@yikun](https://discuss.vllm.ai/u/yikun)
#### Post date: [May 28, 2025, 4:36pm UTC](https://discuss.vllm.ai/t/question-about-vllm-ascend-performance-on-server-with-8-910b3/712/5 "2025-05-28T16:36:10Z")
@kyle Thanks for your info, for qwen3
- For v0.7.3, we will add qwen3 support the v0.7.3.post1 [[release] 0.7.3.post1 release checklist · Issue #871 · vllm-project/vllm-ascend · GitHub](https://github.com/vllm-project/vllm-ascend/issues/871)
- For v0.8/0.9, We are also working on graph mode to optimize, here is also a guide: [[Guide]: Usage on Graph mode · Issue #767 · vllm-project/vllm-ascend · GitHub](https://github.com/vllm-project/vllm-ascend/issues/767) , it will be ready in next week.
---
### Author:  [@RunLLM](https://discuss.vllm.ai/u/RunLLM)
#### Post date: [May 28, 2025, 4:36pm UTC](https://discuss.vllm.ai/t/question-about-vllm-ascend-performance-on-server-with-8-910b3/712/6 "2025-05-28T16:36:42Z")
There are no official Qwen3-32B benchmarks for Ascend 910B3, but vLLM-Ascend v0.7.3-dev adds Qwen3 support and includes a [performance tuning guide](https://vllm-ascend.readthedocs.io/en/v0.7.3-dev/developer_guide/performance/optimization_and_tuning.html). For v0.8/0.9, graph mode optimizations are in progress and will be released soon ([issue #767](https://github.com/vllm-project/vllm-ascend/issues/767)). You can try the latest dev images for better performance.
Would you like a step-by-step summary of the optimization guide for vLLM-Ascend?
Sources:
- [performance tuning guide](https://vllm-ascend.readthedocs.io/en/v0.7.3-dev/developer_guide/performance/optimization_and_tuning.html)
- [issue #767](https://github.com/vllm-project/vllm-ascend/issues/767)