# 多机多卡推理 ray vllm遇到的报错 **URL:** \ **Category:** General\ **Created:** [January 23, 2026, 7:03am UTC](https://discuss.vllm.ai/t/ray-vllm/2267 "2026-01-23T07:03:58Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![XiaoDouGeGe](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/xiaodougege/32/382_2.png) [@XiaoDouGeGe](https://discuss.vllm.ai/u/XiaoDouGeGe)\ **Post date:** [January 23, 2026, 7:03am UTC](https://discuss.vllm.ai/t/ray-vllm/2267/1 "2026-01-23T07:03:58Z")
2卡16节点（每个节点8张卡），启动Qwen3-235B-A22B的推理， 集群为： ```auto root@root-1-241:/data/llm/app/Qwen3-235B-A22B# ray status ======== Autoscaler status: 2026-01-23 15:03:17.631346 ======== Node status --------------------------------------------------------------- Active: 1 node_60b782f567508b2a8c54c54cb54bb6f6c4e68e62cbe366664003069e 1 node_11657f380647113b616b9b934e410a3daab898575243e4c96a7a02cf Pending: (no pending nodes) Recent failures: (no failures) Resources --------------------------------------------------------------- Total Usage: 0.0/256.0 CPU 0.0/16.0 GPU 0B/1.78TiB memory 0B/190.00GiB object_store_memory Total Constraints: (no request_resources() constraints) Total Demands: (no resource demands) root@root-1-241:/data/llm/app/Qwen3-235B-A22B# ``` 服务命令为： ```auto nohup vllm serve /data/llm/models/Qwen3-235B-A22B -pp 2 -tp 8 --trust-remote-code --distributed-executor-backend ray --dtype bfloat16 --swap-space 16 --gpu-memory-utilization 0.8 --host 0.0.0.0 --port 11235 --max-model-len 20000 --reasoning-parser qwen3 --tool-call-parser hermes --enable-auto-tool-choice > vllm-$(date +%Y%m%d%H%M).log 2>&1 & ``` 遇到报错，如下： > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] EngineCore failed to start. > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] Traceback (most recent call last): > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] File “/opt/conda/lib/python3.12/site-packages/vllm/v1/engine/core.py”, line 833, in run\_engine\_core > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] engine\_core = EngineCoreProc(\*args, \*\*kwargs) > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] File “/opt/conda/lib/python3.12/site-packages/vllm/v1/engine/core.py”, line 606, in \__init_\_ > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] super().\__init_\_( > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] File “/opt/conda/lib/python3.12/site-packages/vllm/v1/engine/core.py”, line 102, in \__init_\_ > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] self.model\_executor = executor\_class(vllm\_config) > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] ^^^^^^^^^^^^^^^^^^^^^^^^^^^ > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] File “/opt/conda/lib/python3.12/site-packages/vllm/v1/executor/abstract.py”, line 101, in \__init_\_ > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] self.\_init\_executor() > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] File “/opt/conda/lib/python3.12/site-packages/vllm/v1/executor/ray\_executor.py”, line 97, in \_init\_executor > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] self.\_init\_workers\_ray(placement\_group) > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] File “/opt/conda/lib/python3.12/site-packages/vllm/v1/executor/ray\_executor.py”, line 370, in \_init\_workers\_ray > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] self.collective\_rpc(“init\_device”) > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] File “/opt/conda/lib/python3.12/site-packages/vllm/v1/executor/ray\_executor.py”, line 493, in collective\_rpc > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] return ray.get(ray\_worker\_outputs, timeout=timeout) > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] File “/opt/conda/lib/python3.12/site-packages/ray/\_private/auto\_init\_hook.py”, line 22, in auto\_init\_wrapper > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] return fn(\*args, \*\*kwargs) > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] ^^^^^^^^^^^^^^^^^^^ > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] File “/opt/conda/lib/python3.12/site-packages/ray/\_private/client\_mode\_hook.py”, line 104, in wrapper > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] return func(\*args, \*\*kwargs) > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] ^^^^^^^^^^^^^^^^^^^^^ > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] File “/opt/conda/lib/python3.12/site-packages/ray/\_private/worker.py”, line 2858, in get > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] values, debugger\_breakpoint = worker.get\_objects(object\_refs, timeout=timeout) > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] File “/opt/conda/lib/python3.12/site-packages/ray/\_private/worker.py”, line 958, in get\_objects > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] raise value.as\_instanceof\_cause() > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] ray.exceptions.RayTaskError(AcceleratorError): ray::RayWorkerWrapper.execute\_method() (pid=974, ip=192.168.205.2 > 5, actor\_id=d3849d9ac13f4515a54d8f8502000000, repr=\) > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] File “/opt/conda/lib/python3.12/site-packages/vllm/v1/worker/worker\_base.py”, line 343, in execute\_method > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] raise e > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] File “/opt/conda/lib/python3.12/site-packages/vllm/v1/worker/worker\_base.py”, line 332, in execute\_method > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] return run\_method(self, method, args, kwargs) > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] File “/opt/conda/lib/python3.12/site-packages/vllm/v1/serial\_utils.py”, line 479, in run\_method > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] return func(\*args, \*\*kwargs) > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] ^^^^^^^^^^^^^^^^^^^^^ > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] File “/opt/conda/lib/python3.12/site-packages/vllm/v1/worker/worker\_base.py”, line 324, in init\_device > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] self.worker.init\_device() # type: ignore > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] ^^^^^^^^^^^^^^^^^^^^^^^^^ > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] File “/opt/conda/lib/python3.12/site-packages/vllm/v1/worker/gpu\_worker.py”, line 216, in init\_device > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] current\_platform.set\_device(self.device) > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] File “/opt/conda/lib/python3.12/site-packages/vllm\_metax/platform.py”, line 148, in set\_device > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] torch.cuda.set\_device(device) > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] File “/opt/conda/lib/python3.12/site-packages/torch/cuda/\__init_\_.py”, line 570, in set\_device > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] torch.\_C.\_cuda\_setDevice(device) > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] torch.AcceleratorError: CUDA error: invalid device ordinal > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] For debugging consider passing CUDA\_LAUNCH\_BLOCKING=1 > (EngineCore\_DP0 pid=9433) ERROR 01-23 14:46:01 [core.py:842] Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions. ---
**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\ **Post date:** [January 23, 2026, 7:04am UTC](https://discuss.vllm.ai/t/ray-vllm/2267/2 "2026-01-23T07:04:44Z")
你的报错“CUDA error: invalid device ordinal”说明vLLM在分配GPU时找不到对应编号的设备，常见于Ray集群多节点/多卡环境下设备分配不一致或资源未正确暴露。根据[官方文档](https://docs.vllm.ai/en/stable/serving/distributed_serving.html#running-vllm-on-multiple-nodes)和[相关issue](https://github.com/vllm-project/vllm/issues/29265)，常见原因有： 1. Ray集群节点的GPU数量与vLLM参数（如–tensor-parallel-size、–pipeline-parallel-size）不匹配，或某节点未正确分配GPU资源。 2. 各节点的CUDA\_VISIBLE\_DEVICES未正确设置，或docker未加–gpus all参数，导致vLLM进程无法访问全部GPU。 3. Ray的placement group未能正确分配GPU，或当前节点没有可用GPU，详见[源码实现](https://github.com/vllm-project/vllm/blob/main/vllm/v1/executor/ray_utils.py)和[API文档](https://docs.vllm.ai/en/latest/api/vllm/#vllm.initialize_ray_cluster)。 建议： - 检查每个节点nvidia-smi，确认8卡都可用，且Ray status显示所有GPU都已注册。 - 确认docker run加了–gpus all --ipc=host --shm-size=16G等参数。 - 检查CUDA\_VISIBLE\_DEVICES环境变量，确保每个节点都暴露了全部GPU。 - 若用Ray，建议每个节点都用相同的vllm serve命令，且–tensor-parallel-size与–pipeline-parallel-size参数总和等于集群总GPU数。 - 可尝试减少并行参数（如tp/pp），排查是否为资源分配问题。 需要更详细排查步骤吗？ Sources: - [https://docs.vllm.ai/en/stable/serving/distributed\_serving.html#running-vllm-on-multiple-nodes](https://docs.vllm.ai/en/stable/serving/distributed_serving.html#running-vllm-on-multiple-nodes) - [https://github.com/vllm-project/vllm/issues/29265](https://github.com/vllm-project/vllm/issues/29265) - [源码实现](https://github.com/vllm-project/vllm/blob/main/vllm/v1/executor/ray_utils.py) - [API文档](https://docs.vllm.ai/en/latest/api/vllm/#vllm.initialize_ray_cluster) Hint: Mention @RunLLM in the post for followups.