RuntimeError: CUDA driver initialization failed, you might not have a CUDA gpu

wangxuasp · June 5, 2026, 12:52am

hardware： CPU：Hygon 7380 + GPU： Nvidia H800*3 + memory：512GB

software：Ubuntu 24+torch==2.11.0+cu130 + vllm==0.21.0

driver：nvidia-smi==Driverversion: 580.159.04 CUDA Version: 13.0

startup log：(Worker pid=122743) ERROR 05-31 14:55:08 [multiproc_executor.py:870] WorkerProc failed to start.
(Worker pid=122743) ERROR 05-31 14:55:08 [multiproc_executor.py:870] Traceback (most recent call last):
(Worker pid=122743) ERROR 05-31 14:55:08 [multiproc_executor.py:870] File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/v1/executor/multiproc_executor.py”, line 837, in worker_main
(Worker pid=122743) ERROR 05-31 14:55:08 [multiproc_executor.py:870] worker = WorkerProc(*args, **kwargs)
(Worker pid=122743) ERROR 05-31 14:55:08 [multiproc_executor.py:870] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=122743) ERROR 05-31 14:55:08 [multiproc_executor.py:870] File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/tracing/otel.py”, line 178, in sync_wrapper
(Worker pid=122743) ERROR 05-31 14:55:08 [multiproc_executor.py:870] return func(*args, **kwargs)
(Worker pid=122743) ERROR 05-31 14:55:08 [multiproc_executor.py:870] ^^^^^^^^^^^^^^^^^^^^^
(Worker pid=122743) ERROR 05-31 14:55:08 [multiproc_executor.py:870] File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/v1/executor/multiproc_executor.py”, line 611, in init
(Worker pid=122743) ERROR 05-31 14:55:08 [multiproc_executor.py:870] self.worker.init_device()
(Worker pid=122743) ERROR 05-31 14:55:08 [multiproc_executor.py:870] File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/v1/worker/worker_base.py”, line 317, in init_device
(Worker pid=122743) ERROR 05-31 14:55:08 [multiproc_executor.py:870] self.worker.init_device() # type: ignore
(Worker pid=122743) ERROR 05-31 14:55:08 [multiproc_executor.py:870] ^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=122743) ERROR 05-31 14:55:08 [multiproc_executor.py:870] File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/tracing/otel.py”, line 178, in sync_wrapper
(Worker pid=122743) ERROR 05-31 14:55:08 [multiproc_executor.py:870] return func(*args, **kwargs)
(Worker pid=122743) ERROR 05-31 14:55:08 [multiproc_executor.py:870] ^^^^^^^^^^^^^^^^^^^^^
(Worker pid=122743) ERROR 05-31 14:55:08 [multiproc_executor.py:870] File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/v1/worker/gpu_worker.py”, line 275, in init_device
(Worker pid=122743) ERROR 05-31 14:55:08 [multiproc_executor.py:870] torch.accelerator.set_device_index(self.device)
(Worker pid=122743) ERROR 05-31 14:55:08 [multiproc_executor.py:870] File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/torch/accelerator/init.py”, line 191, in set_device_index
(Worker pid=122743) ERROR 05-31 14:55:08 [multiproc_executor.py:870] torch._C._accelerator_setDeviceIndex(device_index)
(Worker pid=122743) ERROR 05-31 14:55:08 [multiproc_executor.py:870] File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/torch/cuda/init.py”, line 478, in _lazy_init
(Worker pid=122743) ERROR 05-31 14:55:08 [multiproc_executor.py:870] torch._C._cuda_init()
(Worker pid=122743) ERROR 05-31 14:55:08 [multiproc_executor.py:870] RuntimeError: CUDA driver initialization failed, you might not have a CUDA gpu.
(Worker pid=122742) ERROR 05-31 14:55:08 [multiproc_executor.py:870] WorkerProc failed to start.
(Worker pid=122742) ERROR 05-31 14:55:08 [multiproc_executor.py:870] Traceback (most recent call last):
(Worker pid=122742) ERROR 05-31 14:55:08 [multiproc_executor.py:870] File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/v1/executor/multiproc_executor.py”, line 837, in worker_main
(Worker pid=122742) ERROR 05-31 14:55:08 [multiproc_executor.py:870] worker = WorkerProc(*args, **kwargs)
(Worker pid=122742) ERROR 05-31 14:55:08 [multiproc_executor.py:870] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=122742) ERROR 05-31 14:55:08 [multiproc_executor.py:870] File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/tracing/otel.py”, line 178, in sync_wrapper
(Worker pid=122742) ERROR 05-31 14:55:08 [multiproc_executor.py:870] return func(*args, **kwargs)
(Worker pid=122742) ERROR 05-31 14:55:08 [multiproc_executor.py:870] ^^^^^^^^^^^^^^^^^^^^^
(Worker pid=122742) ERROR 05-31 14:55:08 [multiproc_executor.py:870] File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/v1/executor/multiproc_executor.py”, line 611, in init
(Worker pid=122742) ERROR 05-31 14:55:08 [multiproc_executor.py:870] self.worker.init_device()
(Worker pid=122742) ERROR 05-31 14:55:08 [multiproc_executor.py:870] File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/v1/worker/worker_base.py”, line 317, in init_device
(Worker pid=122742) ERROR 05-31 14:55:08 [multiproc_executor.py:870] self.worker.init_device() # type: ignore
(Worker pid=122742) ERROR 05-31 14:55:08 [multiproc_executor.py:870] ^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=122742) ERROR 05-31 14:55:08 [multiproc_executor.py:870] File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/tracing/otel.py”, line 178, in sync_wrapper
(Worker pid=122742) ERROR 05-31 14:55:08 [multiproc_executor.py:870] return func(*args, **kwargs)
(Worker pid=122742) ERROR 05-31 14:55:08 [multiproc_executor.py:870] ^^^^^^^^^^^^^^^^^^^^^
(Worker pid=122742) ERROR 05-31 14:55:08 [multiproc_executor.py:870] File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/v1/worker/gpu_worker.py”, line 275, in init_device
(Worker pid=122742) ERROR 05-31 14:55:08 [multiproc_executor.py:870] torch.accelerator.set_device_index(self.device)
(Worker pid=122742) ERROR 05-31 14:55:08 [multiproc_executor.py:870] File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/torch/accelerator/init.py”, line 191, in set_device_index
(Worker pid=122742) ERROR 05-31 14:55:08 [multiproc_executor.py:870] torch._C._accelerator_setDeviceIndex(device_index)
(Worker pid=122742) ERROR 05-31 14:55:08 [multiproc_executor.py:870] File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/torch/cuda/init.py”, line 478, in _lazy_init
(Worker pid=122742) ERROR 05-31 14:55:08 [multiproc_executor.py:870] torch._C._cuda_init()
(Worker pid=122742) ERROR 05-31 14:55:08 [multiproc_executor.py:870] RuntimeError: CUDA driver initialization failed, you might not have a CUDA gpu.
(EngineCore pid=122543) ERROR 05-31 14:55:10 [core.py:1140] EngineCore failed to start.
(EngineCore pid=122543) ERROR 05-31 14:55:10 [core.py:1140] Traceback (most recent call last):
(EngineCore pid=122543) ERROR 05-31 14:55:10 [core.py:1140] File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/v1/engine/core.py”, line 1114, in run_engine_core
(EngineCore pid=122543) ERROR 05-31 14:55:10 [core.py:1140] engine_core = EngineCoreProc(*args, engine_index=dp_rank, **kwargs)
(EngineCore pid=122543) ERROR 05-31 14:55:10 [core.py:1140] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=122543) ERROR 05-31 14:55:10 [core.py:1140] File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/tracing/otel.py”, line 178, in sync_wrapper
(EngineCore pid=122543) ERROR 05-31 14:55:10 [core.py:1140] return func(*args, **kwargs)
(EngineCore pid=122543) ERROR 05-31 14:55:10 [core.py:1140] ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=122543) ERROR 05-31 14:55:10 [core.py:1140] File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/v1/engine/core.py”, line 880, in init
(EngineCore pid=122543) ERROR 05-31 14:55:10 [core.py:1140] super().init(
(EngineCore pid=122543) ERROR 05-31 14:55:10 [core.py:1140] File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/v1/engine/core.py”, line 118, in init
(EngineCore pid=122543) ERROR 05-31 14:55:10 [core.py:1140] self.model_executor = executor_class(vllm_config)
(EngineCore pid=122543) ERROR 05-31 14:55:10 [core.py:1140] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=122543) ERROR 05-31 14:55:10 [core.py:1140] File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/v1/executor/multiproc_executor.py”, line 107, in init
(EngineCore pid=122543) ERROR 05-31 14:55:10 [core.py:1140] super().init(vllm_config)
(EngineCore pid=122543) ERROR 05-31 14:55:10 [core.py:1140] File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/tracing/otel.py”, line 178, in sync_wrapper
(EngineCore pid=122543) ERROR 05-31 14:55:10 [core.py:1140] return func(*args, **kwargs)
(EngineCore pid=122543) ERROR 05-31 14:55:10 [core.py:1140] ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=122543) ERROR 05-31 14:55:10 [core.py:1140] File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/v1/executor/abstract.py”, line 109, in init
(EngineCore pid=122543) ERROR 05-31 14:55:10 [core.py:1140] self._init_executor()
(EngineCore pid=122543) ERROR 05-31 14:55:10 [core.py:1140] File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/v1/executor/multiproc_executor.py”, line 200, in _init_executor
(EngineCore pid=122543) ERROR 05-31 14:55:10 [core.py:1140] self.workers = WorkerProc.wait_for_ready(unready_workers)
(EngineCore pid=122543) ERROR 05-31 14:55:10 [core.py:1140] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=122543) ERROR 05-31 14:55:10 [core.py:1140] File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/v1/executor/multiproc_executor.py”, line 747, in wait_for_ready
(EngineCore pid=122543) ERROR 05-31 14:55:10 [core.py:1140] raise e from None
(EngineCore pid=122543) ERROR 05-31 14:55:10 [core.py:1140] Exception: WorkerProc initialization failed due to an exception in a background process. See stack trace for root cause.
(EngineCore pid=122543) Process EngineCore:
(EngineCore pid=122543) Traceback (most recent call last):
(EngineCore pid=122543) File “/usr/lib/python3.12/multiprocessing/process.py”, line 314, in _bootstrap
(EngineCore pid=122543) self.run()
(EngineCore pid=122543) File “/usr/lib/python3.12/multiprocessing/process.py”, line 108, in run
(EngineCore pid=122543) self._target(*self._args, **self._kwargs)
(EngineCore pid=122543) File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/v1/engine/core.py”, line 1144, in run_engine_core
(EngineCore pid=122543) raise e
(EngineCore pid=122543) File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/v1/engine/core.py”, line 1114, in run_engine_core
(EngineCore pid=122543) engine_core = EngineCoreProc(*args, engine_index=dp_rank, **kwargs)
(EngineCore pid=122543) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=122543) File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/tracing/otel.py”, line 178, in sync_wrapper
(EngineCore pid=122543) return func(*args, **kwargs)
(EngineCore pid=122543) ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=122543) File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/v1/engine/core.py”, line 880, in init
(EngineCore pid=122543) super().init(
(EngineCore pid=122543) File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/v1/engine/core.py”, line 118, in init
(EngineCore pid=122543) self.model_executor = executor_class(vllm_config)
(EngineCore pid=122543) ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=122543) File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/v1/executor/multiproc_executor.py”, line 107, in init
(EngineCore pid=122543) super().init(vllm_config)
(EngineCore pid=122543) File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/tracing/otel.py”, line 178, in sync_wrapper
(EngineCore pid=122543) return func(*args, **kwargs)
(EngineCore pid=122543) ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=122543) File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/v1/executor/abstract.py”, line 109, in init
(EngineCore pid=122543) self._init_executor()
(EngineCore pid=122543) File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/v1/executor/multiproc_executor.py”, line 200, in _init_executor
(EngineCore pid=122543) self.workers = WorkerProc.wait_for_ready(unready_workers)
(EngineCore pid=122543) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=122543) File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/v1/executor/multiproc_executor.py”, line 747, in wait_for_ready
(EngineCore pid=122543) raise e from None
(EngineCore pid=122543) Exception: WorkerProc initialization failed due to an exception in a background process. See stack trace for root cause.
(APIServer pid=122183) Traceback (most recent call last):
(APIServer pid=122183) File “/home/aiuser/llm_workspace/venv/bin/vllm”, line 10, in
(APIServer pid=122183) sys.exit(main())
(APIServer pid=122183) ^^^^^^
(APIServer pid=122183) File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/entrypoints/cli/main.py”, line 92, in main
(APIServer pid=122183) args.dispatch_function(args)
(APIServer pid=122183) File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/entrypoints/cli/serve.py”, line 122, in cmd
(APIServer pid=122183) uvloop.run(run_server(args))
(APIServer pid=122183) File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/uvloop/init.py”, line 96, in run
(APIServer pid=122183) return __asyncio.run(
(APIServer pid=122183) ^^^^^^^^^^^^^^
(APIServer pid=122183) File “/usr/lib/python3.12/asyncio/runners.py”, line 194, in run
(APIServer pid=122183) return runner.run(main)
(APIServer pid=122183) ^^^^^^^^^^^^^^^^
(APIServer pid=122183) File “/usr/lib/python3.12/asyncio/runners.py”, line 118, in run
(APIServer pid=122183) return self._loop.run_until_complete(task)
(APIServer pid=122183) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=122183) File “uvloop/loop.pyx”, line 1518, in uvloop.loop.Loop.run_until_complete
(APIServer pid=122183) File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/uvloop/init.py”, line 48, in wrapper
(APIServer pid=122183) return await main
(APIServer pid=122183) ^^^^^^^^^^
(APIServer pid=122183) File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py”, line 693, in run_server
(APIServer pid=122183) await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
(APIServer pid=122183) File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py”, line 707, in run_server_worker
(APIServer pid=122183) async with build_async_engine_client(
(APIServer pid=122183) File “/usr/lib/python3.12/contextlib.py”, line 210, in aenter
(APIServer pid=122183) return await anext(self.gen)
(APIServer pid=122183) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=122183) File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py”, line 100, in build_async_engine_client
(APIServer pid=122183) async with build_async_engine_client_from_engine_args(
(APIServer pid=122183) File “/usr/lib/python3.12/contextlib.py”, line 210, in aenter
(APIServer pid=122183) return await anext(self.gen)
(APIServer pid=122183) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=122183) File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py”, line 136, in build_async_engine_client_from_engine_args
(APIServer pid=122183) async_llm = AsyncLLM.from_vllm_config(
(APIServer pid=122183) ^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=122183) File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/v1/engine/async_llm.py”, line 217, in from_vllm_config
(APIServer pid=122183) return cls(
(APIServer pid=122183) ^^^^
(APIServer pid=122183) File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/v1/engine/async_llm.py”, line 146, in init
(APIServer pid=122183) self.engine_core = EngineCoreClient.make_async_mp_client(
(APIServer pid=122183) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=122183) File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/tracing/otel.py”, line 178, in sync_wrapper
(APIServer pid=122183) return func(*args, **kwargs)
(APIServer pid=122183) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=122183) File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/v1/engine/core_client.py”, line 130, in make_async_mp_client
(APIServer pid=122183) return AsyncMPClient(*client_args)
(APIServer pid=122183) ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=122183) File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/tracing/otel.py”, line 178, in sync_wrapper
(APIServer pid=122183) return func(*args, **kwargs)
(APIServer pid=122183) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=122183) File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/v1/engine/core_client.py”, line 900, in init
(APIServer pid=122183) super().init(
(APIServer pid=122183) File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/v1/engine/core_client.py”, line 535, in init
(APIServer pid=122183) with launch_core_engines(
(APIServer pid=122183) File “/usr/lib/python3.12/contextlib.py”, line 144, in exit
(APIServer pid=122183) next(self.gen)
(APIServer pid=122183) File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/v1/engine/utils.py”, line 1128, in launch_core_engines
(APIServer pid=122183) wait_for_engine_startup(
(APIServer pid=122183) File “/home/aiuser/llm_workspace/venv/lib/python3.12/site-packages/vllm/v1/engine/utils.py”, line 1187, in

RunLLM · June 5, 2026, 12:52am

你的报错“RuntimeError: CUDA driver initialization failed, you might not have a CUDA gpu.”说明PyTorch无法检测到CUDA GPU。根据vLLM官方文档和issue，这通常是环境配置或驱动问题导致的。请确认以下几点：

检查nvidia-smi能否在你的虚拟环境下正常运行，且能看到H800显卡；2. 确认你的PyTorch和CUDA驱动版本完全兼容，vLLM 0.21.0官方推荐CUDA 12.9/13.0和对应PyTorch（如2.11.0+cu130），但驱动需为580.105.08及以上，且必须用官方pip/uv wheel安装，不建议用conda安装PyTorch，否则NCCL等库可能冲突；3. 若用Docker，需加–gpus all参数，并确保容器内nvidia-smi可用；4. 若用裸机，建议新建干净的venv/uv环境，重新pip安装vllm和torch，避免老环境残留影响；5. 若驱动升级后仍报错，可尝试重启服务器或重装驱动。

详细排查步骤和环境变量设置可参考：vLLM官方安装文档和 troubleshooting指南。你需要确保所有依赖都在同一CUDA版本下编译和运行。

需要更详细的排查步骤吗？

Sources:

Hint: Mention @RunLLM in the post for followups.

Topic		Replies	Views
求救各位大佬看看是什么问题。cuda12.9，pytorch2.8，vllm0.11.0 General	3	247	November 14, 2025
How can we use latest vllm if we are using older drivers which only support cuda 12 General	3	92	May 27, 2026
Why is this not working? I corrected it but still NVIDIA GPU Support	1	926	May 8, 2025
RTX PRO 6000 users seek help, LLAMA 4 NVFP4 NVIDIA GPU Support	1	307	November 25, 2025
Can anyone help me? Why is this not working? It used 😭 NVIDIA GPU Support	1	1226	May 8, 2025

RuntimeError: CUDA driver initialization failed, you might not have a CUDA gpu

Related topics