Issue when running vllm on the cpu

Hardware Support CPU Support
1

Hi, when running vllm on the cpu via uv run vllm serve Qwen/Qwen2.5-7B-Instruct I get the following error:

ERROR 08-04 18:59:00 [v1/executor/multiproc_executor.py:901] WorkerProc failed to start.
ERROR 08-04 18:59:00 [v1/executor/multiproc_executor.py:901] Traceback (most recent call last):
ERROR 08-04 18:59:00 [v1/executor/multiproc_executor.py:901]   File "/home/foo/Code/vllm-node/.venv/lib/python3.12/site-packages/vllm/v1/executor/multiproc_executor.py", line 868, in worker_main
ERROR 08-04 18:59:00 [v1/executor/multiproc_executor.py:901]     worker = WorkerProc(*args, **kwargs)
ERROR 08-04 18:59:00 [v1/executor/multiproc_executor.py:901]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 08-04 18:59:00 [v1/executor/multiproc_executor.py:901]   File "/home/foo/Code/vllm-node/.venv/lib/python3.12/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
ERROR 08-04 18:59:00 [v1/executor/multiproc_executor.py:901]     return func(*args, **kwargs)
ERROR 08-04 18:59:00 [v1/executor/multiproc_executor.py:901]            ^^^^^^^^^^^^^^^^^^^^^
ERROR 08-04 18:59:00 [v1/executor/multiproc_executor.py:901]   File "/home/foo/Code/vllm-node/.venv/lib/python3.12/site-packages/vllm/v1/executor/multiproc_executor.py", line 621, in __init__
ERROR 08-04 18:59:00 [v1/executor/multiproc_executor.py:901]     wrapper.init_worker(all_kwargs)
ERROR 08-04 18:59:00 [v1/executor/multiproc_executor.py:901]   File "/home/foo/Code/vllm-node/.venv/lib/python3.12/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
ERROR 08-04 18:59:00 [v1/executor/multiproc_executor.py:901]     return func(*args, **kwargs)
ERROR 08-04 18:59:00 [v1/executor/multiproc_executor.py:901]            ^^^^^^^^^^^^^^^^^^^^^
ERROR 08-04 18:59:00 [v1/executor/multiproc_executor.py:901]   File "/home/foo/Code/vllm-node/.venv/lib/python3.12/site-packages/vllm/v1/worker/worker_base.py", line 319, in init_worker
ERROR 08-04 18:59:00 [v1/executor/multiproc_executor.py:901]     self.worker = worker_class(**kwargs)
ERROR 08-04 18:59:00 [v1/executor/multiproc_executor.py:901]                   ^^^^^^^^^^^^^^^^^^^^^^
ERROR 08-04 18:59:00 [v1/executor/multiproc_executor.py:901]   File "/home/foo/Code/vllm-node/.venv/lib/python3.12/site-packages/vllm/v1/worker/cpu_worker.py", line 71, in __init__
ERROR 08-04 18:59:00 [v1/executor/multiproc_executor.py:901]     torch.ops._C.init_cpu_memory_env([memory_node])
ERROR 08-04 18:59:00 [v1/executor/multiproc_executor.py:901]     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 08-04 18:59:00 [v1/executor/multiproc_executor.py:901]   File "/home/foo/Code/vllm-node/.venv/lib/python3.12/site-packages/torch/_ops.py", line 1379, in __getattr__
ERROR 08-04 18:59:00 [v1/executor/multiproc_executor.py:901]     raise AttributeError(
ERROR 08-04 18:59:00 [v1/executor/multiproc_executor.py:901] AttributeError: '_OpNamespace' '_C' object has no attribute 'init_cpu_memory_env'

Relevant source:

It seems the function torch.ops._C.init_cpu_memory_env does not exist in PyTorch:

>>> import torch
>>> torch.__version__
'2.11.0+cpu'
>>> torch.ops._C.init_cpu_memory_env
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/foo/Code/vllm-node/.venv/lib/python3.12/site-packages/torch/_ops.py", line 1379, in __getattr__
    raise AttributeError(
AttributeError: '_OpNamespace' '_C' object has no attribute 'init_cpu_memory_env'

Any ideas how to fix this?

2

It seems likely that your error is due to a mismatch between the vLLM version and the installed PyTorch version, or a missing custom operator in your PyTorch build. The function torch.ops._C.init_cpu_memory_env is not present in standard PyTorch distributions, and this issue has been reported by others when running vLLM on CPU, especially on ARM or when building from source. The recommended solution is to ensure you are using the correct PyTorch version (often a specific +cpu build) and to follow the official vLLM CPU installation instructions closely, including building vLLM from source with the right dependencies and compiler version (docs, related issue).

Additionally, make sure you have installed all required system libraries (e.g., libnuma-dev, libtcmalloc-minimal4), and that your environment is clean (no conflicting PyTorch installations). If you are on ARM or a non-x86 platform, double-check that your build flags and environment match the ARM-specific instructions. If the problem persists, try using a pre-built wheel for your platform if available, or rebuild both PyTorch and vLLM from source as described in the docs.

Would you like a step-by-step guide for a clean installation on your platform?

Sources:

Hint: Mention @Herald in the post for followups.