# The current vLLM CPU backend is not working properly

**URL:** <https://discuss.vllm.ai/t/the-current-vllm-cpu-backend-is-not-working-properly/2708>\
**Category:** General\
**Created:** [June 9, 2026, 3:42pm UTC](https://discuss.vllm.ai/t/the-current-vllm-cpu-backend-is-not-working-properly/2708 "2026-06-09T15:42:21Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![vim](https://avatars.discourse-cdn.com/v4/letter/v/b4bc9f/32.png) [@vim](https://discuss.vllm.ai/u/vim)\
**Post date:** [June 9, 2026, 3:42pm UTC](https://discuss.vllm.ai/t/the-current-vllm-cpu-backend-is-not-working-properly/2708/1 "2026-06-09T15:42:21Z")

</div>

Hello there,

I am relatively new to this topic, so please pardon any mistakes.

Up until version 0.19.1, vLLM could utilize around 96 CPU threads. Starting with version 0.20.0, however, vLLM utilizes only two CPU threads at 100%. This is a very disappointing performance because the machine I have available for testing has 384 CPU threads (2x AMD Epyc 9655) and 1.5 TB of RAM.

vLLM mentions Triton at startup, so I tried to download and compile the Triton-CPU backend ( [GitHub - triton-lang/triton-cpu: An experimental CPU backend for Triton · GitHub](https://github.com/triton-lang/triton-cpu.git) ), but the compilation process terminated with an error that I was unable to resolve.

For the sake of completeness, I am attaching the log from the vLLM startup on the aforementioned machine.

```auto
$ vllm serve Qwen/Qwen3-8B --port 8000 --max-model-len 32768
INFO 06-09 16:43:11 [importing.py:45] Triton is installed but 0 active driver(s) found (expected 1). Disabling Triton to prevent runtime errors.
INFO 06-09 16:43:11 [importing.py:69] Triton not installed or not compatible; certain GPU-related functions will not be available.
(APIServer pid=72390) INFO 06-09 16:43:12 [utils.py:344] 
(APIServer pid=72390) INFO 06-09 16:43:12 [utils.py:344] █ █ █▄ ▄█
(APIServer pid=72390) INFO 06-09 16:43:12 [utils.py:344] ▄▄ ▄█ █ █ █ ▀▄▀ █ version 0.22.1
(APIServer pid=72390) INFO 06-09 16:43:12 [utils.py:344] █▄█▀ █ █ █ █ model Qwen/Qwen3-8B
(APIServer pid=72390) INFO 06-09 16:43:12 [utils.py:344] ▀▀ ▀▀▀▀▀ ▀▀▀▀▀ ▀ ▀
(APIServer pid=72390) INFO 06-09 16:43:12 [utils.py:344] 
(APIServer pid=72390) INFO 06-09 16:43:12 [utils.py:278] non-default args: {'model_tag': 'Qwen/Qwen3-8B', 'model': 'Qwen/Qwen3-8B', 'max_model_len': 32768}
(APIServer pid=72390) Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
(APIServer pid=72390) INFO 06-09 16:43:13 [model.py:617] Resolved architecture: Qwen3ForCausalLM
(APIServer pid=72390) INFO 06-09 16:43:13 [model.py:1752] Using max model len 32768
(APIServer pid=72390) INFO 06-09 16:43:13 [vllm.py:977] Asynchronous scheduling is enabled.
(APIServer pid=72390) INFO 06-09 16:43:13 [kernel.py:270] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'])
(APIServer pid=72390) WARNING 06-09 16:43:13 [vllm.py:502] Model runner v2 requires Triton; using the v1 model runner instead.
INFO 06-09 16:43:18 [importing.py:45] Triton is installed but 0 active driver(s) found (expected 1). Disabling Triton to prevent runtime errors.
INFO 06-09 16:43:18 [importing.py:69] Triton not installed or not compatible; certain GPU-related functions will not be available.
(EngineCore pid=72591) INFO 06-09 16:43:19 [core.py:112] Initializing a V1 LLM engine (v0.22.1) with config: model='Qwen/Qwen3-8B', speculative_config=None, tokenizer='Qwen/Qwen3-8B', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=32768, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, decode_context_parallel_size=1, dcp_comm_backend=ag_rs, disable_custom_all_reduce=True, quantization=None, quantization_config=None, enforce_eager=False, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=cpu, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False), seed=0, served_model_name=Qwen/Qwen3-8B, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'mode': <CompilationMode.DYNAMO_TRACE_ONCE: 2>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'ir_enable_torch_wrap': False, 'splitting_ops': [], 'compile_mm_encoder': False, 'cudagraph_mm_encoder': False, 'encoder_cudagraph_token_budgets': [], 'encoder_cudagraph_max_vision_items_per_batch': 0, 'encoder_cudagraph_max_frames_per_batch': None, 'compile_sizes': None, 'compile_ranges_endpoints': [2048], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'size_asserts': False, 'alignment_asserts': False, 'scalar_asserts': False, 'dce': True, 'nan_asserts': False, 'epilogue_fusion': True, 'cpp.dynamic_threads': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.NONE: 0>, 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': [], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False, 'fuse_rope_kvcache_cat_mla': False, 'fuse_act_padding': False}, 'max_cudagraph_capture_size': None, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': False, 'static_all_moe_layers': []}, kernel_config=KernelConfig(ir_op_priority=IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native']), enable_flashinfer_autotune=True, moe_backend='auto', linear_backend='auto')
(EngineCore pid=72591) INFO 06-09 16:43:19 [multiproc_executor.py:139] DP group leader: node_rank=0, node_rank_within_dp=0, master_addr=127.0.0.1, mq_connect_ip=10.10.51.190 (local), world_size=1, local_world_size=1
(EngineCore pid=72591) INFO 06-09 16:43:20 [ompmultiprocessing.py:185] OpenMP thread binding info: 
(EngineCore pid=72591) INFO 06-09 16:43:20 [ompmultiprocessing.py:185] VLLM_CPU_OMP_THREADS_BIND='auto', auto_setup=True, skip_setup=False
(EngineCore pid=72591) INFO 06-09 16:43:20 [ompmultiprocessing.py:185] local_world_size=1, reserve_cpu_num=1
(EngineCore pid=72591) INFO 06-09 16:43:20 [ompmultiprocessing.py:185] local_rank=0, core ids=[192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276, 277, 278, 279, 280, 281, 282, 283, 284, 285, 286]
(EngineCore pid=72591) INFO 06-09 16:43:20 [ompmultiprocessing.py:185] reserved_cpus=[287]
INFO 06-09 16:43:22 [importing.py:45] Triton is installed but 0 active driver(s) found (expected 1). Disabling Triton to prevent runtime errors.
INFO 06-09 16:43:22 [importing.py:69] Triton not installed or not compatible; certain GPU-related functions will not be available.
[W609 16:43:24.492166409 utils.cpp:68] Warning: NUMA binding: Using MEMBIND policy for memory allocation on the NUMA nodes (0). Memory allocations will be strictly bound to these NUMA nodes. (function init_cpu_memory_env)
WARNING 06-09 16:43:24 [vllm.py:502] Model runner v2 requires Triton; using the v1 model runner instead.
(Worker pid=72791) INFO 06-09 16:43:24 [parallel_state.py:1422] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://127.0.0.1:36457 backend=gloo
(Worker pid=72791) INFO 06-09 16:43:24 [parallel_state.py:1735] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A, EPLB rank N/A
(Worker pid=72791) INFO 06-09 16:43:24 [cpu_model_runner.py:104] Starting to load model Qwen/Qwen3-8B...
(Worker pid=72791) INFO 06-09 16:43:24 [selector.py:130] Using HND KV cache layout for CPU_ATTN backend.
(Worker pid=72791) INFO 06-09 16:43:25 [weight_utils.py:922] Filesystem type for checkpoints: NFS4. Checkpoint size: 15.26 GiB. Available RAM: 1508.73 GiB.
(Worker pid=72791) INFO 06-09 16:43:25 [weight_utils.py:884] Prefetching checkpoint files into page cache started (in background, num_threads=8, block_size=16777216 bytes)
Loading safetensors checkpoint shards: 0% Completed | 0/5 [00:00<?, ?it/s]
(Worker pid=72791) INFO 06-09 16:43:26 [weight_utils.py:856] Prefetching checkpoint files: 10% (1/5)
(Worker pid=72791) INFO 06-09 16:43:26 [weight_utils.py:856] Prefetching checkpoint files: 20% (2/5)
(Worker pid=72791) INFO 06-09 16:43:26 [weight_utils.py:856] Prefetching checkpoint files: 30% (3/5)
(Worker pid=72791) INFO 06-09 16:43:26 [weight_utils.py:856] Prefetching checkpoint files: 40% (4/5)
(Worker pid=72791) INFO 06-09 16:43:26 [weight_utils.py:856] Prefetching checkpoint files: 50% (5/5)
(Worker pid=72791) INFO 06-09 16:43:26 [weight_utils.py:879] Prefetching checkpoint files into page cache finished in 0.89s
Loading safetensors checkpoint shards: 20% Completed | 1/5 [00:01<00:05, 1.39s/it]
Loading safetensors checkpoint shards: 40% Completed | 2/5 [00:02<00:02, 1.02it/s]
Loading safetensors checkpoint shards: 60% Completed | 3/5 [00:02<00:01, 1.19it/s]
Loading safetensors checkpoint shards: 80% Completed | 4/5 [00:03<00:00, 1.38it/s]
Loading safetensors checkpoint shards: 100% Completed | 5/5 [00:03<00:00, 1.84it/s]
Loading safetensors checkpoint shards: 100% Completed | 5/5 [00:03<00:00, 1.42it/s]
(Worker pid=72791) 
(Worker pid=72791) INFO 06-09 16:43:29 [default_loader.py:397] Loading weights took 3.54 seconds
(Worker pid=72791) INFO 06-09 16:43:37 [cpu_model_runner.py:121] Warming up model for the compilation...
(Worker pid=72791) INFO 06-09 16:44:10 [decorators.py:708] saved AOT compiled function to /home/michalek/.cache/vllm/torch_compile_cache/torch_aot_compile/a84770d3632a2f224a48b63c886c0d28c9b2dfa7750641820143c524c373e06b/rank_0_0/model
(EngineCore pid=72591) INFO 06-09 16:44:38 [shm_broadcast.py:698] No available shared memory broadcast block found in 60 seconds. This typically happens when some processes are hanging or doing some time-consuming work (e.g. compilation, weight/kv cache quantization).
(Worker pid=72791) INFO 06-09 16:45:04 [monitor.py:81] Initial profiling/warmup run took 53.78 s
(Worker pid=72791) INFO 06-09 16:45:09 [cpu_model_runner.py:125] Warming up done.
(Worker pid=72791) INFO 06-09 16:45:10 [cpu_worker.py:211] Auto set (675.87/755.19) GiB for KV cache on node 0, with 694.77 GiB requested memory for the worker. 18.9 GiB memory was consumed by non-kv usages.
(EngineCore pid=72591) INFO 06-09 16:45:10 [kv_cache_utils.py:1733] GPU KV cache size: 4,921,472 tokens
(EngineCore pid=72591) INFO 06-09 16:45:10 [kv_cache_utils.py:1734] Maximum concurrency for 32,768 tokens per request: 150.19x
(EngineCore pid=72591) INFO 06-09 16:46:11 [shm_broadcast.py:698] No available shared memory broadcast block found in 60 seconds. This typically happens when some processes are hanging or doing some time-consuming work (e.g. compilation, weight/kv cache quantization).
(EngineCore pid=72591) INFO 06-09 16:46:42 [core.py:309] init engine (profile, create kv cache, warmup model) took 184.74 s
(EngineCore pid=72591) Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
(EngineCore pid=72591) WARNING 06-09 16:46:44 [vllm.py:502] Model runner v2 requires Triton; using the v1 model runner instead.
(EngineCore pid=72591) INFO 06-09 16:46:44 [vllm.py:977] Asynchronous scheduling is disabled.
(EngineCore pid=72591) WARNING 06-09 16:46:44 [vllm.py:1058] Inductor compilation was disabled by user settings, optimizations settings that are only active during inductor compilation will be ignored.
(EngineCore pid=72591) INFO 06-09 16:46:44 [kernel.py:270] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'])
(APIServer pid=72390) INFO 06-09 16:46:44 [api_server.py:592] Supported tasks: ['generate']
(APIServer pid=72390) WARNING 06-09 16:46:44 [model.py:1509] Default vLLM sampling parameters have been overridden by the model's `generation_config.json`: `{'temperature': 0.6, 'top_k': 20, 'top_p': 0.95}`. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`.
(APIServer pid=72390) INFO 06-09 16:46:46 [hf.py:488] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this.
(APIServer pid=72390) INFO 06-09 16:46:47 [api_server.py:596] Starting vLLM server on http://0.0.0.0:8000
(APIServer pid=72390) INFO 06-09 16:46:47 [launcher.py:37] Available routes are:
(APIServer pid=72390) INFO 06-09 16:46:47 [launcher.py:46] Route: /openapi.json, Methods: GET, HEAD
(APIServer pid=72390) INFO 06-09 16:46:47 [launcher.py:46] Route: /docs, Methods: GET, HEAD
(APIServer pid=72390) INFO 06-09 16:46:47 [launcher.py:46] Route: /docs/oauth2-redirect, Methods: GET, HEAD
(APIServer pid=72390) INFO 06-09 16:46:47 [launcher.py:46] Route: /redoc, Methods: GET, HEAD
(APIServer pid=72390) INFO 06-09 16:46:47 [launcher.py:46] Route: /tokenize, Methods: POST
(APIServer pid=72390) INFO 06-09 16:46:47 [launcher.py:46] Route: /detokenize, Methods: POST
(APIServer pid=72390) INFO 06-09 16:46:47 [launcher.py:46] Route: /load, Methods: GET
(APIServer pid=72390) INFO 06-09 16:46:47 [launcher.py:46] Route: /version, Methods: GET
(APIServer pid=72390) INFO 06-09 16:46:47 [launcher.py:46] Route: /health, Methods: GET
(APIServer pid=72390) INFO 06-09 16:46:47 [launcher.py:46] Route: /metrics, Methods: GET
(APIServer pid=72390) INFO 06-09 16:46:47 [launcher.py:46] Route: /v1/models, Methods: GET
(APIServer pid=72390) INFO 06-09 16:46:47 [launcher.py:46] Route: /ping, Methods: GET
(APIServer pid=72390) INFO 06-09 16:46:47 [launcher.py:46] Route: /ping, Methods: POST
(APIServer pid=72390) INFO 06-09 16:46:47 [launcher.py:46] Route: /invocations, Methods: POST
(APIServer pid=72390) INFO 06-09 16:46:47 [launcher.py:46] Route: /v1/chat/completions, Methods: POST
(APIServer pid=72390) INFO 06-09 16:46:47 [launcher.py:46] Route: /v1/chat/completions/batch, Methods: POST
(APIServer pid=72390) INFO 06-09 16:46:47 [launcher.py:46] Route: /v1/responses, Methods: POST
(APIServer pid=72390) INFO 06-09 16:46:47 [launcher.py:46] Route: /v1/responses/{response_id}, Methods: GET
(APIServer pid=72390) INFO 06-09 16:46:47 [launcher.py:46] Route: /v1/responses/{response_id}/cancel, Methods: POST
(APIServer pid=72390) INFO 06-09 16:46:47 [launcher.py:46] Route: /v1/completions, Methods: POST
(APIServer pid=72390) INFO 06-09 16:46:47 [launcher.py:46] Route: /v1/messages, Methods: POST
(APIServer pid=72390) INFO 06-09 16:46:47 [launcher.py:46] Route: /v1/messages/count_tokens, Methods: POST
(APIServer pid=72390) INFO 06-09 16:46:47 [launcher.py:46] Route: /inference/v1/generate, Methods: POST
(APIServer pid=72390) INFO 06-09 16:46:47 [launcher.py:46] Route: /scale_elastic_ep, Methods: POST
(APIServer pid=72390) INFO 06-09 16:46:47 [launcher.py:46] Route: /is_scaling_elastic_ep, Methods: POST
(APIServer pid=72390) INFO 06-09 16:46:47 [launcher.py:46] Route: /generative_scoring, Methods: POST
(APIServer pid=72390) INFO 06-09 16:46:47 [launcher.py:46] Route: /v1/chat/completions/render, Methods: POST
(APIServer pid=72390) INFO 06-09 16:46:47 [launcher.py:46] Route: /v1/completions/render, Methods: POST
(APIServer pid=72390) INFO: Started server process [72390]
(APIServer pid=72390) INFO: Waiting for application startup.
(APIServer pid=72390) INFO: Application startup complete.

```

Is there a way to improve the performance of the current vLLM releases on CPU-only systems?

Thanks in advance.

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [June 9, 2026, 3:42pm UTC](https://discuss.vllm.ai/t/the-current-vllm-cpu-backend-is-not-working-properly/2708/2 "2026-06-09T15:42:51Z")

</div>

vLLM v0.20.0+ introduced changes to CPU thread management, but it should still utilize many threads if configured correctly. The log shows vLLM is binding a large set of CPU cores (core ids 192–286) and reserving one for the control plane, which is expected on a multi-socket NUMA system. However, if you observe only two threads at 100% and poor scaling, this may be due to thread binding, OpenMP settings, or the lack of Triton CPU backend support (which is still experimental and not required for vLLM CPU inference) — and not a hard-coded thread limit in vLLM itself. The Triton-CPU backend is not required for vLLM CPU backend and is not officially supported in vLLM’s documentation for CPU inference as of now ([see here](https://docs.vllm.ai/en/latest/getting_started/installation/cpu/#__span-53-8)).

To maximize CPU utilization, ensure you set `VLLM_CPU_OMP_THREADS_BIND` to cover all physical cores you want to use (e.g., `0-383` for 384 threads), and check that `OMP_NUM_THREADS` is not set to a low value. Also, verify that your NUMA configuration is optimal and that each rank’s threads are bound to the same NUMA node ([see here](https://docs.vllm.ai/en/latest/getting_started/installation/cpu/#__span-53-8)). If you still see poor scaling, try explicitly setting `VLLM_CPU_OMP_THREADS_BIND` and monitor with `htop` to confirm thread usage. Triton-related warnings can be ignored for CPU-only inference. Would you like a step-by-step example of how to set these environment variables for your system?

Sources:

- [vLLM CPU installation and thread binding](https://docs.vllm.ai/en/latest/getting_started/installation/cpu/#__span-53-8)
- [vLLM CPU NUMA and thread tuning](https://docs.vllm.ai/en/latest/getting_started/installation/cpu/#__span-63-8)

Hint: Mention @RunLLM in the post for followups.

---

<div class="post-metadata">

**Author:** ![vim](https://avatars.discourse-cdn.com/v4/letter/v/b4bc9f/32.png) [@vim](https://discuss.vllm.ai/u/vim)\
**Post date:** [June 10, 2026, 11:18am UTC](https://discuss.vllm.ai/t/the-current-vllm-cpu-backend-is-not-working-properly/2708/3 "2026-06-10T11:18:40Z")

</div>

@RunLLM I have already tried to set `VLLM_CPU_OMP_THREADS_BIND` environment variable but with no effect. When I specify e.g. `VLLM_CPU_OMP_THREADS_BIND='192-287'`, vLLM utilizes core 192 and 288 (as viewed in htop), in case of `VLLM_CPU_OMP_THREADS_BIND='0-383'`, vLLM utilizes core 0 and 97.

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [June 10, 2026, 11:19am UTC](https://discuss.vllm.ai/t/the-current-vllm-cpu-backend-is-not-working-properly/2708/4 "2026-06-10T11:19:04Z")

</div>

This behavior suggests that vLLM is not correctly binding OpenMP threads to the specified CPU cores, and is instead defaulting to using only a couple of threads. This issue can occur if the OpenMP runtime or vLLM’s thread binding logic is not functioning as expected, possibly due to environment, library, or NUMA configuration issues. It is not expected or documented that vLLM should only use two threads when `VLLM_CPU_OMP_THREADS_BIND` is set to a large range ([see docs](https://docs.vllm.ai/en/latest/getting_started/installation/cpu/#__span-53-8)).

Additionally, some users have reported similar issues when the OpenMP library (e.g., Intel OpenMP or libgomp) is not properly preloaded, or when there are conflicts between different OpenMP runtimes ([see issue](https://github.com/vllm-project/vllm/issues/43240)). Try setting `LD_PRELOAD` to include your system’s OpenMP library (e.g., `libiomp5.so` or `libgomp.so.1`) and ensure no conflicting OpenMP libraries are loaded. Also, make sure `OMP_NUM_THREADS` is unset or set to match the number of threads you want. Would you like detailed troubleshooting steps for this setup?

Sources:

- [vLLM CPU installation and thread binding](https://docs.vllm.ai/en/latest/getting_started/installation/cpu/#__span-53-8)
- [vLLM GitHub issue: Performance drop with multiple vLLM on NUMA nodes](https://github.com/vllm-project/vllm/issues/43240)

Hint: Mention @RunLLM in the post for followups.

---

<div class="post-metadata">

**Author:** ![vim](https://avatars.discourse-cdn.com/v4/letter/v/b4bc9f/32.png) [@vim](https://discuss.vllm.ai/u/vim)\
**Post date:** [June 10, 2026, 11:28am UTC](https://discuss.vllm.ai/t/the-current-vllm-cpu-backend-is-not-working-properly/2708/5 "2026-06-10T11:28:17Z")

</div>

@RunLLM I have also already tried to set path to `libiomp5.so` in LD\_PRELOAD with no effect.

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [June 10, 2026, 11:28am UTC](https://discuss.vllm.ai/t/the-current-vllm-cpu-backend-is-not-working-properly/2708/6 "2026-06-10T11:28:41Z")

</div>

If setting `LD_PRELOAD` to include `libiomp5.so` has no effect and vLLM still only uses two CPU threads, this suggests a deeper issue with OpenMP initialization or vLLM’s thread binding logic. The official documentation and recent issues confirm that, when properly configured, vLLM should utilize all specified cores ([see docs](https://docs.vllm.ai/en/latest/getting_started/installation/cpu/#__span-53-8)). There is no documented limitation that would restrict vLLM to two threads in your scenario.

This problem may be related to the OpenMP environment being set too late (after the library is loaded), or to a mismatch between the allowed CPU set and the NUMA node mapping, especially in containerized or resource-constrained environments ([see issue](https://github.com/vllm-project/vllm/issues/43240)). Try launching vLLM from a clean shell with only the essential environment variables set, and verify that no other OpenMP libraries are loaded. If possible, test with `VLLM_CPU_OMP_THREADS_BIND=nobind` to see if OpenMP uses more threads automatically. Would you like a step-by-step diagnostic checklist to further isolate the cause?

Sources:

- [vLLM CPU installation and thread binding](https://docs.vllm.ai/en/latest/getting_started/installation/cpu/#__span-53-8)
- [vLLM GitHub issue: Performance drop with multiple vLLM on NUMA nodes](https://github.com/vllm-project/vllm/issues/43240)

Hint: Mention @RunLLM in the post for followups.

---

<div class="post-metadata">

**Author:** ![vim](https://avatars.discourse-cdn.com/v4/letter/v/b4bc9f/32.png) [@vim](https://discuss.vllm.ai/u/vim)\
**Post date:** [June 10, 2026, 11:38am UTC](https://discuss.vllm.ai/t/the-current-vllm-cpu-backend-is-not-working-properly/2708/7 "2026-06-10T11:38:34Z")

</div>

@RunLLM With `VLLM_CPU_OMP_THREADS_BIND=nobind` vLLM utilizes approximately one half of available CPU threads.

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [June 10, 2026, 11:39am UTC](https://discuss.vllm.ai/t/the-current-vllm-cpu-backend-is-not-working-properly/2708/8 "2026-06-10T11:39:00Z")

</div>

When you set `VLLM_CPU_OMP_THREADS_BIND=nobind`, vLLM disables explicit thread binding and relies on the OpenMP runtime to determine thread placement and count. In this mode, the number of threads is controlled by `OMP_NUM_THREADS` (or the OpenMP default), which often defaults to the number of physical cores, not logical threads, especially on systems with hyperthreading ([see docs](https://docs.vllm.ai/en/latest/getting_started/installation/cpu/#__span-53-8); [PR discussion](https://github.com/vllm-project/vllm/pull/27953)).

To utilize all available threads, explicitly set `OMP_NUM_THREADS` to the desired value (e.g., `OMP_NUM_THREADS=384` for your system) before launching vLLM. Would you like a code example or further details on how to set this up?

Sources:

- [vLLM CPU installation and thread binding](https://docs.vllm.ai/en/latest/getting_started/installation/cpu/#__span-53-8)
- [Add Option for Disabling Binding to Specific CPU Cores (nobind)](https://github.com/vllm-project/vllm/pull/27953)

Hint: Mention @RunLLM in the post for followups.

---

<div class="post-metadata">

**Author:** ![vim](https://avatars.discourse-cdn.com/v4/letter/v/b4bc9f/32.png) [@vim](https://discuss.vllm.ai/u/vim)\
**Post date:** [June 10, 2026, 12:21pm UTC](https://discuss.vllm.ai/t/the-current-vllm-cpu-backend-is-not-working-properly/2708/9 "2026-06-10T12:21:32Z")

</div>

I tried:  
`export OMP_NUM_THREADS=380`  
`VLLM_CPU_OMP_THREADS_BIND='nobind' vllm serve Qwen/Qwen3-8B --port 8000 --max-model-len 32768`  
and vLLM still utilizes approximately one half of CPU threads.

---

<div class="post-metadata">

**Author:** ![frankly](https://avatars.discourse-cdn.com/v4/letter/f/91b2a8/32.png) [@frankly](https://discuss.vllm.ai/u/frankly)\
**Post date:** [June 20, 2026, 3:25am UTC](https://discuss.vllm.ai/t/the-current-vllm-cpu-backend-is-not-working-properly/2708/10 "2026-06-20T03:25:38Z")

</div>

Any chance you ever found a solution? I’m running into the same issue.

Edit: downgrading to 0.19.1 solved the issue for me.

Also make sure when you say only half of threads are being used, youre not looking at HT siblings. You really want half of threads firing cause HT is inefficient