# RTX PRO 6000 users seek help, LLAMA 4 NVFP4 **URL:** \ **Category:** NVIDIA GPU Support\ **Created:** [November 25, 2025, 5:11am UTC](https://discuss.vllm.ai/t/rtx-pro-6000-users-seek-help-llama-4-nvfp4/2010 "2025-11-25T05:11:31Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![win10ogod](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/win10ogod/32/950_2.png) [@win10ogod](https://discuss.vllm.ai/u/win10ogod)\ **Post date:** [November 25, 2025, 5:11am UTC](https://discuss.vllm.ai/t/rtx-pro-6000-users-seek-help-llama-4-nvfp4/2010/1 "2025-11-25T05:11:31Z")
docker run --rm -it --gpus all --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 -p 8000:8000 -v E:/text-generation-webui-1.14/user\_data/models:/model vllm-blackwell:latest python -m vllm.entrypoints.openai.api\_server --model /model/Llama4-Scout17B-NVFP4 --trust-remote-code --host 0.0.0.0 --port 8000 --max-model-len 131072 --gpu-memory-utilization 0.95 --served-model-name Llama4-Scout17B-NVFP4 --chat-template /model/Llama4-Scout17B-NVFP4/chat\_template.jinja --kv-cache-dtype fp8 --no-enable-prefix-caching true --async-scheduling --mm-encoder-tp-mode data --enable-auto-tool-choice --tool-call-parser llama4\_pythonic --quantization modelopt\_fp4 (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] EngineCore failed to start. (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] Traceback (most recent call last): (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] File “/workspace/vllm/vllm/v1/engine/core.py”, line 833, in run\_engine\_core (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] engine\_core = EngineCoreProc(\*args, \*\*kwargs) (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] File “/workspace/vllm/vllm/v1/engine/core.py”, line 609, in \__init_\_ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] super().\__init_\_( (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] File “/workspace/vllm/vllm/v1/engine/core.py”, line 109, in \__init_\_ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] num\_gpu\_blocks, num\_cpu\_blocks, kv\_cache\_config = self.\_initialize\_kv\_caches( (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] ^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] File “/workspace/vllm/vllm/v1/engine/core.py”, line 234, in \_initialize\_kv\_caches (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] available\_gpu\_memory = self.model\_executor.determine\_available\_memory() (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] File “/workspace/vllm/vllm/v1/executor/abstract.py”, line 126, in determine\_available\_memory (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] return self.collective\_rpc(“determine\_available\_memory”) (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] File “/workspace/vllm/vllm/v1/executor/uniproc\_executor.py”, line 75, in collective\_rpc (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] result = run\_method(self.driver\_worker, method, args, kwargs) (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] File “/workspace/vllm/vllm/v1/serial\_utils.py”, line 479, in run\_method (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] return func(\*args, \*\*kwargs) (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] ^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] File “/usr/local/lib/python3.12/dist-packages/torch/utils/\_contextlib.py”, line 116, in decorate\_context (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] return func(\*args, \*\*kwargs) (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] ^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] File “/workspace/vllm/vllm/v1/worker/gpu\_worker.py”, line 313, in determine\_available\_memory (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] self.model\_runner.profile\_run() (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] File “/workspace/vllm/vllm/v1/worker/gpu\_model\_runner.py”, line 4145, in profile\_run (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] hidden\_states, last\_hidden\_states = self.\_dummy\_run( (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] ^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] File “/usr/local/lib/python3.12/dist-packages/torch/utils/\_contextlib.py”, line 116, in decorate\_context (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] return func(\*args, \*\*kwargs) (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] ^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] File “/workspace/vllm/vllm/v1/worker/gpu\_model\_runner.py”, line 3864, in \_dummy\_run (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] outputs = self.model( (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] ^^^^^^^^^^^ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] File “/workspace/vllm/vllm/compilation/cuda\_graph.py”, line 126, in \__call_\_ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] return self.runnable(\*args, \*\*kwargs) (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] File “/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py”, line 1751, in \_wrapped\_call\_impl (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] return self.\_call\_impl(\*args, \*\*kwargs) (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] File “/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py”, line 1762, in \_call\_impl (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] return forward\_call(\*args, \*\*kwargs) (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] File “/workspace/vllm/vllm/model\_executor/models/mllama4.py”, line 901, in forward (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] return self.language\_model( (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] ^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] File “/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py”, line 1751, in \_wrapped\_call\_impl (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] return self.\_call\_impl(\*args, \*\*kwargs) (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] File “/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py”, line 1762, in \_call\_impl (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] return forward\_call(\*args, \*\*kwargs) (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] File “/workspace/vllm/vllm/model\_executor/models/llama.py”, line 617, in forward (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] model\_output = self.model( (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] ^^^^^^^^^^^ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] File “/workspace/vllm/vllm/compilation/decorators.py”, line 471, in \__call_\_ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] output = TorchCompileWithNoGuardsWrapper.\__call_\_(self, \*args, \*\*kwargs) (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] File “/workspace/vllm/vllm/compilation/wrapper.py”, line 149, in \__call_\_ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] return self.\_compiled\_callable(\*args, \*\*kwargs) (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] File “/usr/local/lib/python3.12/dist-packages/torch/\_dynamo/eval\_frame.py”, line 671, in \_fn (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] raise e.remove\_dynamo\_frames() from None # see TORCHDYNAMO\_VERBOSE=1 (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] File “/usr/local/lib/python3.12/dist-packages/torch/\_dynamo/output\_graph.py”, line 1569, in \_call\_user\_compiler (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] raise BackendCompilerFailed( (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] File “/usr/local/lib/python3.12/dist-packages/torch/\_dynamo/output\_graph.py”, line 1544, in \_call\_user\_compiler (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] compiled\_fn = compiler\_fn(gm, self.example\_inputs()) (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] File “/usr/local/lib/python3.12/dist-packages/torch/\_dynamo/repro/after\_dynamo.py”, line 150, in \__call_\_ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] compiled\_gm = compiler\_fn(gm, example\_inputs) (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] File “/usr/local/lib/python3.12/dist-packages/torch/\_dynamo/repro/after\_dynamo.py”, line 150, in \__call_\_ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] compiled\_gm = compiler\_fn(gm, example\_inputs) (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] File “/usr/local/lib/python3.12/dist-packages/torch/\ __init_\_.py”, line 2400, in \__ call_\_ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] return self.compiler\_fn(model\_, inputs\_, \*\*self.kwargs) (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] torch.\_dynamo.exc.BackendCompilerFailed: backend=‘\’ raised: (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] TypeError: VllmBackend.\__call_\_() got an unexpected keyword argument ‘options’ (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] Set TORCHDYNAMO\_VERBOSE=1 for the internal stack trace (please do this especially if you’re reporting a bug to PyTorch). For even more developer context, set TORCH\_LOGS=“+dynamo” (EngineCore\_DP0 pid=221) ERROR 11-25 04:36:24 [core.py:842] (EngineCore\_DP0 pid=221) Process EngineCore\_DP0: (EngineCore\_DP0 pid=221) Traceback (most recent call last): (EngineCore\_DP0 pid=221) File “/usr/lib/python3.12/multiprocessing/process.py”, line 314, in \_bootstrap (EngineCore\_DP0 pid=221) self.run() (EngineCore\_DP0 pid=221) File “/usr/lib/python3.12/multiprocessing/process.py”, line 108, in run (EngineCore\_DP0 pid=221) self.\_target(\*self.\_args, \*\*self.\_kwargs) (EngineCore\_DP0 pid=221) File “/workspace/vllm/vllm/v1/engine/core.py”, line 846, in run\_engine\_core (EngineCore\_DP0 pid=221) raise e (EngineCore\_DP0 pid=221) File “/workspace/vllm/vllm/v1/engine/core.py”, line 833, in run\_engine\_core (EngineCore\_DP0 pid=221) engine\_core = EngineCoreProc(\*args, \*\*kwargs) (EngineCore\_DP0 pid=221) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) File “/workspace/vllm/vllm/v1/engine/core.py”, line 609, in \__init_\_ (EngineCore\_DP0 pid=221) super().\__init_\_( (EngineCore\_DP0 pid=221) File “/workspace/vllm/vllm/v1/engine/core.py”, line 109, in \__init_\_ (EngineCore\_DP0 pid=221) num\_gpu\_blocks, num\_cpu\_blocks, kv\_cache\_config = self.\_initialize\_kv\_caches( (EngineCore\_DP0 pid=221) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) File “/workspace/vllm/vllm/v1/engine/core.py”, line 234, in \_initialize\_kv\_caches (EngineCore\_DP0 pid=221) available\_gpu\_memory = self.model\_executor.determine\_available\_memory() (EngineCore\_DP0 pid=221) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) File “/workspace/vllm/vllm/v1/executor/abstract.py”, line 126, in determine\_available\_memory (EngineCore\_DP0 pid=221) return self.collective\_rpc(“determine\_available\_memory”) (EngineCore\_DP0 pid=221) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) File “/workspace/vllm/vllm/v1/executor/uniproc\_executor.py”, line 75, in collective\_rpc (EngineCore\_DP0 pid=221) result = run\_method(self.driver\_worker, method, args, kwargs) (EngineCore\_DP0 pid=221) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) File “/workspace/vllm/vllm/v1/serial\_utils.py”, line 479, in run\_method (EngineCore\_DP0 pid=221) return func(\*args, \*\*kwargs) (EngineCore\_DP0 pid=221) ^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) File “/usr/local/lib/python3.12/dist-packages/torch/utils/\_contextlib.py”, line 116, in decorate\_context (EngineCore\_DP0 pid=221) return func(\*args, \*\*kwargs) (EngineCore\_DP0 pid=221) ^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) File “/workspace/vllm/vllm/v1/worker/gpu\_worker.py”, line 313, in determine\_available\_memory (EngineCore\_DP0 pid=221) self.model\_runner.profile\_run() (EngineCore\_DP0 pid=221) File “/workspace/vllm/vllm/v1/worker/gpu\_model\_runner.py”, line 4145, in profile\_run (EngineCore\_DP0 pid=221) hidden\_states, last\_hidden\_states = self.\_dummy\_run( (EngineCore\_DP0 pid=221) ^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) File “/usr/local/lib/python3.12/dist-packages/torch/utils/\_contextlib.py”, line 116, in decorate\_context (EngineCore\_DP0 pid=221) return func(\*args, \*\*kwargs) (EngineCore\_DP0 pid=221) ^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) File “/workspace/vllm/vllm/v1/worker/gpu\_model\_runner.py”, line 3864, in \_dummy\_run (EngineCore\_DP0 pid=221) outputs = self.model( (EngineCore\_DP0 pid=221) ^^^^^^^^^^^ (EngineCore\_DP0 pid=221) File “/workspace/vllm/vllm/compilation/cuda\_graph.py”, line 126, in \__call_\_ (EngineCore\_DP0 pid=221) return self.runnable(\*args, \*\*kwargs) (EngineCore\_DP0 pid=221) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) File “/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py”, line 1751, in \_wrapped\_call\_impl (EngineCore\_DP0 pid=221) return self.\_call\_impl(\*args, \*\*kwargs) (EngineCore\_DP0 pid=221) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) File “/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py”, line 1762, in \_call\_impl (EngineCore\_DP0 pid=221) return forward\_call(\*args, \*\*kwargs) (EngineCore\_DP0 pid=221) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) File “/workspace/vllm/vllm/model\_executor/models/mllama4.py”, line 901, in forward (EngineCore\_DP0 pid=221) return self.language\_model( (EngineCore\_DP0 pid=221) ^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) File “/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py”, line 1751, in \_wrapped\_call\_impl (EngineCore\_DP0 pid=221) return self.\_call\_impl(\*args, \*\*kwargs) (EngineCore\_DP0 pid=221) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) File “/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py”, line 1762, in \_call\_impl (EngineCore\_DP0 pid=221) return forward\_call(\*args, \*\*kwargs) (EngineCore\_DP0 pid=221) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) File “/workspace/vllm/vllm/model\_executor/models/llama.py”, line 617, in forward (EngineCore\_DP0 pid=221) model\_output = self.model( (EngineCore\_DP0 pid=221) ^^^^^^^^^^^ (EngineCore\_DP0 pid=221) File “/workspace/vllm/vllm/compilation/decorators.py”, line 471, in \__call_\_ (EngineCore\_DP0 pid=221) output = TorchCompileWithNoGuardsWrapper.\__call_\_(self, \*args, \*\*kwargs) (EngineCore\_DP0 pid=221) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) File “/workspace/vllm/vllm/compilation/wrapper.py”, line 149, in \__call_\_ (EngineCore\_DP0 pid=221) return self.\_compiled\_callable(\*args, \*\*kwargs) (EngineCore\_DP0 pid=221) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) File “/usr/local/lib/python3.12/dist-packages/torch/\_dynamo/eval\_frame.py”, line 671, in \_fn (EngineCore\_DP0 pid=221) raise e.remove\_dynamo\_frames() from None # see TORCHDYNAMO\_VERBOSE=1 (EngineCore\_DP0 pid=221) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) File “/usr/local/lib/python3.12/dist-packages/torch/\_dynamo/output\_graph.py”, line 1569, in \_call\_user\_compiler (EngineCore\_DP0 pid=221) raise BackendCompilerFailed( (EngineCore\_DP0 pid=221) File “/usr/local/lib/python3.12/dist-packages/torch/\_dynamo/output\_graph.py”, line 1544, in \_call\_user\_compiler (EngineCore\_DP0 pid=221) compiled\_fn = compiler\_fn(gm, self.example\_inputs()) (EngineCore\_DP0 pid=221) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) File “/usr/local/lib/python3.12/dist-packages/torch/\_dynamo/repro/after\_dynamo.py”, line 150, in \__call_\_ (EngineCore\_DP0 pid=221) compiled\_gm = compiler\_fn(gm, example\_inputs) (EngineCore\_DP0 pid=221) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) File “/usr/local/lib/python3.12/dist-packages/torch/\_dynamo/repro/after\_dynamo.py”, line 150, in \__call_\_ (EngineCore\_DP0 pid=221) compiled\_gm = compiler\_fn(gm, example\_inputs) (EngineCore\_DP0 pid=221) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) File “/usr/local/lib/python3.12/dist-packages/torch/\ __init_\_.py”, line 2400, in \__ call_\_ (EngineCore\_DP0 pid=221) return self.compiler\_fn(model\_, inputs\_, \*\*self.kwargs) (EngineCore\_DP0 pid=221) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore\_DP0 pid=221) torch.\_dynamo.exc.BackendCompilerFailed: backend=‘\’ raised: (EngineCore\_DP0 pid=221) TypeError: VllmBackend.\__call_\_() got an unexpected keyword argument ‘options’ (EngineCore\_DP0 pid=221) (EngineCore\_DP0 pid=221) Set TORCHDYNAMO\_VERBOSE=1 for the internal stack trace (please do this especially if you’re reporting a bug to PyTorch). For even more developer context, set TORCH\_LOGS=“+dynamo” (EngineCore\_DP0 pid=221) [rank0]:[W1125 04:36:24.755274875 ProcessGroupNCCL.cpp:1505] Warning: WARNING: destroy\_process\_group() was not called before program exit, which can leak resources. For more info, please see [Distributed communication package - torch.distributed — PyTorch 2.9 documentation](https://pytorch.org/docs/stable/distributed.html#shutdown) (function operator()) (APIServer pid=1) Traceback (most recent call last): (APIServer pid=1) File “”, line 198, in \_run\_module\_as\_main (APIServer pid=1) File “”, line 88, in \_run\_code (APIServer pid=1) File “/workspace/vllm/vllm/entrypoints/openai/api\_server.py”, line 2178, in (APIServer pid=1) uvloop.run(run\_server(args)) (APIServer pid=1) File “/usr/local/lib/python3.12/dist-packages/uvloop/\__init_\_.py”, line 96, in run (APIServer pid=1) return \_\_asyncio.run( (APIServer pid=1) ^^^^^^^^^^^^^^ (APIServer pid=1) File “/usr/lib/python3.12/asyncio/runners.py”, line 194, in run (APIServer pid=1) return runner.run(main) (APIServer pid=1) ^^^^^^^^^^^^^^^^ (APIServer pid=1) File “/usr/lib/python3.12/asyncio/runners.py”, line 118, in run (APIServer pid=1) return self.\_loop.run\_until\_complete(task) (APIServer pid=1) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (APIServer pid=1) File “uvloop/loop.pyx”, line 1518, in uvloop.loop.Loop.run\_until\_complete (APIServer pid=1) File “/usr/local/lib/python3.12/dist-packages/uvloop/\__init_\_.py”, line 48, in wrapper (APIServer pid=1) return await main (APIServer pid=1) ^^^^^^^^^^ (APIServer pid=1) File “/workspace/vllm/vllm/entrypoints/openai/api\_server.py”, line 2106, in run\_server (APIServer pid=1) await run\_server\_worker(listen\_address, sock, args, \*\*uvicorn\_kwargs) (APIServer pid=1) File “/workspace/vllm/vllm/entrypoints/openai/api\_server.py”, line 2125, in run\_server\_worker (APIServer pid=1) async with build\_async\_engine\_client( (APIServer pid=1) File “/usr/lib/python3.12/contextlib.py”, line 210, in \__aenter_\_ (APIServer pid=1) return await anext(self.gen) (APIServer pid=1) ^^^^^^^^^^^^^^^^^^^^^ (APIServer pid=1) File “/workspace/vllm/vllm/entrypoints/openai/api\_server.py”, line 196, in build\_async\_engine\_client (APIServer pid=1) async with build\_async\_engine\_client\_from\_engine\_args( (APIServer pid=1) File “/usr/lib/python3.12/contextlib.py”, line 210, in \__aenter_\_ (APIServer pid=1) return await anext(self.gen) (APIServer pid=1) ^^^^^^^^^^^^^^^^^^^^^ (APIServer pid=1) File “/workspace/vllm/vllm/entrypoints/openai/api\_server.py”, line 237, in build\_async\_engine\_client\_from\_engine\_args (APIServer pid=1) async\_llm = AsyncLLM.from\_vllm\_config( (APIServer pid=1) ^^^^^^^^^^^^^^^^^^^^^^^^^^ (APIServer pid=1) File “/workspace/vllm/vllm/utils/func\_utils.py”, line 116, in inner (APIServer pid=1) return fn(\*args, \*\*kwargs) (APIServer pid=1) ^^^^^^^^^^^^^^^^^^^ (APIServer pid=1) File “/workspace/vllm/vllm/v1/engine/async\_llm.py”, line 219, in from\_vllm\_config (APIServer pid=1) return cls( (APIServer pid=1) ^^^^ (APIServer pid=1) File “/workspace/vllm/vllm/v1/engine/async\_llm.py”, line 133, in \__init_\_ (APIServer pid=1) self.engine\_core = EngineCoreClient.make\_async\_mp\_client( (APIServer pid=1) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (APIServer pid=1) File “/workspace/vllm/vllm/v1/engine/core\_client.py”, line 121, in make\_async\_mp\_client (APIServer pid=1) return AsyncMPClient(\*client\_args) (APIServer pid=1) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ (APIServer pid=1) File “/workspace/vllm/vllm/v1/engine/core\_client.py”, line 808, in \__init_\_ (APIServer pid=1) super().\__init_\_( (APIServer pid=1) File “/workspace/vllm/vllm/v1/engine/core\_client.py”, line 469, in \__init_\_ (APIServer pid=1) with launch\_core\_engines(vllm\_config, executor\_class, log\_stats) as ( (APIServer pid=1) File “/usr/lib/python3.12/contextlib.py”, line 144, in \__exit_\_ (APIServer pid=1) next(self.gen) (APIServer pid=1) File “/workspace/vllm/vllm/v1/engine/utils.py”, line 907, in launch\_core\_engines (APIServer pid=1) wait\_for\_engine\_startup( (APIServer pid=1) File “/workspace/vllm/vllm/v1/engine/utils.py”, line 964, in wait\_for\_engine\_startup (APIServer pid=1) raise RuntimeError( (APIServer pid=1) RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {} ---
**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\ **Post date:** [November 25, 2025, 5:12am UTC](https://discuss.vllm.ai/t/rtx-pro-6000-users-seek-help-llama-4-nvfp4/2010/2 "2025-11-25T05:12:09Z")
Your error is caused by a TypeError: `VllmBackend. __call__ () got an unexpected keyword argument 'options'`. This indicates a compatibility issue between your vLLM version and the installed PyTorch version—recent PyTorch releases (2.3+) pass an `options` argument to backend compilers, but older vLLM versions do not expect it, resulting in this crash. This is a known problem and is not specific to your model or Docker setup, but rather to the vLLM/PyTorch interface itself ([see related issues](https://github.com/vllm-project/vllm/issues/29265)). To resolve this, upgrade to the latest vLLM nightly build, which includes a fix for this interface change. Alternatively, you can downgrade PyTorch to a version before 2.3, but upgrading vLLM is the recommended solution ([see discussion](https://github.com/vllm-project/vllm/issues/29265)). No other command-line flags or settings will fix this error. Would you like more detail on how to upgrade vLLM in your Docker environment? Sources: - [vLLM Issue #29265](https://github.com/vllm-project/vllm/issues/29265) Hint: Mention @RunLLM in the post for followups.