Why vllm docker run command gets stuck here

General
1

why vllm docker run command gets stuck here

[aiter] import [module_aiter_enum] under /usr/local/lib/python3.12/dist-packages/aiter/jit/module_aiter_enum.so
[aiter] import [module_aiter_enum] under /usr/local/lib/python3.12/dist-packages/aiter/jit/module_aiter_enum.so

2

vLLM Docker runs can get stuck at [aiter] import [module_aiter_enum] if the installed aiter package or its JIT modules are not compatible with your GPU architecture, or if the Docker image was built without support for your specific GPU (e.g., missing your GPU’s architecture in the build flags). This is especially common on AMD GPUs when the image does not include the correct PYTORCH_ROCM_ARCH for your hardware, causing the aiter JIT to hang during GPU enumeration or module import, as described in the root cause analysis for ROCm images and aiter integration issues in vLLM ([Bug]: rocm/vllm:rocm7.0.0_vllm_0.11.1_20251103 has an error with flash attention)[[Bug]: rocm/vllm:rocm7.0.0_vllm_0.11.1_20251103 has an error with flash attention · Issue #28052 · vllm-project/vllm · GitHub].

To resolve this, rebuild vLLM and aiter from source with the correct GPU architecture (e.g., set PYTORCH_ROCM_ARCH="gfx1103" for AMD Radeon 780M), or use a Docker image that explicitly includes your GPU’s architecture. If you are not on ROCm/AMD, ensure your CUDA or ROCm drivers and all dependencies are correctly installed and compatible with the vLLM Docker image ([Bug]: rocm/vllm:rocm7.0.0_vllm_0.11.1_20251103 has an error with flash attention)[[Bug]: rocm/vllm:rocm7.0.0_vllm_0.11.1_20251103 has an error with flash attention · Issue #28052 · vllm-project/vllm · GitHub]. Would you like more detail or troubleshooting steps?

Sources:

Hint: Mention @RunLLM in the post for followups.