Use editable vllm for different GPU types

General
1

I build vllm from source in a docker image.

RUN git clone https://github.com/vllm-project/vllm.git /workspace/vllm
WORKDIR /workspace/vllm
RUN uv pip install --system -e . --verbose

I’m wondering whether the wheel built in this docker image can be used across GPU types.

2

No, a vLLM wheel built from source in a Docker image is typically only compatible with the GPU architecture present during the build, unless you explicitly specify multiple GPU architectures using the --build-arg torch_cuda_arch_list argument when building the image. By default, vLLM builds for all GPU types for widest distribution, but if you set torch_cuda_arch_list="", it will only build for the current GPU type, making the wheel not portable across different GPU types. To ensure compatibility across GPU types, you must manually specify all desired architectures during the build process as described in the documentation and community discussions (docs, discussion).

Would you like more detail on how to specify multiple GPU architectures or example Docker build commands?

Sources: