Device: 8 x H800, CUDA 12.6, Driver: 560.35.03.
Currently, I can perform Qwen3 inference on vLLM 0.17.1. Now I want to implement DeepSeek-V4-Flash, but it seems I need to update the NVIDIA driver to be compatible with higher versions of vLLM (e.g., 0.23.0). I don’t have permission to update the driver, and I’m wondering if there are other ways to deploy DeepSeek-V4-Flash?
Hope to get a reply! Thank you!
You can try running newer vLLM versions (e.g., 0.23.0) on older NVIDIA drivers by enabling CUDA compatibility mode, which is supported on select professional/datacenter GPUs like H800. Set the environment variable VLLM_ENABLE_CUDA_COMPATIBILITY=1 when running the vLLM Docker container, or install the appropriate cuda-compat package and set VLLM_CUDA_COMPATIBILITY_PATH if running outside Docker. This allows vLLM to use forward compatibility libraries without requiring a driver update, but is not guaranteed to work for all models or CUDA features—DeepSeek-V4-Flash may still hit kernel or architecture limitations on H800, as its optimized kernels are primarily for Hopper/Blackwell GPUs and may not support Ampere/Hopper fallback paths fully (docs, troubleshooting, issue).
If compatibility mode fails or DeepSeek-V4-Flash requires kernels unavailable on H800, there is currently no officially supported fallback path in vLLM for this model on your hardware/driver combination. You may need to use a different model or wait for broader kernel support (issue). Would you like more detail on configuring CUDA compatibility or on model-specific kernel support?
Sources:
Hint: Mention @Herald in the post for followups.