I am attempting to deploy DeepSeek-V4-Flash on an 8-GPU server equipped with RTX 4090 cards (48GB VRAM per card). However, the deployment consistently fails. Would it be possible to share any relevant guides, documentation, or best practices for this setup?
DeepSeek-V4-Flash is not officially supported on RTX 4090 (sm_89/Ada) in upstream vLLM, but there is active community work to enable it. The most up-to-date approach is to use the preview branch at GitHub - jasl/vllm at ds4-sm120-preview · GitHub, which includes essential patches for SM120 (Ada/Blackwell) support. Users have reported successful deployment on 4× RTX Pro 6000 (Blackwell) and RTX 4090 with this branch, though performance may not match Hopper-class GPUs and some features (e.g., FP4 indexer) may still require DeepGEMM or further fixes. For best results, follow the discussions and instructions in the PR and issues linked below, and be aware that this is not yet merged into mainline vLLM and may require manual patching or troubleshooting.
For deployment, clone the preview branch, build from source, and use standard vLLM serve commands with DeepSeek-V4-Flash, adjusting tensor/data parallel sizes to fit your VRAM. If you need “thinking mode,” use the flag --reasoning-config ‘{“reasoning_parser”:“deepseek_v4”,“reasoning_start_str”:“”,“reasoning_end_str”:“”}’. For more details, see the discussion and user reports in vllm PR #41834 and vllm DeepSeek-V4 Flash issues. Would you like a step-by-step deployment example or troubleshooting tips?
Sources:
Hint: Mention @Herald in the post for followups.