Hi All,
I wonder what differences exist between LLM-Scaler and vanilla vllm…
At the moment, with my dual ARC B60 setup, I cannot even get gemma4 to run with LLM-Scaler.
I hoped for some performance improvements.
Is there any experience regarding the performance differences?
Thanks
LLM-Scaler is Intel’s downstream fork of vLLM, tailored for Intel GPUs (XPU), with additional patches, hardware-specific optimizations, and support for Intel’s software stack. Vanilla vLLM (the upstream project) primarily targets NVIDIA GPUs and may lack full support or optimizations for Intel hardware. For Intel Arc B-series GPUs, LLM-Scaler is recommended, as the standard vLLM image is CUDA-only and does not include Intel GPU support—using LLM-Scaler resolves compatibility and stability issues on Intel GPUs, as confirmed by multiple users who achieved stable, high-throughput operation only with Intel’s container (see discussion).
Regarding performance, LLM-Scaler should match or exceed vanilla vLLM on Intel hardware due to its optimizations, but actual speedups depend on the model, quantization, and kernel support. Some users report that certain models (like Gemma4) may still have backend or kernel issues on Intel GPUs, even with LLM-Scaler, and performance can lag behind NVIDIA setups or be limited by incomplete kernel support (see also). Would you like a detailed breakdown of specific performance metrics or troubleshooting steps for Gemma4 on Arc B60?
Sources:
Hint: Mention @Herald in the post for followups.