# vLLM Load balancing

**URL:** <https://discuss.vllm.ai/t/vllm-load-balancing/222>\
**Category:** General\
**Created:** [March 24, 2025, 7:05pm UTC](https://discuss.vllm.ai/t/vllm-load-balancing/222 "2025-03-24T19:05:13Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![prasadwrites](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/prasadwrites/32/65_2.png) [@prasadwrites](https://discuss.vllm.ai/u/prasadwrites)\
**Post date:** [March 24, 2025, 7:05pm UTC](https://discuss.vllm.ai/t/vllm-load-balancing/222/1 "2025-03-24T19:05:13Z")

</div>

Does vLLM come with its own load balancer. Whats the recommended way of load balancing an inference endpoint with vllm

---

<div class="post-metadata">

**Author:** ![comaniac](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/comaniac/32/5_2.png) [@comaniac](https://discuss.vllm.ai/u/comaniac)\
**Post date:** [March 24, 2025, 7:12pm UTC](https://discuss.vllm.ai/t/vllm-load-balancing/222/2 "2025-03-24T19:12:38Z")

</div>

The load balancers currently lie outside of the vLLM repo and are more like a component on top of it. Available options AFAIK are:

- vLLM Production Stack: [GitHub - vllm-project/production-stack: vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization](https://github.com/vllm-project/production-stack)
- AIBrix: [GitHub - vllm-project/aibrix: Cost-efficient and pluggable Infrastructure components for GenAI inference](https://github.com/vllm-project/aibrix)
- Ray Serve: [Serving LLMs — Ray 3.0.0.dev0](https://docs.ray.io/en/master/serve/llm/serving-llms.html)