# Does vllm support deploy multiple docker instance on one gpu

**URL:** <https://discuss.vllm.ai/t/does-vllm-support-deploy-multiple-docker-instance-on-one-gpu/657>\
**Category:** General\
**Created:** [May 21, 2025, 9:40am UTC](https://discuss.vllm.ai/t/does-vllm-support-deploy-multiple-docker-instance-on-one-gpu/657 "2025-05-21T09:40:40Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![781574155](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/781574155/32/308_2.png) [@781574155](https://discuss.vllm.ai/u/781574155)\
**Post date:** [May 21, 2025, 9:40am UTC](https://discuss.vllm.ai/t/does-vllm-support-deploy-multiple-docker-instance-on-one-gpu/657/1 "2025-05-21T09:40:40Z")

</div>

I have an Nvidia H100, I want to deploy multiple small model on this GPU.  
I deploy an qwen2.5-omni-7b with gpu\_memory\_utilization = 0.5.  
I try to deploy another qwen3-4b with gpu\_memory\_utilization = 0.4. Error occures:

> ValueError: No available memory for the cache blocks. Try increasing `gpu_memory_utilization` when initializing the engine.

If I stop qwen2.5-omni-7b and only deploy qwen3-4b, gpu\_memory\_utilization = 0.25 is enough

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [May 21, 2025, 9:41am UTC](https://discuss.vllm.ai/t/does-vllm-support-deploy-multiple-docker-instance-on-one-gpu/657/2 "2025-05-21T09:41:07Z")

</div>

This error occurs because vLLM’s `gpu_memory_utilization` parameter is a per-instance limit, but the memory calculation does not account for memory already used by other vLLM instances on the same GPU. As a result, if you start one model and then another, the second instance may see less available memory than expected, leading to the “No available memory for the cache blocks” error—even if the sum of `gpu_memory_utilization` values is less than 1.0. This is a known issue, and the workaround is to stagger the startup of the services (wait 30 seconds between launches) or increase the `gpu_memory_utilization` for the second instance to account for memory already used by the first instance. See the discussion and workaround in [this issue](https://github.com/vllm-project/vllm/issues/13633).

If you stop the first model, the second can start with a lower `gpu_memory_utilization` because all GPU memory is available. When running multiple vLLM instances, you may need to set a higher `gpu_memory_utilization` for the later-started instance, or experiment with staggered startups, as described above. Would you like more detail on how to tune these parameters or on the underlying memory allocation behavior?

Sources:

- [Feature: Support multiple models per GPU](https://github.com/vllm-project/vllm/issues/13633)