# What is Included in -gpu-memory-utilization

**URL:** <https://discuss.vllm.ai/t/what-is-included-in-gpu-memory-utilization/2559>\
**Category:** General\
**Created:** [April 17, 2026, 3:07am UTC](https://discuss.vllm.ai/t/what-is-included-in-gpu-memory-utilization/2559 "2026-04-17T03:07:03Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Pigcanflysohigh](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/pigcanflysohigh/32/702_2.png) [@Pigcanflysohigh](https://discuss.vllm.ai/u/Pigcanflysohigh)\
**Post date:** [April 17, 2026, 3:07am UTC](https://discuss.vllm.ai/t/what-is-included-in-gpu-memory-utilization/2559/1 "2026-04-17T03:07:03Z")

</div>

What is Included in gpu-memory-utilization?

I know model weight and KVcache are included.And is there anything others is included?

one more question, if the kv-cache is too large, what will happen? for example, 100G GPU memroy is set for weight and KVcache,but one time the KVcache is too large,then overhead 100G, what will happen?

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [April 17, 2026, 3:07am UTC](https://discuss.vllm.ai/t/what-is-included-in-gpu-memory-utilization/2559/2 "2026-04-17T03:07:20Z")

</div>

The gpu\_memory\_utilization parameter in vLLM controls the fraction of total GPU memory reserved for model weights, KV cache, activation memory (intermediate tensors), and system overhead (e.g., CUDA, PyTorch, NCCL, CUDA graphs). It is not just model weights and KV cache; activation buffers and non-torch memory are also included in this allocation. See [docs](https://docs.vllm.ai/projects/vllm-omni/en/latest/configuration/gpu_memory_utilization/#how-memory-is-calculated) and [API reference](https://docs.vllm.ai/en/latest/api/vllm/entrypoints/llm/#vllm.entrypoints.llm.LLM).

If the KV cache is too large and exceeds the reserved memory (e.g., you set gpu\_memory\_utilization so high that the sum of weights, activations, and KV cache needs more than available GPU memory), you will get an out-of-memory (OOM) error and the process will fail. vLLM does not automatically spill excess KV cache to CPU unless you explicitly configure offloading; otherwise, OOM occurs. See [troubleshooting](https://docs.vllm.ai/projects/vllm-omni/en/latest/configuration/gpu_memory_utilization/#troubleshooting) and [issue discussion](https://github.com/vllm-project/vllm/issues/1298).

Would you like more detail on memory profiling or how to tune these parameters?

Sources:

- [https://docs.vllm.ai/en/latest/api/vllm/entrypoints/llm/#vllm.entrypoints.llm.LLM](https://docs.vllm.ai/en/latest/api/vllm/entrypoints/llm/#vllm.entrypoints.llm.LLM)
- [https://github.com/vllm-project/vllm/issues/1298](https://github.com/vllm-project/vllm/issues/1298)

Hint: Mention @RunLLM in the post for followups.

---

<div class="post-metadata">

**Author:** ![Pigcanflysohigh](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/pigcanflysohigh/32/702_2.png) [@Pigcanflysohigh](https://discuss.vllm.ai/u/Pigcanflysohigh)\
**Post date:** [April 24, 2026, 3:48am UTC](https://discuss.vllm.ai/t/what-is-included-in-gpu-memory-utilization/2559/3 "2026-04-24T03:48:45Z")

</div>

but vllm allow KV cache grown with no limited?

---

<div class="post-metadata">

**Author:** ![Pigcanflysohigh](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/pigcanflysohigh/32/702_2.png) [@Pigcanflysohigh](https://discuss.vllm.ai/u/Pigcanflysohigh)\
**Post date:** [April 24, 2026, 8:39am UTC](https://discuss.vllm.ai/t/what-is-included-in-gpu-memory-utilization/2559/4 "2026-04-24T08:39:25Z")

</div>

why don’t you answer my question?

---

<div class="post-metadata">

**Author:** ![Pigcanflysohigh](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/pigcanflysohigh/32/702_2.png) [@Pigcanflysohigh](https://discuss.vllm.ai/u/Pigcanflysohigh)\
**Post date:** [April 24, 2026, 8:39am UTC](https://discuss.vllm.ai/t/what-is-included-in-gpu-memory-utilization/2559/5 "2026-04-24T08:39:58Z")

</div>

Does vLLM permit the KV cache to expand without restriction?
