# Max\_model\_len vs GPU memory Usage **URL:** \ **Category:** General\ **Created:** [June 22, 2026, 5:49am UTC](https://discuss.vllm.ai/t/max-model-len-vs-gpu-memory-usage/2724 "2026-06-22T05:49:28Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![Ashhar-armada](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/ashhar-armada/32/1490_2.png) [@Ashhar-armada](https://discuss.vllm.ai/u/Ashhar-armada)\ **Post date:** [June 22, 2026, 5:49am UTC](https://discuss.vllm.ai/t/max-model-len-vs-gpu-memory-usage/2724/1 "2026-06-22T05:49:28Z")
Typically it is believed that reducing the `max_model_len` would help in reducing the GPU memory usage. But this fact was not observed while i tried deploying a model meta-llama-3.2-3b [https://build.nvidia.com/meta/llama-3.2-3b-instruct?nim=self-hosted](https://build.nvidia.com/meta/llama-3.2-3b-instruct?nim=self-hosted) from NIM model repo o n Tesla T4 GPU (16GB VRAM). Here I tried deploying the model using different values of `max_model_len` but the GPU memory usage remained constant. here are logs: Logs for `max_model_len: 8192` ```auto the current vLLM instance can use total_gpu_memory (14.56GiB) x gpu_memory_utilization (0.90) = 13.11GiB model weights take 6.02GiB; non_torch_memory takes 0.05GiB; PyTorch activation peak memory takes 1.22GiB; the rest of the memory reserved for KV Cache is 5.83GiB. INFO 2026-06-22 05:04:45.292 executor_base.py:111] # cuda blocks: 3409, # CPU blocks: 2340 INFO 2026-06-22 05:04:45.292 executor_base.py:116] Maximum concurrency for 8192 tokens per request: 6.66x ``` logs for `max_model_len: 4096` ```auto the current vLLM instance can use total_gpu_memory (14.56GiB) x gpu_memory_utilization (0.90) = 13.11GiB model weights take 6.02GiB; non_torch_memory takes 0.05GiB; PyTorch activation peak memory takes 1.19GiB; the rest of the memory reserved for KV Cache is 5.85GiB. INFO 2026-06-22 04:43:29.435 executor_base.py:111] # cuda blocks: 3422, # CPU blocks: 2340 INFO 2026-06-22 04:43:29.435 executor_base.py:116] Maximum concurrency for 4096 tokens per request: 13.37x ``` logs for `max_model_len: 16384` ```auto the current vLLM instance can use total_gpu_memory (14.56GiB) x gpu_memory_utilization (0.90) = 13.11GiB model weights take 6.02GiB; non_torch_memory takes 0.05GiB; PyTorch activation peak memory takes 1.26GiB; the rest of the memory reserved for KV Cache is 5.78GiB. INFO 2026-06-22 05:29:37.546 executor_base.py:111] # cuda blocks: 3381, # CPU blocks: 2340 INFO 2026-06-22 05:29:37.546 executor_base.py:116] Maximum concurrency for 16384 tokens per request: 3.30x ``` Now this fact is contrary to what we think that lowering the context length would help us save some GPU memory. Can you explain me the reason for such a behavior. ---
**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\ **Post date:** [June 22, 2026, 5:49am UTC](https://discuss.vllm.ai/t/max-model-len-vs-gpu-memory-usage/2724/2 "2026-06-22T05:49:44Z")
Reducing `max_model_len` in vLLM does not always reduce total GPU memory usage because vLLM pre-allocates GPU memory based on the formula: `(total memory * gpu_memory_utilization) - weights - maximum_activation_size`. The actual memory reserved for the KV cache is what remains after accounting for model weights and the peak activation memory measured during a profiling run with your chosen `max_model_len`. For small to moderate changes in `max_model_len`, the difference in activation memory is minor, so the overall GPU memory usage remains nearly constant—only the number of concurrent requests (concurrency) changes, not the total memory footprint. This is expected vLLM behavior and not a bug, as confirmed in [this discussion](https://github.com/vllm-project/vllm/issues/5416). Would you like a step-by-step breakdown of how vLLM calculates and allocates memory for different `max_model_len` values? Sources: - [https://github.com/vllm-project/vllm/issues/5416](https://github.com/vllm-project/vllm/issues/5416) Hint: Mention @Herald in the post for followups. ---
**Author:** ![Ashhar-armada](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/ashhar-armada/32/1490_2.png) [@Ashhar-armada](https://discuss.vllm.ai/u/Ashhar-armada)\ **Post date:** [June 22, 2026, 5:57am UTC](https://discuss.vllm.ai/t/max-model-len-vs-gpu-memory-usage/2724/3 "2026-06-22T05:57:44Z")
@RunLLM yes, help me understand why lowering the max\_model\_len doesnt help us reduce the gpu usage, and why is this rule used to state that it will help in reducing the GPU usage, is it related to some version of vllm?