# GLM4.5 V memory leak on inference **URL:** \ **Category:** V1 Feedback\ **Created:** [August 28, 2025, 8:36am UTC](https://discuss.vllm.ai/t/glm4-5-v-memory-leak-on-inference/1471 "2025-08-28T08:36:16Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![VesVlad](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/vesvlad/32/656_2.png) [@VesVlad](https://discuss.vllm.ai/u/VesVlad)\ **Post date:** [August 28, 2025, 8:36am UTC](https://discuss.vllm.ai/t/glm4-5-v-memory-leak-on-inference/1471/1 "2025-08-28T08:36:16Z")
Hi! While using vLLM versions 0.10.\*, I encountered the problem that when inference zai-org/GLM 4.5V models, the memory of my GPUs is rapidly clogging up. I am using 8 A100 and gpu\_memory\_utilisation=0.5 and max\_seq\_len=1, but the memory is still clogged. Studying this issue, the most effective solution in terms of stability, but not speed, turned out to be using the VLLM\_USE\_V1=0 flag. The memory is also clogged, but there is no cuda out of memory. Maybe there is a known way to overcome this problem without using VLLM\_USE\_V1=0. My current script: ![image](https://canada1.discourse-cdn.com/flex036/uploads/vllm/original/1X/82171e5f2e523bf7ca88409e2265a1ef95f2efb1.png) ---
**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\ **Post date:** [August 28, 2025, 8:37am UTC](https://discuss.vllm.ai/t/glm4-5-v-memory-leak-on-inference/1471/2 "2025-08-28T08:37:10Z")
vLLM V1 (VLLM\_USE\_V1=1) is known to use more GPU memory due to CUDA graph compilation and increased KV cache allocation, especially with large models like GLM-4.5V, even when `gpu_memory_utilization` is set low and `max_seq_len` is minimal. This can lead to rapid memory consumption and OOM errors. Disabling V1 (VLLM\_USE\_V1=0) improves stability but at the cost of speed, as you observed. There is currently no fully effective workaround in V1 other than reducing `max_num_seqs`, `max_model_len`, or using `enforce_eager=True` to disable CUDA graphs, which also impacts performance. Adjusting these parameters may help, but memory usage will still be higher than V0. This is a known issue, and the only reliable solution for now is to use V0 for stability, as confirmed by multiple users and maintainers ([source](https://github.com/vllm-project/vllm/issues/18582), [source](https://github.com/vllm-project/vllm/issues/15617), [source](https://github.com/vllm-project/vllm/issues/16342), [source](https://docs.vllm.ai/en/latest/configuration/conserving_memory.html)). If you want to experiment, try lowering `max_num_seqs`, `max_model_len`, and/or set `enforce_eager=True` in your config to reduce memory usage, but expect a trade-off in throughput and latency. There is no parameter or flag in the data sources that fully resolves the V1 memory spike for large models without these trade-offs. Would you like a step-by-step breakdown or code example for these adjustments? Sources: - [Conserving Memory - vLLM Docs](https://docs.vllm.ai/en/latest/configuration/conserving_memory.html) - [Upgrading from vllm0.7.3 to vllm0.8.2, but the required GPU memory significantly increases.](https://github.com/vllm-project/vllm/issues/15617) - [https://github.com/vllm-project/vllm/issues/18582](https://github.com/vllm-project/vllm/issues/18582) - [https://github.com/vllm-project/vllm/issues/16342](https://github.com/vllm-project/vllm/issues/16342) ---
**Author:** ![DarkLight1337](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/darklight1337/32/570_2.png) [@DarkLight1337](https://discuss.vllm.ai/u/DarkLight1337)\ **Post date:** [August 28, 2025, 8:40am UTC](https://discuss.vllm.ai/t/glm4-5-v-memory-leak-on-inference/1471/3 "2025-08-28T08:40:33Z")
Can you elaborate on what you mean by “clogging up memory”? You can try disabling CUDA graph as suggested by RunLLM first. ---
**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\ **Post date:** [August 28, 2025, 8:41am UTC](https://discuss.vllm.ai/t/glm4-5-v-memory-leak-on-inference/1471/4 "2025-08-28T08:41:03Z")
“Clogging up memory” refers to vLLM rapidly allocating most or all available GPU memory, often leading to out-of-memory (OOM) errors or leaving little room for other processes. This is common in vLLM V1 due to aggressive KV cache allocation and CUDA graph compilation, which both consume significant memory—even with low `gpu_memory_utilization` and minimal `max_seq_len` ([source](https://github.com/vllm-project/vllm/issues/18582), [source](https://docs.vllm.ai/en/latest/configuration/conserving_memory.html)). Disabling CUDA graph (by setting `enforce_eager=True` in your LLM config or using the `--enforce-eager` CLI flag) can reduce memory usage, as CUDA graph compilation is a major contributor to the memory spike in V1 ([source](https://docs.vllm.ai/en/latest/configuration/conserving_memory.html), [source](https://github.com/vllm-project/vllm/issues/16342)). Would you like a code example for disabling CUDA graph or further details on memory allocation in vLLM? Sources: - [Conserving Memory - vLLM Docs](https://docs.vllm.ai/en/latest/configuration/conserving_memory.html) - [https://github.com/vllm-project/vllm/issues/18582](https://github.com/vllm-project/vllm/issues/18582) - [https://github.com/vllm-project/vllm/issues/16342](https://github.com/vllm-project/vllm/issues/16342) ---
**Author:** ![VesVlad](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/vesvlad/32/656_2.png) [@VesVlad](https://discuss.vllm.ai/u/VesVlad)\ **Post date:** [August 28, 2025, 8:46am UTC](https://discuss.vllm.ai/t/glm4-5-v-memory-leak-on-inference/1471/5 "2025-08-28T08:46:43Z")
> [@DarkLight1337](#): > > Can you elaborate on what you mean by “clogging up memory”? You can try disabling CUDA graph as suggested by RunLLM first. The 40GB free space on each GPU will be completely used up on the inference after loading the model. ---
**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\ **Post date:** [August 28, 2025, 8:47am UTC](https://discuss.vllm.ai/t/glm4-5-v-memory-leak-on-inference/1471/6 "2025-08-28T08:47:18Z")
“Clogging up memory” here means that after loading the model, vLLM quickly allocates all available GPU memory (e.g., the full 40GB on each A100), leaving no free space for other processes or additional inference tasks. This is due to vLLM’s static allocation of memory for the model weights, KV cache, and (in V1) CUDA graph compilation, which can consume nearly all GPU resources regardless of actual inference needs ([source](https://docs.vllm.ai/en/latest/configuration/conserving_memory.html), [source](https://github.com/vllm-project/vllm/issues/4253)). Disabling CUDA graph (by setting `enforce_eager=True`) can reduce this memory pressure, as CUDA graph compilation is a major source of extra allocation in V1. Would you like a code example for this adjustment? Sources: - [Conserving Memory - vLLM Docs](https://docs.vllm.ai/en/latest/configuration/conserving_memory.html) - [https://github.com/vllm-project/vllm/issues/4253](https://github.com/vllm-project/vllm/issues/4253) ---
**Author:** ![DarkLight1337](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/darklight1337/32/570_2.png) [@DarkLight1337](https://discuss.vllm.ai/u/DarkLight1337)\ **Post date:** [August 28, 2025, 8:57am UTC](https://discuss.vllm.ai/t/glm4-5-v-memory-leak-on-inference/1471/7 "2025-08-28T08:57:50Z")
What parameters did you pass when using V1 Engine? ---
**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\ **Post date:** [August 28, 2025, 8:58am UTC](https://discuss.vllm.ai/t/glm4-5-v-memory-leak-on-inference/1471/8 "2025-08-28T08:58:14Z")
The main parameters affecting memory usage in vLLM V1 Engine are `gpu_memory_utilization`, `max_model_len`, `max_num_seqs`, and optionally `enforce_eager`. For example, you might use: ```python llm = LLM( model="your-model", gpu_memory_utilization=0.5, max_model_len=1, max_num_seqs=1, enforce_eager=True # to disable CUDA graph and reduce memory usage ) ``` or via CLI: ```auto python -m vllm.entrypoints.openai.api_server --model your-model --gpu-memory-utilization 0.5 --max-model-len 1 --max-num-seqs 1 --enforce-eager ``` These parameters control how much GPU memory is pre-allocated and how large the context/batch can be ([source](https://docs.vllm.ai/en/latest/configuration/conserving_memory.html), [source](https://docs.vllm.ai/en/latest/cli/index.html#-block-size)). Would you like more details on each parameter? Sources: - [Conserving Memory - vLLM Docs](https://docs.vllm.ai/en/latest/configuration/conserving_memory.html) - [CLI Reference - vLLM Docs](https://docs.vllm.ai/en/latest/cli/index.html#-block-size) ---
**Author:** ![VesVlad](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/vesvlad/32/656_2.png) [@VesVlad](https://discuss.vllm.ai/u/VesVlad)\ **Post date:** [August 28, 2025, 9:03am UTC](https://discuss.vllm.ai/t/glm4-5-v-memory-leak-on-inference/1471/9 "2025-08-28T09:03:30Z")
```auto --tensor-parallel-size 4 --pipeline-parallel-size 2 --gpu-memory-utilization 0.5 --swap-space 32 --max-num-seqs 1 --disable-mm-preprocessor-cache --enable-expert-parallel --enforce-eager (I'm trying it now) ``` ---
**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\ **Post date:** [August 28, 2025, 9:04am UTC](https://discuss.vllm.ai/t/glm4-5-v-memory-leak-on-inference/1471/10 "2025-08-28T09:04:06Z")
Your parameters are valid and align with vLLM’s documented options for controlling memory and parallelism. Using `--enforce-eager` disables CUDA graph, which should help reduce memory usage in V1, though it may lower throughput ([source](https://docs.vllm.ai/en/latest/configuration/conserving_memory.html)). The other flags (`--tensor-parallel-size`, `--pipeline-parallel-size`, `--gpu-memory-utilization`, `--swap-space`, `--enable-expert-parallel`) are all supported and affect how the model is distributed and how much memory is reserved ([source](https://docs.vllm.ai/en/latest/cli/index.html#-block-size)). If memory is still fully consumed, further reduction of `--max-model-len` or `--max-num-seqs` may be necessary. Would you like a breakdown of how each parameter impacts memory? Sources: - [Conserving Memory - vLLM Docs](https://docs.vllm.ai/en/latest/configuration/conserving_memory.html)