# V1 Feedback

**URL:** https://discuss.vllm.ai/c/v1-feedback/12.md

[Latest](https://discuss.vllm.ai/latest.md) · [Categories](https://discuss.vllm.ai/categories.md)

---

## [About the V1 Feedback category](https://discuss.vllm.ai/t/about-the-v1-feedback-category/78)

<div class="topic-metadata">

**Author:** [@youkaichao](https://discuss.vllm.ai/u/youkaichao)\
**Replies:** 0

</div>

---

## [Clarification: EP All-to-All Communication Across TP×DP — Diagram Validation](https://discuss.vllm.ai/t/clarification-ep-all-to-all-communication-across-tpxdp-diagram-validation/2479)

<div class="topic-metadata">

**Author:** [@DamodarYekkuluri](https://discuss.vllm.ai/u/DamodarYekkuluri)\
**Replies:** 1\
**Last updated:** [March 22, 2026, 12:15pm UTC](https://discuss.vllm.ai/t/clarification-ep-all-to-all-communication-across-tpxdp-diagram-validation/2479 "2026-03-22T12:15:27Z")

</div>

I’m trying to understand how vLLM implements Expert Parallelism. My current understanding: EP group spans TP × DP Experts are globally sharded Tokens are routed via all-to-all across all EP ranks I created this diagr…

---

## [vLLM v1 forces me to pre-allocate a huge, non-reclaimable GPU KV cache for long contexts, and none of the current offload or quantization options solve the resulting VRAM bloat without crippling speed.](https://discuss.vllm.ai/t/vllm-v1-forces-me-to-pre-allocate-a-huge-non-reclaimable-gpu-kv-cache-for-long-contexts-and-none-of-the-current-offload-or-quantization-options-solve-the-resulting-vram-bloat-without-crippling-speed/1502)

<div class="topic-metadata">

**Author:** [@abhiram1809](https://discuss.vllm.ai/u/abhiram1809)\
**Replies:** 5\
**Last updated:** [September 8, 2025, 6:35am UTC](https://discuss.vllm.ai/t/vllm-v1-forces-me-to-pre-allocate-a-huge-non-reclaimable-gpu-kv-cache-for-long-contexts-and-none-of-the-current-offload-or-quantization-options-solve-the-resulting-vram-bloat-without-crippling-speed/1502 "2025-09-08T06:35:56Z")

</div>

I’m running into a vLLM v1 limitation: for very long contexts (e.g. 128k tokens), the KV cache alone is ~20 GB, plus ~14 GB model weights and 2–3 GB runtime overhead, which forces me to reserve ~37 GB of VRAM. vLLM v1 re…

---

## [GLM4.5 V memory leak on inference](https://discuss.vllm.ai/t/glm4-5-v-memory-leak-on-inference/1471)

<div class="topic-metadata">

**Author:** [@VesVlad](https://discuss.vllm.ai/u/VesVlad)\
**Replies:** 9\
**Last updated:** [August 28, 2025, 9:04am UTC](https://discuss.vllm.ai/t/glm4-5-v-memory-leak-on-inference/1471 "2025-08-28T09:04:06Z")

</div>

Hi! While using vLLM versions 0.10.\*, I encountered the problem that when inference zai-org/GLM 4.5V models, the memory of my GPUs is rapidly clogging up. I am using 8 A100 and gpu\_memory\_utilisation=0.5 and max\_seq\_len=…

---

## [自定义多模态模型注册集成到vllm的问题一个问题反馈](https://discuss.vllm.ai/t/vllm/1327)

<div class="topic-metadata">

**Author:** [@sanjer](https://discuss.vllm.ai/u/sanjer)\
**Replies:** 1\
**Last updated:** [August 7, 2025, 10:13am UTC](https://discuss.vllm.ai/t/vllm/1327 "2025-08-07T10:13:16Z")

</div>

模型是私有的多模态嵌入模型（用于OCR识别的），通过注册插件机制已完成对接集成，在vLLM最新版本0.10.0（only V1）下基本正常，但还剩一个关于Prefixe Cahce下的问题： 现象描述： 1）关闭Prefix Cache情况下：相同的请求（prompt + 图片不变、temperature=0）推理结果是相同的且内容符合预期——模型注册集成到vLLM应该没大问题 2）但开启Prefix Cache情况下：只有第一次…

---

## [Cudagraph in V1](https://discuss.vllm.ai/t/cudagraph-in-v1/1016)

<div class="topic-metadata">

**Author:** [@eric\_tan](https://discuss.vllm.ai/u/eric_tan)\
**Replies:** 3\
**Last updated:** [July 21, 2025, 6:24pm UTC](https://discuss.vllm.ai/t/cudagraph-in-v1/1016 "2025-07-21T18:24:16Z")

</div>

I want to know in the v1 version of peicewise CUDA Graph, when it will be used and when it won’t. Can CUDA Graph be used for prefill?

---

## [vLLM V1 - Default max CUDA graph size](https://discuss.vllm.ai/t/vllm-v1-default-max-cuda-graph-size/357)

<div class="topic-metadata">

**Author:** [@cthi](https://discuss.vllm.ai/u/cthi)\
**Replies:** 2\
**Last updated:** [June 30, 2025, 7:42am UTC](https://discuss.vllm.ai/t/vllm-v1-default-max-cuda-graph-size/357 "2025-06-30T07:42:28Z")

</div>

Hi, I am doing some testing with vLLM V1. My model encountered an OOM during the CUDA graph capture. Upon debugging, I noticed that in vLLM V1, we cap the max CUDA graph size based off of max\_num\_batched\_tokens, which i…

---

## [Dose vllm V1 support asynchronous scheduling?](https://discuss.vllm.ai/t/dose-vllm-v1-support-asynchronous-scheduling/446)

<div class="topic-metadata">

**Author:** [@longer-is-better](https://discuss.vllm.ai/u/longer-is-better)\
**Replies:** 3\
**Last updated:** [June 27, 2025, 1:28pm UTC](https://discuss.vllm.ai/t/dose-vllm-v1-support-asynchronous-scheduling/446 "2025-06-27T13:28:18Z")

</div>

I think it’s still Serial execution of: schedule n → forward n → schedule n+1 → forward n+1 in vllm 0.7.3 version of vllm V1. dose this feature already implemented in vllm v0.7.3 or vllm v0.8.3?

---

## [V1 has lower end-to-end performance than V0(--num-scheduler-steps=8)](https://discuss.vllm.ai/t/v1-has-lower-end-to-end-performance-than-v0-num-scheduler-steps-8/795)

<div class="topic-metadata">

**Author:** [@lianjie](https://discuss.vllm.ai/u/lianjie)\
**Replies:** 6\
**Last updated:** [June 12, 2025, 7:07am UTC](https://discuss.vllm.ai/t/v1-has-lower-end-to-end-performance-than-v0-num-scheduler-steps-8/795 "2025-06-12T07:07:48Z")

</div>

Testing Environment: vllm version: 0.8.5 Operating System: Ubuntu 22.04 GPU: Nvidia A100 Performance Comparison: For Llama/qwen2 series models, the end-to-end inference performance of V1 is 5% to 10% lower than that…

---

## [\[Spec Decode\] Why does the acceptance rate look close to the paper, but the throughput is still not high?](https://discuss.vllm.ai/t/spec-decode-why-does-the-acceptance-rate-look-close-to-the-paper-but-the-throughput-is-still-not-high/499)

<div class="topic-metadata">

**Author:** [@wayne](https://discuss.vllm.ai/u/wayne)\
**Replies:** 1\
**Last updated:** [April 21, 2025, 6:29am UTC](https://discuss.vllm.ai/t/spec-decode-why-does-the-acceptance-rate-look-close-to-the-paper-but-the-throughput-is-still-not-high/499 "2025-04-21T06:29:00Z")

</div>

ori: bs 1 tp=1: \[05:55\<00:00, 4.45s/it, est. speed input: 17.02 toks/s, output: 46.43 toks/s with ealge: K=4 AL: 2.06 bs 1 tp=1 k=4: \[04:07\<00:00, 3.09s/it, est. speed input: 24.49 toks/s, output: 66.86 t…

---

## [Why V1 not support Mamba Models?](https://discuss.vllm.ai/t/why-v1-not-support-mamba-models/496)

<div class="topic-metadata">

**Author:** [@qscqesze](https://discuss.vllm.ai/u/qscqesze)\
**Replies:** 1\
**Last updated:** [April 21, 2025, 3:17am UTC](https://discuss.vllm.ai/t/why-v1-not-support-mamba-models/496 "2025-04-21T03:17:57Z")

</div>

Hi team, While reading the documentation, I noticed that V1 currently does not support Mamba models due to the use of non-standard attention mechanisms, such as selective state-space mechanisms. I’m curious — what would…

---

## [The new V1 way to ~--cpu-offload-gb](https://discuss.vllm.ai/t/the-new-v1-way-to-cpu-offload-gb/432)

<div class="topic-metadata">

**Author:** [@chris](https://discuss.vllm.ai/u/chris)\
**Replies:** 5\
**Last updated:** [April 13, 2025, 3:22pm UTC](https://discuss.vllm.ai/t/the-new-v1-way-to-cpu-offload-gb/432 "2025-04-13T15:22:33Z")

</div>

I attended the 8 April 2025 vLLM office hours. A lot of the talk sounded like topic(s) covered effectively supplant the need for v0’s --cpu-offload-gb param. For my 16GB vRAM Nvidia RTX 5070ti and my dummy/n00b status…

---

## [Does the vLLM v1 support Speculative Decoding now?](https://discuss.vllm.ai/t/does-the-vllm-v1-support-speculative-decoding-now/191)

<div class="topic-metadata">

**Author:** [@RookieChenTaoYu](https://discuss.vllm.ai/u/RookieChenTaoYu)\
**Replies:** 4\
**Last updated:** [March 25, 2025, 8:04am UTC](https://discuss.vllm.ai/t/does-the-vllm-v1-support-speculative-decoding-now/191 "2025-03-25T08:04:34Z")

</div>

In the “vllm/v1/core/scheduler.py” at the Scheduler.update\_from\_output() method, there is annotation says that “# NOTE: once we support N tokens per step (spec decode), the outer lists can be of length \> 1.”. Does this c…
