# Vllm中，deepseek的模型 刷新kvcache的地方在哪

**URL:** <https://discuss.vllm.ai/t/vllm-deepseek-kvcache/2410>\
**Category:** General\
**Created:** [February 28, 2026, 8:36am UTC](https://discuss.vllm.ai/t/vllm-deepseek-kvcache/2410 "2026-02-28T08:36:34Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![yangshanjun](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/yangshanjun/32/900_2.png) [@yangshanjun](https://discuss.vllm.ai/u/yangshanjun)\
**Post date:** [February 28, 2026, 8:36am UTC](https://discuss.vllm.ai/t/vllm-deepseek-kvcache/2410/1 "2026-02-28T08:36:34Z")

</div>

vllm中，deepseek的模型 刷新kvcache的地方在哪

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [February 28, 2026, 8:36am UTC](https://discuss.vllm.ai/t/vllm-deepseek-kvcache/2410/2 "2026-02-28T08:36:53Z")

</div>

在vLLM中，Deepseek模型的KV cache（键值缓存）刷新主要由KV cache管理器负责。KV cache的分配和释放通常发生在推理过程中，例如在生成新token或处理新请求时，相关代码位于模型的forward方法和attention实现中。以DeepseekV2/DeepseekV3为例，KV cache的初始化和管理逻辑在`DeepseekV32IndexerCache`等类的构造函数和相关方法中实现，具体见[vllm.model\_executor.models.deepseek\_v2](https://docs.vllm.ai/en/latest/api/vllm/model_executor/models/deepseek_v2/#deepseekv32indexercache)和[源码实现](https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/models/deepseek_v2.py)。

KV cache的释放（即“刷新”）通常在推理引擎的调度器中进行，例如在请求结束或缓存空间不足时，相关逻辑可参考[KV cache block管理机制](https://docs.vllm.ai/en/latest/_images/overview.png)和[KV cache释放代码](https://github.com/vllm-project/vllm/blob/60dc62dc9e53428912953276e0d12a034b353fb6/vllm/engine/llm_engine.py#L429-L435)。KV cache不会每次请求都完全清空，而是按需分配和回收以提升性能。

需要更详细的源码分析或具体调用流程吗？

Sources:

- [源码实现](https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/models/deepseek_v2.py)
- [Figure](https://docs.vllm.ai/en/latest/design/v1/prefix_caching.html#data-structure)

Hint: Mention @RunLLM in the post for followups.

---

<div class="post-metadata">

**Author:** ![yangshanjun](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/yangshanjun/32/900_2.png) [@yangshanjun](https://discuss.vllm.ai/u/yangshanjun)\
**Post date:** [March 5, 2026, 9:25am UTC](https://discuss.vllm.ai/t/vllm-deepseek-kvcache/2410/3 "2026-03-05T09:25:03Z")

</div>

@RunLLM chunk size 切的越小，ttft的时间会越差吗

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [March 5, 2026, 9:25am UTC](https://discuss.vllm.ai/t/vllm-deepseek-kvcache/2410/4 "2026-03-05T09:25:29Z")

</div>

Chunk size 越小，TTFT（Time To First Token）通常会略高，因为prefill被分成更多小块，每块有额外的调度和内存开销；但小chunk有利于降低inter-token latency（ITL），提升解码响应速度。大chunk则TTFT更低，但可能拖慢ITL。最佳chunk size需在TTFT和ITL之间权衡，推荐在大模型和大GPU上设置较大（如8192或更高）以优化吞吐。[详见官方优化文档](https://docs.vllm.ai/en/latest/performance/optimization.html#performance-tuning-with-chunked-prefill)。

需要更详细的原理分析或调优建议吗？

Sources:

- [Performance Tuning with Chunked Prefill](https://docs.vllm.ai/en/latest/performance/optimization.html#performance-tuning-with-chunked-prefill)

Hint: Mention @RunLLM in the post for followups.

---

<div class="post-metadata">

**Author:** ![yangshanjun](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/yangshanjun/32/900_2.png) [@yangshanjun](https://discuss.vllm.ai/u/yangshanjun)\
**Post date:** [March 5, 2026, 9:26am UTC](https://discuss.vllm.ai/t/vllm-deepseek-kvcache/2410/5 "2026-03-05T09:26:36Z")

</div>

> [@yangshanjun](#):
>
> chunk size 切的越小，ttft的时间会越差吗

@RunLLM pp 并行，chunk size 一般会切成多少

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [March 5, 2026, 9:26am UTC](https://discuss.vllm.ai/t/vllm-deepseek-kvcache/2410/6 "2026-03-05T09:26:54Z")

</div>

chunk size 越小，TTFT（首token延迟）通常会变差，因为prefill阶段被分成更多小块，每块都需调度和内存访问，带来额外开销；但小chunk有利于降低ITL（token间延迟）。pp（pipeline parallel）并行下，chunk size 推荐根据硬件和模型规模动态调整，常见设置为2048~8192，大模型/大GPU可设更高（如16384），以平衡TTFT和吞吐。[详见官方优化建议](https://docs.vllm.ai/en/latest/performance/optimization.html#performance-tuning-with-chunked-prefill)。

是否需要详细的调优原理或实际案例说明？

Sources:

- [Performance Tuning with Chunked Prefill](https://docs.vllm.ai/en/latest/performance/optimization.html#performance-tuning-with-chunked-prefill)

Hint: Mention @RunLLM in the post for followups.

---

<div class="post-metadata">

**Author:** ![yangshanjun](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/yangshanjun/32/900_2.png) [@yangshanjun](https://discuss.vllm.ai/u/yangshanjun)\
**Post date:** [March 9, 2026, 8:58am UTC](https://discuss.vllm.ai/t/vllm-deepseek-kvcache/2410/7 "2026-03-09T08:58:30Z")

</div>

@RunLLM 如何进行vllm 的单元测试

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [March 9, 2026, 8:58am UTC](https://discuss.vllm.ai/t/vllm-deepseek-kvcache/2410/8 "2026-03-09T08:58:44Z")

</div>

vLLM 的单元测试使用 pytest 运行，命令为 `pytest tests/`。你可以只测试某个文件，例如 `pytest -s -v tests/test_logger.py`。[官方文档说明](https://docs.vllm.ai/en/latest/contributing/model/tests.html)和[社区答复](https://github.com/vllm-project/vllm/issues/30380)均推荐使用 pytest。

需要更详细的测试用例编写或测试环境配置说明吗？

Sources:

- [Writing Unit Tests — vLLM](https://docs.vllm.ai/en/latest/contributing/model/tests.html)
- [github issue 30380](https://github.com/vllm-project/vllm/issues/30380)

Hint: Mention @RunLLM in the post for followups.

---

<div class="post-metadata">

**Author:** ![yangshanjun](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/yangshanjun/32/900_2.png) [@yangshanjun](https://discuss.vllm.ai/u/yangshanjun)\
**Post date:** [March 10, 2026, 3:19am UTC](https://discuss.vllm.ai/t/vllm-deepseek-kvcache/2410/9 "2026-03-10T03:19:58Z")

</div>

@RunLLM Reqid: chatcmpl-9843d786c874b9a9-b6618c40, Total tokens 146, LMCache hit tokens: 0, need to load: -128 (vllm\_v1\_adapter.py:1286:lmcache.integration.vllm.vllm\_v1\_adapter)  
(APIServer pid=96924) INFO: 127.0.0.1:48472 - “POST /v1/chat/completions HTTP/1.1” 200 OK 这个打印正常吗

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [March 10, 2026, 3:20am UTC](https://discuss.vllm.ai/t/vllm-deepseek-kvcache/2410/10 "2026-03-10T03:20:16Z")

</div>

该日志中“need to load: -128”不正常，正常情况下“need to load”应为0或正数，表示需要从LMCache加载的token数。出现负数通常是调度或缓存命中逻辑有误，建议检查LMCache和vLLM的版本兼容性或相关实现。[详见官方LMCache适配器源码](https://github.com/vllm-project/vllm/blob/main/vllm/distributed/kv_transfer/kv_connector/v1/lmcache_integration/vllm_v1_adapter.py)。

需要进一步排查负数原因的详细分析吗？

Sources:

- [lmcache\_integration/vllm\_v1\_adapter.py](https://github.com/vllm-project/vllm/blob/main/vllm/distributed/kv_transfer/kv_connector/v1/lmcache_integration/vllm_v1_adapter.py)

Hint: Mention @RunLLM in the post for followups.

---

<div class="post-metadata">

**Author:** ![yangshanjun](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/yangshanjun/32/900_2.png) [@yangshanjun](https://discuss.vllm.ai/u/yangshanjun)\
**Post date:** [March 11, 2026, 5:46am UTC](https://discuss.vllm.ai/t/vllm-deepseek-kvcache/2410/11 "2026-03-11T05:46:55Z")

</div>

@RunLLM tp2pp4， rank6 没有走到 execute\_model 里面，可能是什么原因

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [March 11, 2026, 5:47am UTC](https://discuss.vllm.ai/t/vllm-deepseek-kvcache/2410/12 "2026-03-11T05:47:13Z")

</div>

在tp=2, pp=4的分布式配置下，如果rank6没有进入execute\_model，常见原因包括：进程间通信/调度异常、rank映射或初始化出错、部分worker未正确分配任务或死锁。建议检查分布式环境变量、rank分配、NCCL/Ray等通信初始化日志，并确认所有worker均已成功启动。[相关案例和调试建议见官方issue](https://github.com/vllm-project/vllm/issues/30128)。

需要详细的排查步骤或日志分析方法吗？

Sources:

- [github issue 30128](https://github.com/vllm-project/vllm/issues/30128)

Hint: Mention @RunLLM in the post for followups.

---

<div class="post-metadata">

**Author:** ![yangshanjun](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/yangshanjun/32/900_2.png) [@yangshanjun](https://discuss.vllm.ai/u/yangshanjun)\
**Post date:** [March 11, 2026, 5:57am UTC](https://discuss.vllm.ai/t/vllm-deepseek-kvcache/2410/13 "2026-03-11T05:57:30Z")

</div>

@RunLLM Executor负责把请求broadcast到各个workers上，这个广播的代码在哪里

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [March 11, 2026, 5:57am UTC](https://discuss.vllm.ai/t/vllm-deepseek-kvcache/2410/14 "2026-03-11T05:57:53Z")

</div>

Executor将请求广播到各个worker的核心代码在`MessageQueue`和`broadcast`相关实现中。以OmniDiffusion为例，调度器通过`self.mq.enqueue(requests)`实现广播，代码位于[vllm\_omni/diffusion/scheduler.py](https://github.com/vllm-project/vllm-omni/blob/main/docs/design/module/dit_module.md)和[官方设计文档](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/module/dit_module/#architecture)。对于主线vLLM，分布式executor会调用如`broadcast_tensor_dict`等方法，具体见[vllm.distributed.communication\_op](https://docs.vllm.ai/en/latest/genindex.html#broadcast_tensor_dict)。

需要更详细的源码路径或调用链说明吗？

Sources:

- [OmniDiffusion调度器设计](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/module/dit_module/#architecture)
- [dit\_module.md](https://github.com/vllm-project/vllm-omni/blob/main/docs/design/module/dit_module.md)

Hint: Mention @RunLLM in the post for followups.

---

<div class="post-metadata">

**Author:** ![yangshanjun](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/yangshanjun/32/900_2.png) [@yangshanjun](https://discuss.vllm.ai/u/yangshanjun)\
**Post date:** [March 11, 2026, 6:00am UTC](https://discuss.vllm.ai/t/vllm-deepseek-kvcache/2410/15 "2026-03-11T06:00:28Z")

</div>

@RunLLM

```
def step(self) -> tuple\[dict\[int, EngineCoreOutputs\], bool\]:

    """Schedule, execute, and make output.

    Returns tuple of outputs and a flag indicating whether the model

    was executed.

    """

    if envs.VLLM_ZERO_OVERHEAD:

        return engine_core_step(self)

    \# Check for any requests remaining in the scheduler - unfinished,

    \# or finished and not yet removed from the batch.

    if not self.scheduler.has_requests():

        return {}, False

    scheduler_output = self.scheduler.schedule()

    model_output = self.execute_model(scheduler_output)

    engine_core_outputs = self.scheduler.update_from_output(

        scheduler_output, model_output) # type: ignore

    return (engine_core_outputs,

            scheduler_output.total_num_scheduled_tokens > 0)

```

这里的 execute\_model， 会广播给所有worker 进程吗

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [March 11, 2026, 6:01am UTC](https://discuss.vllm.ai/t/vllm-deepseek-kvcache/2410/16 "2026-03-11T06:01:00Z")

</div>

是的，这里的`execute_model(scheduler_output)`会将模型执行请求广播给所有worker进程。具体实现中，Executor会通过如`collective_rpc`、`broadcast_tensor_dict`等机制，将`scheduler_output`分发到所有worker，确保每个worker都能参与模型推理。[相关源码说明](https://docs.vllm.ai/en/latest/api/vllm/vllm.executor.executor_base.html#vllm.executor.executor_base.DistributedExecutorBase.execute_model)。

需要详细的分布式广播流程或底层通信机制说明吗？

Sources:

- [https://github.com/vllm-project/vllm/issues/5775](https://github.com/vllm-project/vllm/issues/5775)
- [https://github.com/vllm-project/vllm/issues/32103](https://github.com/vllm-project/vllm/issues/32103)

Hint: Mention @RunLLM in the post for followups.

---

<div class="post-metadata">

**Author:** ![yangshanjun](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/yangshanjun/32/900_2.png) [@yangshanjun](https://discuss.vllm.ai/u/yangshanjun)\
**Post date:** [March 11, 2026, 9:28am UTC](https://discuss.vllm.ai/t/vllm-deepseek-kvcache/2410/17 "2026-03-11T09:28:18Z")

</div>

@RunLLM

vllm pp 并行，最后一个rank 执行的时候卡住，可能是什么原因

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [March 11, 2026, 9:28am UTC](https://discuss.vllm.ai/t/vllm-deepseek-kvcache/2410/18 "2026-03-11T09:28:45Z")

</div>

vLLM在pipeline parallel（pp）并行时，最后一个rank执行卡住，常见原因包括：进程间通信异常（如NCCL死锁）、部分rank提前退出或OOM、分布式环境变量配置不一致、NUMA跨节点通信瓶颈等。建议检查NCCL日志、各rank启动和分配情况，并确认CUDA/NCCL版本兼容。[相关案例和调试建议见官方issue](https://github.com/vllm-project/vllm/issues/26631)。

需要详细的排查步骤或日志分析方法吗？

Sources:

- [github issue 26631](https://github.com/vllm-project/vllm/issues/26631)

Hint: Mention @RunLLM in the post for followups.

---

<div class="post-metadata">

**Author:** ![yangshanjun](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/yangshanjun/32/900_2.png) [@yangshanjun](https://discuss.vllm.ai/u/yangshanjun)\
**Post date:** [March 11, 2026, 9:33am UTC](https://discuss.vllm.ai/t/vllm-deepseek-kvcache/2410/19 "2026-03-11T09:33:11Z")

</div>

@RunLLM

```auto
CUDA_LAUNCH_BLOCKING 这个环境变量是干什么的

```

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [March 11, 2026, 9:33am UTC](https://discuss.vllm.ai/t/vllm-deepseek-kvcache/2410/20 "2026-03-11T09:33:29Z")

</div>

`CUDA_LAUNCH_BLOCKING=1` 环境变量用于让CUDA每次kernel调用都同步执行，强制CPU等待GPU操作完成后再继续。这有助于定位CUDA错误的具体位置，便于调试和排查问题。[详见官方调试文档](https://docs.vllm.ai/en/latest/usage/troubleshooting/#enable-more-logging)。

需要更详细的原理说明或使用场景吗？

Sources:

- [https://github.com/vllm-project/vllm/issues/21731](https://github.com/vllm-project/vllm/issues/21731)
- [https://github.com/vllm-project/vllm/issues/3786](https://github.com/vllm-project/vllm/issues/3786)

Hint: Mention @RunLLM in the post for followups.

[Next page](https://discuss.vllm.ai/t/vllm-deepseek-kvcache/2410.md?page=2)