# Decode KV writeback: can multi-turn chat preserve cache reuse with default templates? **URL:** \ **Category:** KV-Cache\ **Created:** [September 10, 2026, 1:50am UTC](https://discuss.vllm.ai/t/decode-kv-writeback-can-multi-turn-chat-preserve-cache-reuse-with-default-templates/2955 "2026-09-10T01:50:24Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![lhecust](https://avatars.discourse-cdn.com/v4/letter/l/ce73a5/32.png) [@lhecust](https://discuss.vllm.ai/u/lhecust)\ **Post date:** [September 10, 2026, 1:50am UTC](https://discuss.vllm.ai/t/decode-kv-writeback-can-multi-turn-chat-preserve-cache-reuse-with-default-templates/2955/1 "2026-09-10T01:50:24Z")
Hi vLLM community, I would like to understand the practical benefits of Decode-phase KV writeback when server-side chat templates change the token sequence between conversation turns. This relates to the **Decode-Phase KV Cache Put** item in the Mooncake Store Connector roadmap #45036 and the related work in #52466, #53129, and #54307. - #52466, merged: adds Decode KV offloading, including consumer-side writeback with `save_decode_cache`. - #53129, merged: enables heterogeneous-TP Store sharing, including Decode writeback into a common Store layout. - #54307, currently open: extends heterogeneous-TP sharing to hybrid KV caches. Assume the client faithfully replays all conversation history, including reasoning where applicable. The concern is what happens **after vLLM receives those messages and applies the chat template**. I tested the process with Qwen3. The result is as follower: ### **Non-thinking mode: Qwen3** The affected default template adds `\n\n\n\n` before generating an answer, but omits it when rendering that answer as historical assistant content in the next request: ```auto Turn 1 execution: System | User1 | \n\n\n\n | Answer1 Turn 2 input after template rendering: System | User1 | Answer1 | User2 | \n\n\n\n ``` ![qwen3_nonthinking_four_turn_kv_reuse](https://canada1.discourse-cdn.com/flex036/uploads/vllm/original/2X/9/90560938620be4af66d426088157f5003f386ea4.png) This behavior is also reported in [Qwen3 issue #1826](https://github.com/QwenLM/Qwen3/issues/1826). ### **Thinking mode: historical reasoning removed** For Qwen3 and DeepSeek V4 configurations that discard historical reasoning, the template can remove the reasoning even when the client supplies it: ```auto Turn 1 execution: System | User1 | Reasoning1 | Answer1 Turn 2 input after template rendering: System | User1 | Answer1 | User2 | ... ``` DeepSeek V4 provides `drop_thinking`, with retention options and tool-use exceptions; this example concerns the path where historical reasoning is dropped. See the [official encoding reference](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash/blob/main/encoding/README.md). ![thinking_four_turn_kv_reuse](https://canada1.discourse-cdn.com/flex036/uploads/vllm/original/2X/2/289d0bd1bb019f0f351f0a70e66bed6a126fd338.png) In both cases, Decode may successfully save KV, but the next Prefill receives a different token prefix. Complete blocks before the first difference can still hit; the affected block and subsequent prefix-dependent blocks cannot reuse that Decode execution’s KV. Identical final-answer text is insufficient because its KV was computed with a different preceding context. ### **Questions** 1. **Does vLLM provide, or plan to provide, a server-side path that preserves the previous Decode token prefix when rendering subsequent chat turns?** With complete history replay, can enabling Decode KV writeback lead to next-turn reuse under default templates, or are template/configuration changes a prerequisite? 2. **What next-turn reuse and performance benefits have been demonstrated with Decode writeback enabled for these models?** In particular, do the demonstrated results in PR include normal multi-turn Chat API requests processed by the default templates, with cache hits extending into the previous answer’s generated tokens? 3. **When historical reasoning is intentionally removed server-side, what useful scenarios remain for Decode KV writeback?** If ordinary next-turn reuse is limited, which supported workloads are expected to benefit from saving those generated KV blocks? I would appreciate clarification on the intended server-side behavior and the conditions under which Decode KV writeback delivers practical benefits. ---
**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\ **Post date:** [September 10, 2026, 1:50am UTC](https://discuss.vllm.ai/t/decode-kv-writeback-can-multi-turn-chat-preserve-cache-reuse-with-default-templates/2955/2 "2026-09-10T01:50:44Z")
Decode-phase KV writeback only enables next-turn reuse if the _exact_ token sequence—including all template-injected tokens like `` or reasoning—is preserved between turns. With default Qwen3 and DeepSeek V4 templates, server-side chat rendering often strips or alters these tokens (e.g., omitting `` or dropping reasoning), so the next turn’s prompt diverges from the previous decode’s output. As a result, only the prefix up to the first difference can hit the cache; answer tokens computed after a changed prefix (like a missing ``) cannot be reused, and Decode KV writeback provides little or no benefit in standard multi-turn chat flows using these templates. This is confirmed in [Qwen3 issue #1826](https://github.com/QwenLM/Qwen1/issues/1826) and the [DeepSeek encoding reference](https://github.com/deepseek-ai/DeepSeek-V2/blob/main/encoding/encoding_dsv4.py). vLLM does not currently provide a server-side mechanism to force preservation of the previous decode’s token prefix in subsequent turns under default templates. Practical reuse of Decode-phase KV blocks across turns requires either (a) a template/configuration that faithfully replays all generated tokens (including reasoning/thinking markers), or (b) a custom workflow where the client reconstructs the _exact_ prior output as input. Demonstrated performance benefits in PRs like [#52466](https://github.com/vllm-project/vllm/pull/52466) and [#53129](https://github.com/vllm-project/vllm/pull/53129) are limited to scenarios where such exact replay is possible; for standard multi-turn Chat API usage with default templates, cache hits do **not** extend into previous answers’ generated tokens. When historical reasoning is intentionally removed, Decode KV writeback is only useful for specialized workloads (e.g., tool-calling, agentic flows, or custom templates that preserve all tokens). For most default chat scenarios, its practical benefit is minimal unless template or client behavior is changed. Would you like a step-by-step breakdown of the cache hit logic or example code for custom template handling? Sources: - [Discussion of template-induced hash divergence](https://github.com/vllm-project/vllm/issues/39321) - [Decode KV writeback and template mismatch](https://github.com/vllm-project/vllm/issues/43094) Hint: Mention @Herald in the post for followups.