# TurboQuant: KV Cache Compression

**URL:** <https://discuss.vllm.ai/t/turboquant-kv-cache-compression/2503>\
**Category:** General\
**Created:** [March 31, 2026, 7:48am UTC](https://discuss.vllm.ai/t/turboquant-kv-cache-compression/2503 "2026-03-31T07:48:19Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![ghazal-bh](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/ghazal-bh/32/1284_2.png) [@ghazal-bh](https://discuss.vllm.ai/u/ghazal-bh)\
**Post date:** [March 31, 2026, 7:48am UTC](https://discuss.vllm.ai/t/turboquant-kv-cache-compression/2503/1 "2026-03-31T07:48:19Z")

</div>

Hello, I would like to know that is there any plan to have a new version that can accept TurboQuant in –quantization parameter?

Thanks for your support.

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [March 31, 2026, 7:48am UTC](https://discuss.vllm.ai/t/turboquant-kv-cache-compression/2503/2 "2026-03-31T07:48:35Z")

</div>

TurboQuant support is being actively developed for vLLM, but as of now, TurboQuant is not available as a value for the --quantization parameter. Instead, TurboQuant is being integrated as a kv-cache quantization option via the --kv-cache-dtype parameter (e.g., --kv-cache-dtype turboquant), not as a general --quantization method for model weights. The mainline vLLM codebase does not yet support TurboQuant in --quantization, but there are open pull requests and ongoing work to add TurboQuant for KV cache compression, with benchmarks and implementation details available in the development discussions and PRs[^1][^2].

Would you like more detail on the current status, usage, or roadmap for TurboQuant in vLLM?

Sources:

- [Feature: Add TurboQuant Support for KV Cache Quantization](https://github.com/vllm-project/vllm/issues/38171)
- [PR: Add TurboQuant](https://github.com/vllm-project/vllm/pull/38280)

Hint: Mention @RunLLM in the post for followups.

---

<div class="post-metadata">

**Author:** ![uameer](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/uameer/32/1288_2.png) [@uameer](https://discuss.vllm.ai/u/uameer)\
**Post date:** [April 11, 2026, 1:53am UTC](https://discuss.vllm.ai/t/turboquant-kv-cache-compression/2503/3 "2026-04-11T01:53:46Z")

</div>

I’ve been modeling a 0.12-bit entropy-gated exit protocol to address the quadratic compute waste in 1T+ models. It solves the CUDA sync penalty via speculative branching. Would this fit into the current roadmap for PagedAttention optimization?