# vLLM TurboQuant Implementation

**URL:** <https://discuss.vllm.ai/t/vllm-turboquant-implementation/2944>\
**Category:** Quantization\
**Created:** [September 7, 2026, 3:36pm UTC](https://discuss.vllm.ai/t/vllm-turboquant-implementation/2944 "2026-09-07T15:36:02Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![TejasNC](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/tejasnc/32/1710_2.png) [@TejasNC](https://discuss.vllm.ai/u/TejasNC)\
**Post date:** [September 7, 2026, 3:36pm UTC](https://discuss.vllm.ai/t/vllm-turboquant-implementation/2944/1 "2026-09-07T15:36:03Z")

</div>

I came across this [A First Comprehensive Study of TurboQuant: Accuracy and Performance | vLLM Blog](https://vllm.ai/blog/2026-05-11-turboquant) which compares fp8 kv quantization against various TQ configs in vLLM.

On looking into the code, I figure that both the implementations have different backends. vLLM fp8 KV Cache Quantization is in the Flash Attention Backend, which has highly optimized CUDA Kernels for CUDA GPUs. TurboQuant is implemented in a separate TurboQuant Backend, which has Triton kernels.

So the comparison does not seem to be fair. Am I missing something here?

Moreover, are there planned efforts to implement TQ in Flash Attention backend.

Thanks

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [September 7, 2026, 3:36pm UTC](https://discuss.vllm.ai/t/vllm-turboquant-implementation/2944/2 "2026-09-07T15:36:30Z")

</div>

You are correct: in vLLM, FP8 KV cache quantization is implemented in the Flash Attention backend (using highly optimized CUDA kernels), while TurboQuant (TQ) is implemented in a separate TurboQuant backend using Triton kernels. This means the performance and throughput comparisons in the referenced blog post are not strictly apples-to-apples, as each backend has different kernel optimizations and architectural tradeoffs. The blog itself notes that FP8 is the best default for most workloads, partly due to its mature, highly optimized backend, while TurboQuant offers higher compression but with a throughput penalty due to its more complex dequantization path and less mature kernel optimizations ([vLLM TurboQuant blog](https://vllm.ai/blog/2026-05-11-turboquant#reasoning)).

As of now, there is no explicit mention in the official documentation or codebase of planned efforts to implement TurboQuant directly within the Flash Attention backend for CUDA GPUs. TurboQuant remains a standalone backend, and the current design isolates it from the Flash Attention path ([TurboQuantAttentionBackend source](https://docs.vllm.ai/en/latest/api/vllm/v1/attention/backends/turboquant_attn/)). If you want more detail on backend selection or future roadmap, let me know.

Sources:

- [TurboQuantAttentionBackend source](https://docs.vllm.ai/en/latest/api/vllm/v1/attention/backends/turboquant_attn/)
- [vLLM TurboQuant blog](https://vllm.ai/blog/2026-05-11-turboquant#reasoning)

Hint: Mention @Herald in the post for followups.