# Does the vLLM v1 support Speculative Decoding now?

**URL:** <https://discuss.vllm.ai/t/does-the-vllm-v1-support-speculative-decoding-now/191>\
**Category:** V1 Feedback\
**Created:** [March 22, 2025, 11:50am UTC](https://discuss.vllm.ai/t/does-the-vllm-v1-support-speculative-decoding-now/191 "2025-03-22T11:50:29Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![RookieChenTaoYu](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/rookiechentaoyu/32/37_2.png) [@RookieChenTaoYu](https://discuss.vllm.ai/u/RookieChenTaoYu)\
**Post date:** [March 22, 2025, 11:50am UTC](https://discuss.vllm.ai/t/does-the-vllm-v1-support-speculative-decoding-now/191/1 "2025-03-22T11:50:29Z")

</div>

In the “vllm/v1/core/scheduler.py” at the Scheduler.update\_from\_output() method, there is annotation says that “# NOTE: once we support N tokens per step (spec decode), the outer lists can be of length \> 1.”. Does this comment mean that vllm v1 not support Speculative Decoding yet?

 ![spec](https://canada1.discourse-cdn.com/flex036/uploads/vllm/original/1X/d7d2b2ebe5bac6d73228246c1232ceb1dcec42d9.png)

---

<div class="post-metadata">

**Author:** ![LiuXiaoxuanPKU](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/liuxiaoxuanpku/32/43_2.png) [@LiuXiaoxuanPKU](https://discuss.vllm.ai/u/LiuXiaoxuanPKU)\
**Post date:** [March 23, 2025, 5:25am UTC](https://discuss.vllm.ai/t/does-the-vllm-v1-support-speculative-decoding-now/191/2 "2025-03-23T05:25:32Z")

</div>

v1 supports ngram speculative decoding. You can try as this example shows: [vllm/tests/v1/e2e/test\_ngram\_spec\_decode.py at main · vllm-project/vllm · GitHub](https://github.com/vllm-project/vllm/blob/main/tests/v1/e2e/test_ngram_spec_decode.py)

---

<div class="post-metadata">

**Author:** ![RookieChenTaoYu](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/rookiechentaoyu/32/37_2.png) [@RookieChenTaoYu](https://discuss.vllm.ai/u/RookieChenTaoYu)\
**Post date:** [March 23, 2025, 6:38am UTC](https://discuss.vllm.ai/t/does-the-vllm-v1-support-speculative-decoding-now/191/3 "2025-03-23T06:38:06Z")

</div>

Thanks for your reply. I also think `vllm v1` can surpport the spec decode, so does this annotation mean that the `logprobs` feature is not available in `vllm v1` with `spec decode` now, but may implement in future?

---

<div class="post-metadata">

**Author:** ![LiuXiaoxuanPKU](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/liuxiaoxuanpku/32/43_2.png) [@LiuXiaoxuanPKU](https://discuss.vllm.ai/u/LiuXiaoxuanPKU)\
**Post date:** [March 24, 2025, 5:28am UTC](https://discuss.vllm.ai/t/does-the-vllm-v1-support-speculative-decoding-now/191/4 "2025-03-24T05:28:49Z")

</div>

right, currently spec decode does not support logprobs as shown here: [vllm/vllm/v1/spec\_decode/utils.py at d20e261199d3e4bffe9ee0e495ba7f7454229f11 · vllm-project/vllm · GitHub](https://github.com/vllm-project/vllm/blob/d20e261199d3e4bffe9ee0e495ba7f7454229f11/vllm/v1/spec_decode/utils.py#L17).

---

<div class="post-metadata">

**Author:** ![RookieChenTaoYu](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/rookiechentaoyu/32/37_2.png) [@RookieChenTaoYu](https://discuss.vllm.ai/u/RookieChenTaoYu)\
**Post date:** [March 25, 2025, 8:04am UTC](https://discuss.vllm.ai/t/does-the-vllm-v1-support-speculative-decoding-now/191/5 "2025-03-25T08:04:34Z")

</div>

Thanks very much for your explanation.