# Structural GPU waste in DeepSeek-R1 inference anyone else measuring this

**URL:** <https://discuss.vllm.ai/t/structural-gpu-waste-in-deepseek-r1-inference-anyone-else-measuring-this/2826>\
**Category:** General\
**Created:** [July 23, 2026, 8:55pm UTC](https://discuss.vllm.ai/t/structural-gpu-waste-in-deepseek-r1-inference-anyone-else-measuring-this/2826 "2026-07-23T20:55:56Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Devo](https://avatars.discourse-cdn.com/v4/letter/d/f04885/32.png) [@Devo](https://discuss.vllm.ai/u/Devo)\
**Post date:** [July 23, 2026, 8:55pm UTC](https://discuss.vllm.ai/t/structural-gpu-waste-in-deepseek-r1-inference-anyone-else-measuring-this/2826/1 "2026-07-23T20:55:56Z")

</div>

Anyone running DeepSeek-R1 on dedicated instances, curious if others are seeing this. Ran a structural waste analysis on Alibaba’s NSDI 2026 production data (3.54B real requests) and found 44% structural GPU waste driven purely by output token length. DeepSeek-R1 averages 2,073 output tokens vs 82 for a general 14B model each reasoning request holds the GPU 25x longer, which widens the peak/average gap structurally. The waste floor is `min_waste = 1 - 1/burstiness` no batching or framework tuning moves it, only pooling does. Curious if anyone has measured this on their own clusters.

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [July 23, 2026, 8:56pm UTC](https://discuss.vllm.ai/t/structural-gpu-waste-in-deepseek-r1-inference-anyone-else-measuring-this/2826/2 "2026-07-23T20:56:18Z")

</div>

Yes, this is a known structural effect with DeepSeek-R1 and similar reasoning models: their much longer average output (2,073 tokens vs ~82 for typical 14B models) causes each request to monopolize a GPU for far longer, which increases the gap between peak and average utilization. As a result, even with perfect batching, the minimum achievable waste is set by the burstiness of the workload: min\_waste = 1 - 1/burstiness. No amount of batching or framework-level tuning can eliminate this waste—only pooling (i.e., sharing GPUs across multiple models or workloads) can reduce it. This phenomenon has been observed in production-scale deployments and is not unique to Alibaba’s NSDI 2026 data; it is a structural property of long-output reasoning models like DeepSeek-R1. See discussion in [DeepSeek-V3 Recipe](https://docs.vllm.ai/projects/recipes/en/latest/DeepSeek/DeepSeek-V3.html) and related performance analyses.

Would you like more detail on how pooling or workload shaping can mitigate this waste?

Sources:

- [DeepSeek-V3 Recipe](https://docs.vllm.ai/projects/recipes/en/latest/DeepSeek/DeepSeek-V3.html)

Hint: Mention @Herald in the post for followups.