|
Disagg Prefill timeout
|
|
1
|
202
|
July 7, 2025
|
|
MoE quantization
|
|
9
|
1642
|
July 2, 2025
|
|
Why is cuda graph capture sizes limited by max_num_seqs
|
|
1
|
1289
|
June 29, 2025
|
|
Scheduler in vllm
|
|
1
|
460
|
June 26, 2025
|
|
Prompt_embeds usage in vllm openai completion api
|
|
4
|
324
|
June 17, 2025
|
|
W8a8两种量化方式有详细介绍吗
|
|
1
|
298
|
June 15, 2025
|
|
Seqence Parallelism Support - Source Code Location
|
|
0
|
65
|
June 10, 2025
|
|
Something weired about the reading procedure of q_vecs in page attention kernel
|
|
3
|
115
|
June 9, 2025
|
|
Computation time remain consistent across chunks in chunked-prefill despite linearly growing attention complexity?
|
|
1
|
207
|
June 2, 2025
|
|
Why does computation time remain consistent across chunks in chunked-prefill despite linearly growing attention complexity?
|
|
3
|
260
|
June 2, 2025
|
|
APC Slowdown with block-size=1
|
|
1
|
114
|
May 26, 2025
|
|
RunBot's math-to-text on NVIDIA NeMo Framework AutoModel
|
|
11
|
372
|
May 19, 2025
|
|
Issue with DynamicYaRN and Key-Value Cache Reuse in vLLM
|
|
1
|
223
|
May 18, 2025
|
|
VUA - library code for LLM inference engines for external storage of KV caches
|
|
1
|
188
|
May 13, 2025
|
|
Specifying special tokens
|
|
5
|
828
|
May 8, 2025
|
|
vLLM cannot connect to existing Ray cluster
|
|
16
|
1510
|
May 8, 2025
|
|
Support for (sparse) key value caching
|
|
16
|
1076
|
May 3, 2025
|
|
How to use speculative decoding?
|
|
3
|
1315
|
May 1, 2025
|
|
Grammar CPU bound performance
|
|
9
|
802
|
April 29, 2025
|
|
Does vLLM support multiple model_executor?
|
|
1
|
482
|
April 28, 2025
|
|
Spec decode with eagle get very low Draft acceptance rate
|
|
1
|
637
|
April 25, 2025
|
|
LoRA Adapter enabling with vLLM is not working
|
|
4
|
704
|
April 21, 2025
|
|
Goodput Guided Speculative Decoding
|
|
2
|
351
|
April 19, 2025
|
|
Is structured output compatible with automatic prefix caching?
|
|
1
|
201
|
April 14, 2025
|
|
Tool calling using Offline Inference?
|
|
1
|
235
|
April 14, 2025
|
|
How to crop kv_caches?
|
|
0
|
96
|
April 13, 2025
|
|
Minimum requirements for Disaggregated Prefilling?
|
|
0
|
127
|
April 9, 2025
|
|
Can Lora adapters be loaded on different GPUs
|
|
1
|
204
|
April 7, 2025
|
|
Do we have regression tests for structured output? Especially speed regression?
|
|
0
|
81
|
March 31, 2025
|
|
Why remove bonus token of requset in draft model?
|
|
0
|
76
|
March 30, 2025
|