|
Welcome to vLLM Forums! :wave:
|
|
1
|
401
|
March 24, 2025
|
|
Specifying special tokens
|
|
5
|
15
|
May 8, 2025
|
|
What is the deal with the runllm bot?
|
|
4
|
32
|
May 8, 2025
|
|
MoE quantization
|
|
3
|
18
|
May 8, 2025
|
|
Why is this not working? I corrected it but still
|
|
1
|
19
|
May 8, 2025
|
|
Can anyone help me? Why is this not working? It used ðŸ˜
|
|
1
|
14
|
May 8, 2025
|
|
vLLM cannot connect to existing Ray cluster
|
|
16
|
20
|
May 8, 2025
|
|
Questions on piecewise torch compile design
|
|
5
|
18
|
May 7, 2025
|
|
Docker explosion this morning after it worked fine for a long while
|
|
6
|
13
|
May 6, 2025
|
|
Offline multi-node inference
|
|
7
|
12
|
May 6, 2025
|
|
How can I find top-down architecture of vLLM?
|
|
1
|
24
|
May 5, 2025
|
|
32GB vs 48GB vRam
|
|
1
|
15
|
May 3, 2025
|
|
Support for (sparse) key value caching
|
|
16
|
23
|
May 3, 2025
|
|
What is the optimal combination of parallelism when deploying deepseek-r1 with multiple nodes?
|
|
0
|
12
|
May 2, 2025
|
|
How to use speculative decoding?
|
|
3
|
27
|
May 1, 2025
|
|
Run on B200/5090 without building from source?
|
|
1
|
14
|
May 1, 2025
|
|
Running Gemma 3 on multi-chip TPU failure
|
|
5
|
99
|
May 1, 2025
|
|
Go Deeper / Go Dumber by vLLM AI bot
|
|
5
|
57
|
May 1, 2025
|
|
Flash Attention 3 FP8 Support
|
|
1
|
22
|
April 30, 2025
|
|
Grammar CPU bound performance
|
|
9
|
103
|
April 29, 2025
|
|
Setting up VllmConfig for custom gpt2
|
|
2
|
19
|
April 29, 2025
|
|
I published a performance test result of vllm vs sglang but can someone help me explain it?
|
|
3
|
55
|
April 29, 2025
|
|
Which arguments affect GPU memory
|
|
1
|
26
|
April 29, 2025
|
|
Is Batch Inference for Multimodal Models Truly Batch Inference?
|
|
3
|
21
|
April 29, 2025
|
|
How to get `http_*` metrics as this doc suggests are available?
|
|
1
|
17
|
April 28, 2025
|
|
Extract class log probabilities from LLM classifier using async vLLM and AWS LMI containers
|
|
1
|
8
|
April 28, 2025
|
|
Does vLLM support multiple model_executor?
|
|
1
|
10
|
April 28, 2025
|
|
Question about V1 code(v0.8.2)
|
|
3
|
21
|
April 28, 2025
|
|
Failed to run distribute Inference with vllm 0.8.2
|
|
6
|
54
|
April 27, 2025
|
|
Intergate async-llm to openrlfh, when tensor_parallel_size > 1 will cause ray-error
|
|
1
|
19
|
April 27, 2025
|