Is there an async version of the LLM.chat() method? I checked the source code but may have missed it… or is there a simple way to implement it on top of AsyncLLMEngine without having to redo what chat() does?
Thanks!
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| Is is possible to initialize an AsyncLLMEngine inside the LLM object? | 4 | 125 | April 12, 2025 | |
| Dose vllm V1 support asynchronous scheduling? | 1 | 85 | April 14, 2025 | |
| Welcome to vLLM Forums! :wave: | 1 | 481 | March 24, 2025 | |
| Tool calling using Offline Inference? | 1 | 38 | April 14, 2025 | |
| Irrelevant Responses with Unsloth Fine-tuned Llama 3.1 8B using vLLM | 3 | 51 | April 10, 2025 |