Welcome to vLLM Forums! 👋

General
1

We are so glad you joined us.

vLLM Forums

A high-throughput and memory-efficient inference and serving engine for LLMs

Here are some things you can do to get started:

:speaking_head: Introduce yourself by adding your picture and information about yourself and your interests to your profile. What is one thing you’d like to be asked about?

:open_book: Get to know the community by browsing discussions that are already happening here. When you find a post interesting, informative, or entertaining, use the :heart: to show your appreciation or support!

:handshake: Contribute by commenting, sharing your own perspective, asking questions, or offering feedback in the discussion. Before replying or starting new topics, please review the Community Guidelines.

If you need help or have a suggestion, feel free to ask in Site Feedback or contact the admins.

4 Likes
2

Welcome! I would like explain more about the purpose of this forum, courtesy of @comaniac:

  • If you are confused about where to put their questions (forum vs. Github issues), especially GitHub issues are more accessible than the forum. A simple principle could be: all questions should go to forum, and Github issues should be something the vLLM developers want to track.

  • If you creates an GH issue for questions, we will reply saying that please ask this question in the forum, and close the issue.

  • If a topic in the forum ends up being a bug report or feature request, we should encourage the author to create an issue in the Github.

  • We encourage all community members to answer questions in the forum, whatever you’re a committer or not. Answering questions could also be an important metric for nominating committers (we attached forum activity link in the committer nomination letter).

  • We also encourage beginners to answer simple questions. This is a very good starting point for beginners to participate the community.

4 Likes
3

我想问一下，vllm0.11支持cuda13吗。如果不支持，有什么方法吗

4

vLLM 0.11 官方预编译 wheel 目前主要支持 CUDA 12.8/12.9 及以下版本，未直接提供 CUDA 13 的 wheel，但可以通过安装 cuda-compat 包或手动兼容库文件，使 CUDA 13 驱动环境兼容 vLLM 的 CUDA 12.9 PTX 代码，从而运行 vLLM 0.11。例如，安装 cuda-compat-12-9 并设置 LD_LIBRARY_PATH，或手动下载高版本驱动的兼容库文件，参考官方文档和社区实践操作即可解决兼容性问题。具体方法详见官方文档和社区经验。

如果你在 Jetson AGX Thor 等只能安装 CUDA 13 的设备上，建议优先尝试上述兼容方法，或自行编译 vLLM 源码以适配本地 CUDA 13 环境。官方未直接发布 CUDA 13 的 wheel，但通过兼容包或源码编译可实现支持。如需详细操作步骤，请告知。

Sources:

Hint: Mention @RunLLM in the post for followups.