# Install using --torch-backend=cu129 but try to import cu13

**URL:** <https://discuss.vllm.ai/t/install-using-torch-backend-cu129-but-try-to-import-cu13/2601>\
**Category:** General\
**Created:** [April 29, 2026, 12:39am UTC](https://discuss.vllm.ai/t/install-using-torch-backend-cu129-but-try-to-import-cu13/2601 "2026-04-29T00:39:07Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![cieske](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/cieske/32/1350_2.png) [@cieske](https://discuss.vllm.ai/u/cieske)\
**Post date:** [April 29, 2026, 12:39am UTC](https://discuss.vllm.ai/t/install-using-torch-backend-cu129-but-try-to-import-cu13/2601/1 "2026-04-29T00:39:07Z")

</div>

I want to serve DeepSeek-v4 w/ H100, driver version 535.129.03, so install vllm w/ `uv pip install vllm --torch-backend=cu129` as described at [v0.20.0 releases](https://github.com/vllm-project/vllm/releases).

After installing, tried to serve `uv run vllm serve ~~`, below error occured:

```auto
Traceback (most recent call last):
  File "/user-volume/latest_serve/.venv/bin/vllm", line 10, in <module>
    sys.exit(main())
             ^^^^^^
  File "/user-volume/latest_serve/.venv/lib/python3.12/site-packages/vllm/entrypoints/cli/main.py", line 18, in main
    import vllm.entrypoints.cli.benchmark.main
  File "/user-volume/latest_serve/.venv/lib/python3.12/site-packages/vllm/entrypoints/cli/benchmark/main.py", line 10, in <module>
    from vllm.entrypoints.utils import VLLM_SUBCMD_PARSER_EPILOG
  File "/user-volume/latest_serve/.venv/lib/python3.12/site-packages/vllm/entrypoints/utils.py", line 19, in <module>
    from vllm.engine.arg_utils import EngineArgs
  File "/user-volume/latest_serve/.venv/lib/python3.12/site-packages/vllm/engine/arg_utils.py", line 35, in <module>
    from vllm.config import (
  File "/user-volume/latest_serve/.venv/lib/python3.12/site-packages/vllm/config/ __init__.py", line 6, in <module>
    from vllm.config.compilation import (
  File "/user-volume/latest_serve/.venv/lib/python3.12/site-packages/vllm/config/compilation.py", line 22, in <module>
    from vllm.platforms import current_platform
  File "/user-volume/latest_serve/.venv/lib/python3.12/site-packages/vllm/platforms/ __init__.py", line 278, in __getattr__
    _current_platform = resolve_obj_by_qualname(platform_cls_qualname)()
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/user-volume/latest_serve/.venv/lib/python3.12/site-packages/vllm/utils/import_utils.py", line 109, in resolve_obj_by_qualname
    module = importlib.import_module(module_name)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/importlib/ __init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/user-volume/latest_serve/.venv/lib/python3.12/site-packages/vllm/platforms/cuda.py", line 21, in <module>
    import vllm._C # noqa
    ^^^^^^^^^^^^^^
ImportError: libcudart.so.13: cannot open shared object file: No such file or directory

```

Same error occured when using `--torch-backend=cu128`, etc. How can I solve it? Using latest cuda version raise below error, and can’t upgrade driver right now:

```auto
RuntimeError: The NVIDIA driver on your system is too old (found version 12020). Please update your GPU driver by downloading and installing a new version from the URL: http://www.nvidia.com/Download/index.aspx Alternatively, go to: https://pytorch.org to install a PyTorch version that has been compiled with your version of the CUDA driver.

```

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [April 29, 2026, 12:39am UTC](https://discuss.vllm.ai/t/install-using-torch-backend-cu129-but-try-to-import-cu13/2601/2 "2026-04-29T00:39:27Z")

</div>

Your error ImportError: libcudart.so.13: cannot open shared object file: No such file or directory means the CUDA 13 runtime library is missing from your system, but vLLM was installed with CUDA 13 support. Your driver (535.129.03) only supports up to CUDA 12.2, so CUDA 13 binaries will not work. You cannot use --torch-backend=cu129 or cu130 unless your system has the matching CUDA runtime and a newer driver installed.

To fix this, install vLLM and PyTorch with CUDA 12.2 or 12.1 support, matching your driver. For example, use:

```bash
uv pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
uv pip install vllm --extra-index-url https://wheels.vllm.ai/0.19.0/cu121 --extra-index-url https://download.pytorch.org/whl/cu121 --index-strategy unsafe-best-match

```

If you need CUDA 12.2, replace cu121 with cu122 in the URLs. You cannot use CUDA 13 wheels or runtime until you upgrade your NVIDIA driver to a version that supports CUDA 13+ ([source](https://github.com/vllm-project/vllm/issues/31018), [source](https://github.com/vllm-project/vllm/issues/25143), [source](https://docs.vllm.ai/en/latest/getting_started/installation/gpu.html)). Would you like more detail or step-by-step instructions?

Sources:

- [https://github.com/vllm-project/vllm/issues/31018](https://github.com/vllm-project/vllm/issues/31018)
- [https://github.com/vllm-project/vllm/issues/25143](https://github.com/vllm-project/vllm/issues/25143)
- [https://docs.vllm.ai/en/latest/getting\_started/installation/gpu.html](https://docs.vllm.ai/en/latest/getting_started/installation/gpu.html)

Hint: Mention @RunLLM in the post for followups.

---

<div class="post-metadata">

**Author:** ![cieske](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/cieske/32/1350_2.png) [@cieske](https://discuss.vllm.ai/u/cieske)\
**Post date:** [April 29, 2026, 12:55am UTC](https://discuss.vllm.ai/t/install-using-torch-backend-cu129-but-try-to-import-cu13/2601/4 "2026-04-29T00:55:00Z")

</div>

@RunLLM

I successfully running Qwen3.5 family w/ `vllm==0.19 --torch-backend cu129`, so this statement is wrong: `You cannot use --torch-backend=cu129 unless your system has the matching CUDA runtime and a newer driver installed.`

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [April 29, 2026, 12:55am UTC](https://discuss.vllm.ai/t/install-using-torch-backend-cu129-but-try-to-import-cu13/2601/5 "2026-04-29T00:55:44Z")

</div>

You are correct that, in some cases, vLLM with --torch-backend=cu129 can run on a system with an older driver (e.g., 535.129.03) if you install the CUDA 12.9 compatibility libraries (cuda-compat-12-9) and set LD\_LIBRARY\_PATH accordingly. This enables forward compatibility for certain professional/datacenter GPUs, allowing CUDA 12.9 binaries to run on drivers that do not natively support CUDA 12.9 ([NVIDIA CUDA compatibility](https://docs.nvidia.com/deploy/cuda-compatibility/), [vLLM troubleshooting](https://docs.vllm.ai/en/latest/usage/troubleshooting/#cuda-error-the-provided-ptx-was-compiled-with-an-unsupported-toolchain)).

However, if you do not install the compatibility libraries, or if your GPU/driver combination does not support forward compatibility, you will encounter the ImportError for libcudart.so.13 or similar errors. So, the statement is not universally wrong, but there are supported workarounds for some environments. Would you like a step-by-step guide for setting up CUDA compatibility?

Sources:

- [https://docs.vllm.ai/en/latest/usage/troubleshooting/#cuda-error-the-provided-ptx-was-compiled-with-an-unsupported-toolchain](https://docs.vllm.ai/en/latest/usage/troubleshooting/#cuda-error-the-provided-ptx-was-compiled-with-an-unsupported-toolchain)

Hint: Mention @RunLLM in the post for followups.

---

<div class="post-metadata">

**Author:** ![cieske](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/cieske/32/1350_2.png) [@cieske](https://discuss.vllm.ai/u/cieske)\
**Post date:** [April 29, 2026, 1:26am UTC](https://discuss.vllm.ai/t/install-using-torch-backend-cu129-but-try-to-import-cu13/2601/6 "2026-04-29T01:26:20Z")

</div>

@RunLLM Install `cuda-nvcc-12-9 cuda-cudart-dev-12-9 libcublas-dev-12-9 libcusparse-dev-12-9 libcusolver-dev-12-9 cuda-compat-12-9` and set `LD_LIBRARY_PATH`, but same error occured. Can you provide step-by-step guide for setting CUDA compatibility?

When serving Qwen3.5 family w/ `vllm==0.19, cu126`, just install `cuda-nvcc-12-6 cuda-cudart-dev-12-6 libcublas-dev-12-6 libcusparse-dev-12-6 libcusolver-dev-12-6` works.

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [April 29, 2026, 1:26am UTC](https://discuss.vllm.ai/t/install-using-torch-backend-cu129-but-try-to-import-cu13/2601/7 "2026-04-29T01:26:49Z")

</div>

To enable CUDA compatibility for vLLM with CUDA 12.9 (cu129) on a system with an older driver, you must install the cuda-compat-12-9 package and set the environment variables as described in the official troubleshooting guide. Installing only the runtime and dev libraries is not enough—cuda-compat provides forward compatibility libraries required for this setup ([vLLM troubleshooting](https://docs.vllm.ai/en/latest/usage/troubleshooting/#cuda-error-the-provided-ptx-was-compiled-with-an-unsupported-toolchain)).

**Step-by-step guide:**

1. Install the compatibility package:

```bash
sudo apt-get install cuda-compat-12-9

```

2. Set environment variables before running vLLM:

```bash
export VLLM_ENABLE_CUDA_COMPATIBILITY=1
export VLLM_CUDA_COMPATIBILITY_PATH="/usr/local/cuda-12.9/compat"
export LD_LIBRARY_PATH="/usr/local/cuda-12.9/compat:$LD_LIBRARY_PATH"

```

3. Run a minimal test to verify:

```bash
python3 -c "import vllm; import torch; print(torch.cuda.is_available())"

```

4. If this works, run your vLLM serve command as usual.

Note: Forward compatibility only works on select professional/datacenter GPUs. For consumer GPUs, this method may not work ([NVIDIA CUDA compatibility](https://docs.nvidia.com/deploy/cuda-compatibility/)). Would you like more troubleshooting steps?

Sources:

- [https://docs.vllm.ai/en/latest/usage/troubleshooting/#cuda-error-the-provided-ptx-was-compiled-with-an-unsupported-toolchain](https://docs.vllm.ai/en/latest/usage/troubleshooting/#cuda-error-the-provided-ptx-was-compiled-with-an-unsupported-toolchain)

Hint: Mention @RunLLM in the post for followups.

---

<div class="post-metadata">

**Author:** ![cieske](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/cieske/32/1350_2.png) [@cieske](https://discuss.vllm.ai/u/cieske)\
**Post date:** [April 29, 2026, 2:19am UTC](https://discuss.vllm.ai/t/install-using-torch-backend-cu129-but-try-to-import-cu13/2601/8 "2026-04-29T02:19:21Z")

</div>

@RunLLM Install & setting options you mentioned, terminal test returns `True`, but same `ImportError: libcudart.so.13: cannot open shared object file: No such file or directory` error occured when running `uv run vllm serve ...`

Below are some result of `collect_env.py`:

CUDA\_HOME=/usr/local/cuda-12.9  
CUDA\_HOME=/usr/local/cuda-12.9  
LD\_LIBRARY\_PATH=/usr/local/cuda-12.9/compat:/usr/local/cuda-12.9/targets/x86\_64-linux/lib/:/usr/local/cuda-12.9/lib64:  
VLLM\_CUDA\_COMPATIBILITY\_PATH=/usr/local/cuda-12.9/compat

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [April 29, 2026, 2:19am UTC](https://discuss.vllm.ai/t/install-using-torch-backend-cu129-but-try-to-import-cu13/2601/9 "2026-04-29T02:19:53Z")

</div>

If you still get ImportError: libcudart.so.13: cannot open shared object file after installing cuda-compat-12-9 and setting LD\_LIBRARY\_PATH and VLLM\_CUDA\_COMPATIBILITY\_PATH, it means vLLM or PyTorch is trying to load CUDA 13 libraries, not 12.9. This usually happens if you installed vLLM or torch with cu130/cu131 wheels, not cu129 ([vLLM troubleshooting](https://docs.vllm.ai/en/latest/usage/troubleshooting/#cuda-error-the-provided-ptx-was-compiled-with-an-unsupported-toolchain), [NVIDIA CUDA compatibility](https://docs.nvidia.com/deploy/cuda-compatibility/)).

**Solution:**  
Uninstall vllm and torch, then reinstall both with cu129 wheels only. For example:

```bash
uv pip uninstall vllm torch torchvision torchaudio
uv pip install torch==<version> --index-url https://download.pytorch.org/whl/cu129
uv pip install vllm --extra-index-url https://wheels.vllm.ai/0.19.0/cu129 --extra-index-url https://download.pytorch.org/whl/cu129

```

Make sure no cu130/cu131 wheels are present in your environment. Would you like more detail on how to check your installed packages?

Sources:

- [https://docs.vllm.ai/en/latest/usage/troubleshooting/#cuda-error-the-provided-ptx-was-compiled-with-an-unsupported-toolchain](https://docs.vllm.ai/en/latest/usage/troubleshooting/#cuda-error-the-provided-ptx-was-compiled-with-an-unsupported-toolchain)

Hint: Mention @RunLLM in the post for followups.

---

<div class="post-metadata">

**Author:** ![cieske](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/cieske/32/1350_2.png) [@cieske](https://discuss.vllm.ai/u/cieske)\
**Post date:** [April 29, 2026, 8:07am UTC](https://discuss.vllm.ai/t/install-using-torch-backend-cu129-but-try-to-import-cu13/2601/10 "2026-04-29T08:07:53Z")

</div>

Actually I need to install `vllm==0.20` to serve DeepSeek-v4, installing whl file from `https://wheels.vllm.ai/0.20.0/cu129/vllm` works!