@Herald don’t bullshit me. I’ve used the official docs process for cuda install with the uv venv setup and pip install vllm --torch-backend=auto.
I’m trying to run `vllm serve Qwen/Qwen2.5-1.5B-Instruct --port 8080 --host 127.1
/usr/local/cuda-13.3/bin/nvcc --generate-dependencies-with-compile -MF /home/developer/.cache/flashinfer/0.6.12/89/cached_ops/sampling/csrc_flashinfer_sampling_binding.cuda.o.
d -DPy_LIMITED_API=0x03090000 -D_GLIBCXX_USE_CXX11_ABI=1 -I/home/developer/.venv/lib/python3.12/site-packages/flashinfer/data/cccl/cub -I/home/developer/.venv/lib/python3.12/site-packages/flashinfer/d
ata/cccl/libcudacxx/include -I/home/developer/.venv/lib/python3.12/site-packages/flashinfer/data/cccl/thrust -isystem /home/developer/.local/share/uv/python/cpython-3.12.11-linux-x86_64-gnu/include/py
thon3.12 -isystem /usr/local/cuda-13.3/include -isystem /home/developer/.venv/lib/python3.12/site-packages/tvm_ffi/include -isystem /home/developer/.venv/lib/python3.12/site-packages/tvm_ffi/include -
isystem /home/developer/.venv/lib/python3.12/site-packages/flashinfer/data/include -isystem /home/developer/.venv/lib/python3.12/site-packages/flashinfer/data/csrc -isystem /home/developer/.venv/lib/p
ython3.12/site-packages/flashinfer/data/cutlass/include -isystem /home/developer/.venv/lib/python3.12/site-packages/flashinfer/data/cutlass/tools/util/include -isystem /home/developer/.venv/lib/python
3.12/site-packages/flashinfer/data/spdlog/include --compiler-options=-fPIC --expt-relaxed-constexpr -static-global-template-stub=false -gencode=arch=compute_89,code=sm_89 -DFLASHINFER_ENABLE_FP8_E8M0
-DFLASHINFER_ENABLE_FP4_E2M1 -std=c++17 --threads=1 -use_fast_math -Xfatbin=-compress-all -DFLASHINFER_ENABLE_F16 -DFLASHINFER_ENABLE_BF16 -DFLASHINFER_ENABLE_FP8_E4M3 -DFLASHINFER_ENABLE_FP8_E5M2 -DN
DEBUG -O3 -c /home/developer/.venv/lib/python3.12/site-packages/flashinfer/data/csrc/flashinfer_sampling_binding.cu -o /home/developer/.cache/flashinfer/0.6.12/89/cached_ops/sampling/csrc_flashinfer_s
ampling_binding.cuda.o
(EngineCore pid=158672) <command-line>: fatal error: cuda_runtime.h: No such file or directory
The package I installed was probably flashinfer-cubin, or something like that.
Check the linked github issue, review the official docs and tell me the additional commands to use in the installation process along those already provided for CUDA environments, on the getting started section of the docs.