# Does setting TRITON\_CACHE\_DIR break the bundled (AOT) static-autotuner cubin restore? **URL:** \ **Category:** torch.compile integration\ **Created:** [September 11, 2026, 5:03pm UTC](https://discuss.vllm.ai/t/does-setting-triton-cache-dir-break-the-bundled-aot-static-autotuner-cubin-restore/2965 "2026-09-11T17:03:55Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![kaiste99](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/kaiste99/32/1727_2.png) [@kaiste99](https://discuss.vllm.ai/u/kaiste99)\ **Post date:** [September 11, 2026, 5:03pm UTC](https://discuss.vllm.ai/t/does-setting-triton-cache-dir-break-the-bundled-aot-static-autotuner-cubin-restore/2965/1 "2026-09-11T17:03:55Z")
Context vLLM **0.29.0** (pip install, not Docker), torch **2.13.0** NVIDIA **GB10** (DGX Spark), single device, driver 580.173.02 / CUDA 13.0 `Qwen/Qwen3.6-35B-A3B-FP8`, `--max-cudagraph-capture-size 8192`, `--max-model-len 262144`, `--kv-cache-dtype fp8`, `--gpu-memory-utilization 0.85` AOT compile cache in use (default path `~/.cache/vllm/torch_compile_cache/torch_aot_compile//…`) Observation On **every** server start I get 13 occurrences of ```auto RuntimeError: ('Cubin file saved by TritonBundler not found at %s ``` each immediately followed by `reconstructed serializable fn from standalone compile artifacts`. Affected kernels are e.g. `triton_poi_fused_zeros_1/2/3`, `triton_poi_fused_mul_per_token_group_fp8_quant_permute_sigmoid_view_0`, `triton_red_fused__to_copy_add_fused_add_rms_norm_per_token_group_fp8_quant_permute_1`. Correctness is fine — `torch/_inductor/triton_bundler.py` catches the `RuntimeError` and `continue`s, so those static autotuners are skipped and the kernels are recompiled at runtime. It looks like a warm-up cost only. What the lookup actually resolved to I wrapped `os.path.exists` to log failing `*.cubin` lookups (env + directory listing at that moment). One entry: ```auto missing path : $TRITON_CACHE_DIR//triton_poi_fused_zeros_1.cubin (parent directory does not exist) TORCHINDUCTOR_CACHE_DIR : ~/.cache/vllm/torch_compile_cache/torch_aot_compile//inductor_cache <- i.e. vLLM had already set this correctly for the artifact being restored TRITON_CACHE_DIR (env) : ~/.cache/triton ``` The **same file does exist on disk with an identical hash directory name** , only under a different prefix: ```auto /triton/0//triton_poi_fused_zeros_1.cubin ^ device level present ``` Mechanism as I read it `torch/_inductor/runtime/cache_dir_utils.py`: ```auto def triton_cache_dir(device: int) -> str: if (directory := os.getenv("TRITON_CACHE_DIR")) is not None: return directory # verbatim — no "/{device}" return os.path.join(cache_dir(), "triton", str(device)) ``` `triton_bundler.py` `read_and_emit()` re-emits the bundled artifacts to `os.path.join(triton_cache_dir(artifacts.device), artifacts.kernel_hash)` (JSON payloads carry a `[REPLACE]` sentinel that is substituted with that directory). `triton_heuristics.py` `reload_cubin_path()` recomputes `os.path.join(triton_cache_dir(device), triton_hash_to_path_key(kernel.hash), f"{name}.cubin")`. So emit and reload agree only when `triton_cache_dir()` resolves to the same directory in both cases. With `TRITON_CACHE_DIR` set it resolves **without** the device level, while the artifacts on disk sit under `…/triton/0/…` → miss. Related: pytorch/pytorch#163336 (“Potential bug in `_inductor/runtime/cache_dir_utils.py`”) raised the “device id is not appended when `TRITON_CACHE_DIR` is set” part and was closed as intentional (one shared cache across ranks, atomic renames). I am **not** re-litigating that — my question is about the configuration and the emit/reload agreement. Questions Is setting `TRITON_CACHE_DIR` supported alongside vLLM’s AOT compile cache? I set it because on this host `/tmp` is wiped on **every** boot (`systemd-tmpfiles` rule `D /tmp 1777 root root 30d`), so torch’s default `/tmp/torchinductor_$USER` is always cold. Should users instead set only `TORCHINDUCTOR_CACHE_DIR` and leave `TRITON_CACHE_DIR` unset? If it is not supported, would it make sense to document that — or to not depend on the global resolution when restoring a bundle? Alternatively: is the 13-kernel rebuild per start simply expected, and the warning can be ignored? Reproduction (to see the count on any setup using the compile cache): ```auto journalctl -u | grep -c 'Failed to reload cubin' ``` Happy to share the full trace JSON (missing path, env, directory listings) or to run further ---
**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\ **Post date:** [September 11, 2026, 5:04pm UTC](https://discuss.vllm.ai/t/does-setting-triton-cache-dir-break-the-bundled-aot-static-autotuner-cubin-restore/2965/2 "2026-09-11T17:04:15Z")
Setting TRITON\_CACHE\_DIR alongside vLLM’s AOT compile cache is not supported and leads to the exact issue you observed: Triton expects cubins in $TRITON\_CACHE\_DIR//…, but vLLM (and PyTorch Inductor) emit them under …/triton///… when TRITON\_CACHE\_DIR is unset. This mismatch causes the “Failed to reload cubin file” warnings and runtime recompilation, as described in your logs and confirmed by upstream discussions ([pytorch/pytorch#163336](https://github.com/pytorch/pytorch/issues/163336)) and vLLM documentation ([vLLM cache directory security](https://docs.vllm.ai/en/latest/usage/security/#cache-directory-configuration), [compiler interface](https://docs.vllm.ai/en/latest/api/vllm/compilation/compiler_interface/#__span-0-468)). The recommended approach is to set only TORCHINDUCTOR\_CACHE\_DIR and leave TRITON\_CACHE\_DIR unset. vLLM will then manage both Inductor and Triton caches under its own cache root, ensuring emit and reload agree and avoiding unnecessary recompilation. The 13-kernel rebuilds per start are not expected if the cache is configured correctly; the warning can be ignored for correctness, but it does indicate a suboptimal cache setup ([compiler interface](https://docs.vllm.ai/en/latest/api/vllm/compilation/compiler_interface/#__span-0-468), [security docs](https://docs.vllm.ai/en/latest/usage/security/#cache-directory-configuration)). Would you like a step-by-step explanation of how vLLM manages these cache directories and why this mismatch occurs? Sources: - [vLLM cache directory security](https://docs.vllm.ai/en/latest/usage/security/#cache-directory-configuration) - [compiler interface](https://docs.vllm.ai/en/latest/api/vllm/compilation/compiler_interface/#__span-0-468) Hint: Mention @Herald in the post for followups.