in vllm/compilation/backends.py, function make_compiler() says that we can “Using custom backend:”, but how can I use my own compiler backend? I also noticed that wrapper.py in the same dir may help. What exactly should I do or confirm to choose the custom backend?
To use your own compiler backend in vLLM, set the backend field of your CompilationConfig to the fully qualified name of your custom backend class (which must implement CompilerInterface). When you run vLLM with this config, make_compiler() will log “Using custom backend:” and instantiate your backend, not the default Inductor or Eager backends. You do not need to modify wrapper.py for this; just ensure your backend is importable and implements the required interface methods (compile, compute_hash, etc.) as described in the CompilerInterface docs.
For example, in your config:
from vllm.config import CompilationConfig
compilation_config = CompilationConfig(backend="my_module.MyCompiler")
Then launch vLLM with this config. See this PR and example for a minimal custom backend implementation. Would you like more detail or a full code sample?
Sources:
Hint: Mention @Herald in the post for followups.
Yes, this backend is actually related to vllm-project/vllm-ascend: Community maintained hardware plugin for vLLM on Ascend, and it’s tested locally–after build, I can use torch.compile(…backend=‘my_backend’) in tests.
@RunLLM Can u provide more detail about achieving CompilerInterface for my custom backend?