# Registering with Transformers Auto Classes for out-of-tree models **URL:** \ **Category:** General\ **Created:** [July 29, 2025, 10:26pm UTC](https://discuss.vllm.ai/t/registering-with-transformers-auto-classes-for-out-of-tree-models/1229 "2025-07-29T22:26:46Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![agharwal](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/agharwal/32/548_2.png) [@agharwal](https://discuss.vllm.ai/u/agharwal)\ **Post date:** [July 29, 2025, 10:26pm UTC](https://discuss.vllm.ai/t/registering-with-transformers-auto-classes-for-out-of-tree-models/1229/1 "2025-07-29T22:26:46Z")
### Question How to register custom out-of-tree models with Transformers Auto Classes for running `AsyncLLMEngine` in the `OPENAI_API_SERVER` usage context, without modifying vLLM source code? ### Context I’m currently doing so by calling my registration method from within `vllm/model_executor/models/registry.py`, which works fine, but I’d like to avoid modifying vLLM source code to speed-up vLLM version bumps in deployments. As I understand it, the V1 Engine spawns Workers but it seems like the Transformers registration, if done from the top-level app, does not carry over to the Workers that error out while interpreting HF configs. ---
**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\ **Post date:** [July 29, 2025, 10:27pm UTC](https://discuss.vllm.ai/t/registering-with-transformers-auto-classes-for-out-of-tree-models/1229/2 "2025-07-29T22:27:10Z")
To register custom out-of-tree models (OOT) with Transformers Auto Classes for use with AsyncLLMEngine and the OpenAI API server—without modifying vLLM source code—you should use vLLM’s plugin system. This ensures your registration code is executed in every worker process, which is necessary for distributed and multiprocessing setups, including V1 Engine and AsyncLLMEngine. Place your model registration logic in a plugin entrypoint, and install your plugin package so vLLM can discover and load it automatically at startup. This approach is officially recommended and avoids the pitfalls of top-level registration, which does not propagate to worker processes and leads to errors when loading custom models in distributed contexts. See [vLLM’s Plugin System documentation](https://docs.vllm.ai/en/stable/design/plugin_system.html) and [Model Registration guide](https://docs.vllm.ai/en/stable/contributing/model/registration.html#out-of-tree-models) for details. Example plugin registration code (in your plugin’s entrypoint, e.g., `register()`): ```python def register(): from vllm import ModelRegistry from transformers import AutoConfig from your_code import YourModelForCausalLM, YourConfigClass AutoConfig.register("your_model_type", YourConfigClass, exist_ok=True) ModelRegistry.register_model("YourModelForCausalLM", YourModelForCausalLM) ``` Then, set up your package’s `setup.py` with an entry point under `vllm.general_plugins`, and install it in your environment. This ensures the registration is executed in all vLLM processes, including workers spawned by the API server or AsyncLLMEngine, without modifying vLLM source code. Would you like a step-by-step example of the plugin packaging and installation process? Sources: - [vLLM’s Plugin System — vLLM](https://docs.vllm.ai/en/stable/design/plugin_system.html) - [GitHub Issue: Cannot register new model](https://github.com/vllm-project/vllm/issues/16228) - [GitHub Issue: Service launch failed with v1 code and custom models](https://github.com/vllm-project/vllm/issues/11055) ---
**Author:** ![Sun-Xiaohui](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/sun-xiaohui/32/197_2.png) [@Sun-Xiaohui](https://discuss.vllm.ai/u/Sun-Xiaohui)\ **Post date:** [July 14, 2026, 6:46am UTC](https://discuss.vllm.ai/t/registering-with-transformers-auto-classes-for-out-of-tree-models/1229/3 "2026-07-14T06:46:00Z")
How to organize the files in a package? I want to register an OOT model named `QwenNemotronForCausalLM` and a config named `QwenNemotronConfig`, Could you please give a step-by-step example of the plugin packaging and installation process? @RunLLM ---
**Author:** ![Sun-Xiaohui](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/sun-xiaohui/32/197_2.png) [@Sun-Xiaohui](https://discuss.vllm.ai/u/Sun-Xiaohui)\ **Post date:** [July 14, 2026, 10:02am UTC](https://discuss.vllm.ai/t/registering-with-transformers-auto-classes-for-out-of-tree-models/1229/4 "2026-07-14T10:02:37Z")
I copy the folder [https://github.com/vllm-project/vllm/tree/main/tests/plugins/vllm\_add\_dummy\_model](https://github.com/vllm-project/vllm/tree/main/tests/plugins/vllm_add_dummy_model), run `pip install -e .` then run vllm inference, raise an error: `ERROR 07-14 17:16:46 [` **`init`** `.py:64] Failed to load plugin register_dummy_model` `ERROR 07-14 17:16:46 [` **`init`** `.py:64] Traceback (most recent call last):` `ERROR 07-14 17:16:46 [` **`init`** `.py:64] File "/home/lenovo/miniconda3/envs/vllm_dflash/lib/python3.13/site-packages/vllm/plugins/` **`init`** `.py", line 61, in load_plugins_by_group` `ERROR 07-14 17:16:46 [` **`init`** `.py:64] func = plugin.load()` `ERROR 07-14 17:16:46 [` **`init`** `.py:64] File "/home/lenovo/miniconda3/envs/vllm_dflash/lib/python3.13/importlib/metadata/` **`init`** `.py", line 181, in load` `ERROR 07-14 17:16:46 [` **`init`** `.py:64] return functools.reduce(getattr, attrs, module)` `ERROR 07-14 17:16:46 [` **`init`** `.py:64] ~~~~~~~~~~~~~~~~ ^^^^^^^^^^^^^^^^^^^^^^^^` `ERROR 07-14 17:16:46 [` **`init`** `.py:64] AttributeError: module 'vllm_add_dummy_model' has no attribute 'register'` How to solve this? ---
**Author:** ![Sun-Xiaohui](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/sun-xiaohui/32/197_2.png) [@Sun-Xiaohui](https://discuss.vllm.ai/u/Sun-Xiaohui)\ **Post date:** [July 15, 2026, 2:10am UTC](https://discuss.vllm.ai/t/registering-with-transformers-auto-classes-for-out-of-tree-models/1229/5 "2026-07-15T02:10:15Z")
OK, Solved. It’s due to NameSpace Packages conflict.