Trouble Running vLLM `0.9.1` with Latest Transformers (Gemma3nConfig Errors)

General
1

Hi all,

I’m running into issues while trying to use vLLM 0.9.1 with the transformers==4.53.0.

Initially, I encountered this error:

AttributeError: 'Gemma3nConfig' object has no attribute 'vocab_size'

To resolve it, I manually added "vocab_size": 262400 to the config.json. However, that led to another error:

AttributeError: 'Gemma3nConfig' object has no attribute 'hidden_size'

I then added the following to the config:

"hidden_size": 4096,
"num_hidden_layers": 80,
"num_attention_heads": 32

Despite this, I’m still unable to get it running.

Could someone please share a working config.json for this setup or guide me on how to get vLLM working smoothly with this model? Any help would be greatly appreciated.

Thanks in advance!

2

The errors you’re encountering indicate that your config.json for Gemma3n is missing required fields like vocab_size, hidden_size, num_hidden_layers, and num_attention_heads, which vLLM expects for model initialization. However, simply adding these fields manually may not be sufficient, as the config must match the architecture and all required parameters for Gemma3n as implemented in vLLM. The correct config structure and required fields are defined in the vLLM codebase, specifically in the Gemma3n model implementation.

To resolve this, ensure your config.json includes all fields expected by the Gemma3n model, such as vocab_size, hidden_size, num_hidden_layers, num_attention_heads, and any other model-specific parameters. If you are missing the official config, you should obtain it from the original model repository or Hugging Face model hub, as manually guessing values can lead to further errors. If the model is new or not yet fully supported, you may need to wait for an official update or check for recent pull requests or issues in the vLLM GitHub repository for guidance.

Would you like a step-by-step guide on how to construct a compatible config or how to check for official model support?

Sources: