Vllm-omni cannot load z-image-turbo

Model Support
1

docker image: vllm-omni:v0.11.0rc1
model: Tongyi-MAI/Z-Image-Turbo
command: vllm serve /model/Z-Image-Turbo --served-model-name z-image-turbo --port 18005 --host 0.0.0.0 --tensor-parallel-size 1 --max-model-len 32768 --enable-chunked-prefill --gpu-memory-utilization 0.90 --trust-remote-code --omni’
Error:
APIServer pid=7) pydantic_core._pydantic_core.ValidationError: 1 validation error for ModelConfig
(APIServer pid=7) Value error, Invalid repository ID or local directory specified: ‘/model/Z-Image-Turbo’.
(APIServer pid=7) Please verify the following requirements:
(APIServer pid=7) 1. Provide a valid Hugging Face repository ID.
(APIServer pid=7) 2. Specify a local directory that contains a recognized configuration file.
(APIServer pid=7) - For Hugging Face models: ensure the presence of a ‘config.json’.
(APIServer pid=7) - For Mistral models: ensure the presence of a ‘params.json’.
(APIServer pid=7) 3. For GGUF: pass the local path of the GGUF checkpoint.
(APIServer pid=7) Loading GGUF from a remote repo directly is not yet supported.
(APIServer pid=7) [type=value_error, input_value=ArgsKwargs((), {‘model’: …rocessor_plugin’: None}), input_type=ArgsKwargs]

image
image1748×1132 182 KB

2

The error means vLLM cannot find a valid config file in your model directory. For Hugging Face models, vLLM requires a config.json file, but your directory only has configuration.json and model_index.json, which are not recognized by vLLM as valid configuration files. You must have a config.json (for Hugging Face) or params.json (for Mistral) in the model directory for vLLM to load the model successfully. Rename or create a config.json as needed, and ensure it matches the expected Hugging Face format. See this requirement and detailed config logic.

Would you like more detail on how to create or convert a config file for vLLM?

Sources:

Hint: Mention @RunLLM in the post for followups.

3

@RunLLM Tongyi didn’t provide the config.json in the model repo, either HF or Modelscope.

4

If the model repo does not provide a config.json, vLLM cannot load the model directly—this file is required for Hugging Face models, as vLLM uses it to determine model architecture and parameters. Without it, you will encounter errors like “does not appear to have a file named config.json” and vLLM will not proceed with loading the model. This is a hard requirement according to vLLM documentation and issue discussions.

You must manually create a compatible config.json (possibly by adapting one from a similar model) and place it in the model directory for vLLM to recognize and load the model. There is no workaround in vLLM for missing config files.

Sources:

Hint: Mention @RunLLM in the post for followups.