Mistral-small-3.2: Unable to locate consolidated.safetensors.index.json

Model Support
1

When I’m attempting to serve `mistral-small-3.2`, vllm is unable to locate consolidated.safetensors.index.json

Error: (EngineCore_DP0 pid=63) INFO 02-08 06:51:32 [weight_utils.py:567] No consolidated.safetensors.index.json found in remote.

Serve Command -
`vllm serve mistralai/Mistral-Small-3.2-24B-Instruct-2506
–tokenizer_mode mistral --config_format mistral
–load_format mistral --tool-call-parser mistral
–enable-auto-tool-choice --limit-mm-per-prompt ‘{“image”:10}’
–tensor-parallel-size 2`

2

The error “No consolidated.safetensors.index.json found in remote” means vLLM is looking for the consolidated weights index file required for Mistral models in “mistral” format, but cannot find it in the specified model directory or remote repository. This file is necessary for loading the model in mistral format, and its absence will prevent successful model loading. According to vLLM documentation and code, vLLM expects this file when using --load_format mistral and will fail if it’s missing.

To resolve this, ensure that the model directory contains both the consolidated.safetensors weight shards and the consolidated.safetensors.index.json file. If you downloaded the model manually, re-download it using the Hugging Face CLI or ensure all files are present. If the file is missing from the Hugging Face repo, you may need to wait for the model provider to upload it or use a different model version that includes the required files. This requirement is also discussed in related GitHub issues.

Would you like more detail on how to check or download the required files?

Sources:

Hint: Mention @RunLLM in the post for followups.