Hi, I’m trying to use Microsoft Semantic Kernel with vllm however I keep getting an error that the validation has failed due to the presence of the strict option in the request.
The error I am getting is
message=2 validation errors for ChatCompletionRequest
tools.0.function.strict
Extra inputs are not permitted [type=extra_forbidden, input_value=False, input_type=bool]
For further information visit Redirecting...
The request JSON is as follows. Notice the strict: false field. I think this is a new OpenAI protocol feature. Is there a version of vllm that will support this?
{
“tools”: [
{
“function”: {
“description”: “Gets a list of lights and their current state”,
“name”: “Lights-get_lights”,
“strict”: false,
“parameters”: {
“type”: “object”,
“required”: ,
“properties”: {}
}
},
“type”: “function”
},
{
“function”: {
“description”: “Gets a a specified light by name”,
“name”: “Lights-get_light”,
“strict”: false,
“parameters”: {
“type”: “object”,
“required”: [ “lightName” ],
“properties”: { “lightName”: { “type”: “string” } }
}
},
“type”: “function”
},
{
“function”: {
“description”: “Changes the state of the light”,
“name”: “Lights-change_state”,
“strict”: false,
“parameters”: {
“type”: “object”,
“required”: [ “id”, “isOn” ],
“properties”: {
“id”: { “type”: “integer” },
“isOn”: { “type”: “boolean” }
}
}
},
“type”: “function”
}
],
“messages”: [
{
“role”: “system”,
“content”: “You are an AI assistant that can call functions when appropriate. When needed respond in a plain text format”
},
{
“role”: “user”,
“content”: “What is the lights state?”
}
],
“model”: “mistralai/Mistral-7B-Instruct-v0.3”,
“tool_choice”: “auto”
}
Currently, vLLM does not support the strict field in function definitions within the tools parameter. If you include "strict": false (or true), vLLM will return a validation error: “Extra inputs are not permitted” for the strict field, as you observed. This is because the vLLM implementation of function calling does not yet recognize or allow the strict parameter, which was introduced by OpenAI after vLLM’s function calling support was added. There is no released version of vLLM that supports this field as of now, and the feature is tracked as a known limitation and open issue in the vLLM repository (see here, see also).
As a workaround, you must remove the strict field from your request payload to avoid the validation error. If you are using a client library that automatically adds strict, you may need to patch or override its behavior until vLLM adds support for this field. Would you like more detail on the current status or possible workarounds?
While OpenAIBaseModel in vLLM does set extra = "allow" to permit extra fields at the top level, the FunctionDefinition model (used for function tools) does not inherit this setting and instead uses Pydantic’s default, which forbids extra fields. As a result, fields like strict in function definitions are rejected with a validation error, even though top-level extra fields are allowed. This is a known limitation and is discussed in vLLM issues and PRs (see here).
So, the problem is that only models inheriting from OpenAIBaseModel with extra="allow" accept extra fields, but FunctionDefinition does not, causing the error when strict is present. Would you like more detail on the model hierarchy or possible fixes?
I do not think so. I see the definition includes inherit from OpenAiBaseModel, and it should allow extra. I tested with llama3.3-70b-instruct, with strict, it worked fine. I do not think there is such issue?
If you tested with Llama-3.3-70B-Instruct and the request containing the strict field worked, it’s likely because recent changes in vLLM (see PR #10463 and PR #16862) have improved support for extra fields at the top level and possibly in some nested models. However, the official vLLM documentation and issue tracker still indicate that extra fields in nested objects (like function definitions in tools) may not always be accepted, depending on the model and tokenizer mode used. For example, Mistral models with --tokenizer-mode mistral still forbid extra params due to their own parsing logic (see here).
So, while your test case may work for Llama-3.3-70B-Instruct, this is not guaranteed for all models or configurations. The underlying issue is model- and configuration-dependent. Would you like more detail on which models or settings are affected?
Certainly! Here are the details on which models and settings are affected by the handling of extra fields (like strict) in vLLM:
Llama and Most HuggingFace Models:
For models like Llama-3.3-70B-Instruct and other HuggingFace-format models, vLLM’s OpenAI-compatible server now allows extra fields in requests—including nested fields—due to changes in PR #10463 and PR #16862. This means you can include fields like strict in function definitions, and they will be ignored (not cause an error).
Mistral Models with --tokenizer-mode mistral:
If you run Mistral models (e.g., mistralai/Mistral-Large-Instruct-2411) with --tokenizer-mode mistral, extra fields in nested objects (like strict in function definitions) are still forbidden. This is because the Mistral tokenizer uses its own Pydantic base model with extra=forbid, which overrides vLLM’s more permissive settings (see here). As a result, requests with extra fields will fail validation for these models.
Summary Table:
Model Type / Setting
Extra Fields Allowed in Nested Objects?
Llama, HuggingFace models (default)
Yes (fields like strict are ignored)
Mistral with --tokenizer-mode mistral
No (fields like strict cause error)
Workarounds:
For Mistral models, you can try using the HuggingFace format instead of the Mistral format, or avoid passing extra fields in nested objects.
For other models, extra fields should be ignored as expected.
Let me know if you want a code example or more technical details.
Thanks! I would check. Could you please show me the codes in vLLM, where mistra models do not allow extra fields? Does that mean even adding this field into vLLM does not work for mistra models? Thanks
Yes, even if vLLM’s OpenAIBaseModel allows extra fields, Mistral models run with --tokenizer-mode mistral use their own message parsing logic that forbids extra fields in nested objects. This is enforced by the Mistral tokenizer’s dependency on mistral-common, which sets extra = "forbid" in its Pydantic base model.
Relevant vLLM code and explanation:
In vLLM, when you use --tokenizer-mode mistral, message parsing for Mistral models is delegated to the mistral-common library.
The relevant code is in mistral_common/base.py, which sets extra = "forbid" in the Pydantic model config.
vLLM’s own code references this behavior in PR #10463:
“Mistral models ran with --tokenizer-mode mistral use mistral-common for parsing the messages, and they forbid extra params on their side: see here”
Summary:
Even if vLLM is patched to allow extra fields, Mistral models with --tokenizer-mode mistral will still reject extra fields in nested objects due to the strict validation in mistral-common. This cannot be overridden from vLLM alone.
If you are running mistral-nemo-instruct and the request with the strict field works, it likely means you are not using --tokenizer-mode mistral or the model is loaded in HuggingFace format. In this case, vLLM’s OpenAIBaseModel allows extra fields, so strict is ignored and does not cause an error (see discussion).
However, if you run the same model with --tokenizer-mode mistral, extra fields like strict in function definitions will be rejected due to the strict validation in mistral-common (see here). Would you like a code example to illustrate the difference?
You need --tokenizer-mode mistral for official Mistral-format models because it ensures correct encoding and decoding, especially for chat and tool-calling features. The Mistral team recommends this mode for compatibility and accuracy, as HuggingFace tokenizers may not fully support all Mistral-specific behaviors (see docs, see warning).
Without --tokenizer-mode mistral, you may encounter subtle bugs, incorrect outputs, or unsupported features when using Mistral-format checkpoints. The warning is shown at startup if you use a Mistral model without this flag.
No, vLLM does not automatically set --tokenizer-mode mistral for Mistral models; users must specify it manually. This is because some model repositories may contain both Mistral and HuggingFace formats, and auto-selection could cause ambiguity or unexpected behavior (see discussion, see docs). There is an open feature request to improve this, but currently, explicit user input is required.
Would you like more detail on the reasoning or possible future improvements?
If you do not use --tokenizer-mode mistral with official Mistral-format models, you may encounter issues such as incorrect tokenization, broken chat formatting, or failures in tool-calling and structured outputs. For example, users have reported garbled or nonsensical JSON outputs, missing special tokens, and problems with function/tool call parsing (see issue, see docs). The vLLM codebase also emits a warning recommending this flag to avoid such problems.
Would you like more technical details or specific error messages?