Is guided decoding used during automatic function calling (–enable-auto-tool-choice)?
As I read vllm doc (Tool Calling - vLLM), it seems that guided decoding is enabled by default only when named function calling or required function calling is used.
Guided decoding is not used during automatic function calling (--enable-auto-tool-choice). Guided decoding is only enabled by default for named function calling (when you specify a function in tool_choice) and for required function calling (tool_choice="required"). For automatic tool choice (tool_choice="auto"), vLLM relies on the model’s own output and does not enforce the schema via guided decoding, so the output may not always strictly match the tool schema. This is explicitly stated in the official documentation and tool calling docs.
Would you like more detail or code examples?
Sources: