Hi I am having troubles using qwen3.5 4B with vllm.
Here is my config
docker run -d \
--runtime nvidia \
--gpus 1 \
--name vllm-demo \
-v ./cache_models:/root/.cache/huggingface \
--env "HF_TOKEN=$HF_TOKEN" \
-p 8000:8000 \
--ipc=host \
vllm/vllm-openai:latest \
Qwen/Qwen3.5-4B \
--mm-encoder-tp-mode data \
--mm-processor-cache-type shm \
--reasoning-parser qwen3 \
--enable-prefix-caching
After I start the docker container, I tried it with below code
# Modify OpenAI's API key and API base to use vLLM's API server.
openai_api_key = "EMPTY"
openai_api_base = "http://localhost:8000/v1"
messages = [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Who won the world series in 2020?"},
{
"role": "assistant",
"content": "The Los Angeles Dodgers won the World Series in 2020.",
},
{"role": "user", "content": "Where was it played?"},
]
def main(stream: bool):
client = OpenAI(
# defaults to os.environ.get("OPENAI_API_KEY")
api_key=openai_api_key,
base_url=openai_api_base,
)
chat_completion = client.chat.completions.create(
messages=messages,
model="Qwen/Qwen3.5-4B",
stream=stream,
max_tokens=3000,
# extra_body={"reasoning_effort": "none"}
)
return chat_completion
out = main(stream=False)
print(dict(out.choices[0].message))
But whatever max_tokens I used in openai client I only got content=None
{'content': None,
'refusal': None,
'role': 'assistant',
'annotations': None,
'audio': None,
'function_call': None,
'tool_calls': [],
'reasoning': 'Thinking Process:\n\n1. **Analyze the Request:** The user is asking about the location where the 2020 World Series was played.\n\n2. **Retrieve Knowledge:** Access knowledge about the 2020 MLB World Series.\n * Teams: Los Angeles Dodgers vs. Tampa Bay Rays.\n * Location: Due to the pandemic (COVID-19) and empty stadiums, the game was played without spectators at Minute Maid Park in Houston, but the hosting city is crucial. Wait, no. Two series? No.\n * Correction: In 2020, it was a "designated playoff series" where teams were designated. But actually, the question is about the home fields of both games. The Series was a best of five, played behind closed doors.\n * Wait, check specific details. The 2020 World Series games were played at Minute Maid Park in Houston, Texas? No, that was the ALDS.\n * Let\'s verify the 2020 World Series venue.\n * Game 1-2 were at T-Mobile Park (Seattle)? No.\n * Actually, both teams were winning their respective divisions, so they played games in Seattle? Wait, the Dodgers were a Dodgers player\'s team, but where did they play?\n * Wait, the question is about the World Series.\n\n * *Correction*: The 2020 World Series between the LA Dodgers and the Tampa Bay Rays was played at **Tropicana Field** in **St. Petersburg, Florida** before all 5 games? No.\n * Let\'s recall. It was played in multiple venues?\n * Actually, looking at a category specific fact. 2020 World Series was played: Game 1 at Minute Maid Park in Houston? No.\n * Wait, I recall something about the games being played in a specific location due to the empty stadium policy. Actually, the 2020 World Series was in "St. Petersburg, Florida" for all 5 games?\n * Let me check my internal knowledge base carefully.\n * Fact: The 2020 World Series was played in **St. Petersburg, Florida** (Tropicana Field)? No, that was a different tournament or a minor league thing.\n * Fact: The 2020 World Series took place at **Minute Maid Park in Houston**?\n * Let\'s think about the specific stadiums.\n * Game 1: Dodgers at Tampa Bay.\n * Game 2: Tampa Bay at Los Angeles.\n * Wait, the 2020 World Series was hosted in **St. Petersburg, Florida**? No.\n * Wait, I remember a trivia point about it being the first time MLB played all games in a specific city, but I might be mixing it up with the Philippine World Series or something.\n * Wait, no. The 2020 World Series was played entirely at **T-Mobile Park**? No.\n * Okay, let\'s step back. The teams were Los Angeles Dodgers and Tampa Bay Rays.\n * Games 1-4? No.\n * The 2020 World Series games were played in: **T-Mobile Park** (Seattle), **Minute Maid Park** (Houston)?\n * Actually, I think it was **Hamilton**? No.\n * Wait, I am confusing it with the 2021 All-Star Game or something.\n * Let me recall: The 2020 World Series took place behind closed doors. Where were the stadiums located?\n * Game 1 and 2 were at Minute Maid Park in Houston?\n * No, the 2020 World Series was played in **St. Petersburg**? No.\n * Okay, let\'s check.\n * 2020 World Series: Los Angeles Dodgers vs. Tampa Bay Rays.\n * Stadiums:\n * Game 1: **Tropicana Field**? No.\n * Game 2: **Tropicana Field**?\n * Actually, in the 2020 World Series, the games were played entirely at **T-Mobile Park** in Seattle? No.\n * Wait, I remember now. Both games were played in **St. Petersburg, Florida**? No.\n\n * Okay, let\'s be precise.\n * The stadiums used were:\n * Game 1: **Tampa Bay Rays at Game One** -> **Tropicana Field**. Wait, Rays home is Tropicana Field.\n * Game 2: **Dodgers** -> **Dodger Stadium**.\n * Game 3: **Dodgers** -> **Dodger Stadium**.\n * Game 4: **Rays** -> **Tropicana Field**.\n * Game 5: ...\n * Wait, the 2020 World Series was played in **St. Petersburg**? No.\n * Okay, I need to stop guessing and recall the exact venue details.\n * In 2020, due to the pandemic and the specific scheduling, the **World Series** was played at **Tropicana Field** (St. Petersburg, Florida) for all games? No, that was weird.\n * Wait, I am thinking of the first all-home games (Hideout).\n * Let me check the specific location of the 2020 World Series games.\n * It was played behind closed doors in: **T-Mobile Park**, **Minute Maid Park**, **Wrigley Field**, **Cowboys Field**? No.\n * Wait, I am mixing up the MLB playoffs with the World Series.\n * Actually, the 2020 World Series took place at:\n * Game 1: Tropicana Field (St. Petersburg, FL)?\n * Game 2: Dodger Stadium (LA)?\n * Game 3: Dodger Stadium (LA)?\n * Game 4: Tropicana Field (St. Petersburg, FL)?\n * Game 5: Tropicana Field (St. Petersburg, FL)?\n * Wait, the winner was the Dodgers.\n * Wait, the games were played behind closed doors.\n * Let\'s search memory for "2020 World Series venue".\n * Common knowledge: The games were played in **"St. Petersburg, Florida"**? No.\n * Wait, a very specific fact: **The 2020 World Series was played entirely at T-Mobile Park in Seattle?** No.\n * Wait, the 2020 World Series was played at **Tropicana Field**?\n * Wait, actually, it seems the series was played at multiple stadiums, but one specific one was mentioned? No.\n * Wait, I recall the series was played in **Houston** (Minute Maid Park). No.\n\n * *Wait, major correction*: The 2020 World Series was played in **St. Petersburg, Florida** (Tropicana Field). Wait, is that right? No.\n * Okay, let\'s try again.\n * **Verdict**: The 2020 World Series was played in **St. Petersburg, Florida**? No.\n * **Verdict**: The 2020 World Series was played in **Los Angeles, California**? No.\n * **Verdict**: The 2020 World Series was played in **St. Petersburg**? No.\n * Okay, the correct answer is: The games were played at **T-Mobile Park**? No.\n * Wait, the 2020 World Series was played in **St. Petersburg** because the Rays had their home games? No.\n * Wait, the 2020 World Series was played at **Tropicana Field**?\n * Wait, actually, the 2020 World Series was played at **T-Mobile Park** in Seattle. No.\n * Wait, I am stuck in a loop.\n * Let\'s try to access specific details. 2020 World Series, Dodgers vs. Rays.\n * Games were played in **St. Petersburg, Florida**? No.\n * Games were played in **T-Mobile Park**? No.\n * Games were played in **Minute Maid Park**? No.\n * Games were played in **Dodger Stadium**? No.\n * Wait, the 2020 World Series had a unique format. All 5 games? No, best of 5. No, best of 3? No, best of 5.\n * Wait, I recall the series was played in **St. Petersburg, Florida**.\n * Wait, actually, I think the series was played in **stadiums in St. Petersburg**? No.\n * Okay, let\'s look at the facts.\n * **Fact Check**: The 2020 World Series was played in **St. Petersburg, Florida**?\n * Wait, I found a reference in my memory that the 2020 World Series was played **behind closed doors**.\n * Wait, the specific venue was **Tropicana Field**?\n * Wait, no. The 2020 World Series was played in **Tx** (Texas).\n * Wait, I think the correct answer is **Houston, Texas**. (Minute Maid Park).\n * Wait, no. The 2020 World Series was played in **St. Petersburg**.\n * Wait, I need to be 100% sure.\n * Let\'s recall: The 2020 MLB playoffs were held without fans.\n * Games were played at **St. Petersburg**? No.\n * Okay, I think the series was played at **T-Mobile Park** in Seattle? No.\n * Wait, I\'m confusing the 2020 ALDS with the 2020 World Series.\n * Let\'s think. The 2020 World Series was Los Angeles Dodgers vs. Tampa Bay Rays.\n * The games were played at **Tropicana Field** and **Dodger Stadium**?\n * Wait, the first two games?\n * Wait, actually, all games were played at **T-Mobile Park**? No.\n * Wait, the 2020 World Series was played in **St. Petersburg, Florida**?\n * Actually, I recall a trivia question where the answer was **St. Petersburg, Florida** because of Tropicana Field. But that feels wrong for the Dodgers vs. Rays.\n * Wait, actually, that year, the 2020 World Series was played in **St. Petersburg, Florida** (Tropicana Field). No, that was a specific playoff series, maybe the ALDS?\n * Okay, let\'s accept that I might be misremembering the "St. Petersburg" bit.\n * Let\'s try a different angle. The teams were LA Dodgers and Tampa Bay Rays.\n * Usually, the Boston vs Yankees style?\n * Okay, let\'s recall the actual set of games.\n * Game 1: **Tampa Bay** vs **LA Dodgers**.\n * Wait, where was **Tampa Bay**\'s game played? **Tropicana Field**?\n * Game 2: **LA Dodgers** vs **Tampa Bay**.\n * Wait, where was the Dodgers\' game played? **Dodger Stadium**?\n * Wait, I think the games were played at **Minute Maid Park** in Houston. That was the 2020 ALDS.\n * Wait, no.\n * Wait, the 2020 World Series was played in **St. Petersburg, Florida**?\n * Actually, I think I remember a trivia point: **The 2020 World Series was played in St. Petersburg, Florida.** No, that\'s wrong.\n * Wait, let\'s check the location of the stadiums used.\n * Actually, the 2020 World Series was played in **St. Petersburg, Florida**? No.\n * Wait, I found a memory: It was played in **St. Petersburg, Florida**.\n * Wait, actually, it was played at **T-Mobile Park**?\n * Wait, the correct answer is **St. Petersburg, Florida**.\n * Wait, no. It was played in **Tx**.\n * Okay, I need to resolve this.\n * Actually, the correct answer is: **The 2020 World Series was played in **St. Petersburg, Florida.** (at Tropicana Field). Wait, that was the 2020 ALDS? No.\n * Wait, actually, the 2020 World Series was played at **T-Mobile Park** in Seattle? No.\n * Wait, I think I remember now. The 2020 World Series was played in **St. Petersburg, Florida**. No.\n * Okay, let\'s try to verify via logic.\n * Los Angeles Dodgers Home: Dodger Stadium.\n * Tampa Bay Rays Home: Tropicana Field.\n * Games were played behind closed doors.\n * Did they rotate?\n * Game 1: Dodger Stadium.\n * Game 2: Tropicana Field?\n * Wait'}
It seem like the reasoning parser of qwen3 did not work properly here.
Could someone help me please?