# 调用vllm的python接口，推理Qwen3-VL模型

**URL:** <https://discuss.vllm.ai/t/vllm-python-qwen3-vl/2083>\
**Category:** General\
**Created:** [December 5, 2025, 9:42am UTC](https://discuss.vllm.ai/t/vllm-python-qwen3-vl/2083 "2025-12-05T09:42:53Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![sxcnmslll](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/sxcnmslll/32/946_2.png) [@sxcnmslll](https://discuss.vllm.ai/u/sxcnmslll)\
**Post date:** [December 5, 2025, 9:42am UTC](https://discuss.vllm.ai/t/vllm-python-qwen3-vl/2083/1 "2025-12-05T09:42:53Z")

</div>

我要通过vllm的python接口，实例化一个Qwen3-VL 8B模型。再使用这个模型，对我的多模态数据集进行推理。帮我实现完整的代码

我的多模态数据如下：

{“messages”: [{“role”: “user”, “content”: “这张图片有什么”}, {“role”: “assistant”, “content”: “小狗”}], “images”: [“/xxx/x.jpg”]}  
{“messages”: [{“role”: “user”, “content”: “这张图片有什么”}, {“role”: “assistant”, “content”: “小猫”}], “images”: [“/YYY/Z.jpg”]}

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [December 5, 2025, 9:43am UTC](https://discuss.vllm.ai/t/vllm-python-qwen3-vl/2083/2 "2025-12-05T09:43:24Z")

</div>

目前vLLM官方文档和代码中没有关于Qwen3-VL-8B模型的多模态推理完整示例，但Qwen2-VL和Qwen2.5-VL的多模态推理流程可参考。你需要：1）实例化LLM对象，2）用AutoProcessor处理messages和图片，3）构造prompt和multi\_modal\_data，4）调用llm.generate进行推理。Qwen3-VL-8B的具体支持情况和API参数未在数据源中明确说明，以下代码以Qwen2-VL为例，Qwen3-VL-8B如接口一致可直接套用，否则需查阅Qwen3-VL-8B官方文档。

```python
from transformers import AutoProcessor
from vllm import LLM, SamplingParams
from qwen_vl_utils import process_vision_info
from PIL import Image

MODEL_PATH = "Qwen/Qwen2-VL-7B-Instruct" # Qwen3-VL-8B请替换为实际路径
llm = LLM(model=MODEL_PATH, limit_mm_per_prompt={"image": 1})

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": [
        {"type": "image", "image": "/xxx/x.jpg"},
        {"type": "text", "text": "这张图片有什么"}
    ]}
]
processor = AutoProcessor.from_pretrained(MODEL_PATH)
prompt = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
image_inputs, _ = process_vision_info(messages)
llm_inputs = {"prompt": prompt, "multi_modal_data": {"image": image_inputs}}
sampling_params = SamplingParams(temperature=0.1, max_tokens=256)
outputs = llm.generate([llm_inputs], sampling_params=sampling_params)
print(outputs[0].outputs[0].text)

```

如需批量处理多组数据，将llm\_inputs列表化即可。Qwen3-VL-8B如有特殊参数或API，请参考其官方文档。

需要更详细的逐步说明吗？

Sources:

- [vLLM多模态输入官方文档](https://docs.vllm.ai/en/stable/serving/multimodal_inputs.html)
- [Qwen2-VL多模态推理示例](https://github.com/vllm-project/vllm/issues/10153)
- [Qwen2-VL多模态代码示例](https://github.com/vllm-project/vllm/blob/main/examples/offline_inference/vision_language_multi_image.py)

Hint: Mention @RunLLM in the post for followups.

---

<div class="post-metadata">

**Author:** ![sxcnmslll](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/sxcnmslll/32/946_2.png) [@sxcnmslll](https://discuss.vllm.ai/u/sxcnmslll)\
**Post date:** [December 5, 2025, 9:47am UTC](https://discuss.vllm.ai/t/vllm-python-qwen3-vl/2083/3 "2025-12-05T09:47:30Z")

</div>

@RunLLM  
我的数据集有所变化：

{“messages”: [{“role”: “user”, “content”: “两张图片有什么区别”}, {“role”: “assistant”, “content”: “前一张是小猫，后一张是小狗”}], “images”: [“/xxx/x.jpg”, “/xxx/x.png”]}

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [December 5, 2025, 9:47am UTC](https://discuss.vllm.ai/t/vllm-python-qwen3-vl/2083/4 "2025-12-05T09:47:55Z")

</div>

你可以将多张图片作为列表传递给multi\_modal\_data的"image"字段，并在prompt中用合适的模板插入多个图片占位符。例如：

```python
from transformers import AutoProcessor
from vllm import LLM, SamplingParams
from qwen_vl_utils import process_vision_info

MODEL_PATH = "Qwen/Qwen2-VL-7B-Instruct" # Qwen3-VL-8B请替换为实际路径
llm = LLM(model=MODEL_PATH, limit_mm_per_prompt={"image": 2})

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": [
        {"type": "image", "image": "/xxx/x.jpg"},
        {"type": "image", "image": "/xxx/x.png"},
        {"type": "text", "text": "两张图片有什么区别"}
    ]}
]
processor = AutoProcessor.from_pretrained(MODEL_PATH)
prompt = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
image_inputs, _ = process_vision_info(messages)
llm_inputs = {"prompt": prompt, "multi_modal_data": {"image": image_inputs}}
sampling_params = SamplingParams(temperature=0.1, max_tokens=256)
outputs = llm.generate([llm_inputs], sampling_params=sampling_params)
print(outputs[0].outputs[0].text)

```

如需支持多图，需设置limit\_mm\_per\_prompt={“image”: 2}，并保证prompt模板与图片数量对应。Qwen3-VL-8B如有特殊要求请查阅其官方文档。

需要更详细的解释吗？

Sources:

- [多图推理官方文档](https://docs.vllm.ai/en/stable/serving/multimodal_inputs.html)
- [多图推理代码示例](https://github.com/vllm-project/vllm/blob/main/docs/features/multimodal_inputs.md)

Hint: Mention @RunLLM in the post for followups.

---

<div class="post-metadata">

**Author:** ![sxcnmslll](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/sxcnmslll/32/946_2.png) [@sxcnmslll](https://discuss.vllm.ai/u/sxcnmslll)\
**Post date:** [December 5, 2025, 9:50am UTC](https://discuss.vllm.ai/t/vllm-python-qwen3-vl/2083/5 "2025-12-05T09:50:46Z")

</div>

@RunLLM 执行报错了：  
(vllm) python kk.py  
INFO 12-05 17:49:09 [**init**.py:216] Automatically detected platform cuda.  
Traceback (most recent call last):  
File “/mnt/data/cpfs/lishan/model\_train/kk.py”, line 3, in   
from qwen\_vl\_utils import process\_vision\_info  
ModuleNotFoundError: No module named ‘qwen\_vl\_utils’

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [December 5, 2025, 9:51am UTC](https://discuss.vllm.ai/t/vllm-python-qwen3-vl/2083/6 "2025-12-05T09:51:19Z")

</div>

你的报错是“ModuleNotFoundError: No module named ‘qwen\_vl\_utils’”，说明你的环境中没有安装`qwen-vl-utils`库。你需要先安装它：

```bash
pip install qwen-vl-utils

```

安装后即可正常导入`process_vision_info`。如需更详细的排查步骤请告知。

Sources:

- [qwen-vl-utils安装提示](https://docs.vllm.ai/en/latest/examples/offline_inference/vision_language_multi_image/#__span-0-1038)

Hint: Mention @RunLLM in the post for followups.

---

<div class="post-metadata">

**Author:** ![sxcnmslll](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/sxcnmslll/32/946_2.png) [@sxcnmslll](https://discuss.vllm.ai/u/sxcnmslll)\
**Post date:** [December 5, 2025, 11:07am UTC](https://discuss.vllm.ai/t/vllm-python-qwen3-vl/2083/7 "2025-12-05T11:07:47Z")

</div>

@RunLLM IMAGE\_MAX\_TOKEN\_NUM如何配置？我希望动态适配我实际的图像token数量

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [December 5, 2025, 11:08am UTC](https://discuss.vllm.ai/t/vllm-python-qwen3-vl/2083/8 "2025-12-05T11:08:08Z")

</div>

vLLM并没有直接提供IMAGE\_MAX\_TOKEN\_NUM参数的配置接口。多模态图片token数量的分配，主要通过`limit_mm_per_prompt`和`mm_processor_kwargs`（如max\_pixels、width、height）等参数间接控制。你可以通过设置`mm_processor_kwargs`中的`max_pixels`，动态调整图片token数量的上限，从而更贴合实际图片分辨率。例如：

```python
llm = LLM(
    model="Qwen/Qwen2.5-VL-3B-Instruct",
    limit_mm_per_prompt={"image": 8},
    mm_processor_kwargs={"max_pixels": 512 * 512},
)

```

这样vLLM会按你设定的max\_pixels进行内存和token预算，而不是用模型的最大默认值。注意：这些参数只影响内存profiling和token预算，不会影响实际推理时图片的处理方式。  
如需进一步动态适配不同图片，可在服务启动前根据实际图片分布调整max\_pixels参数，但vLLM不支持每次推理动态更改。

需要更详细的原理说明或代码示例吗？

Sources:

- [内存与token预算配置说明](https://docs.vllm.ai/en/latest/configuration/conserving_memory/#__span-6-5)
- [相关issue讨论](https://github.com/vllm-project/vllm/issues/18550)
- [多模态配置官方文档](https://github.com/vllm-project/vllm/blob/main/docs/configuration/conserving_memory.md)

Hint: Mention @RunLLM in the post for followups.

---

<div class="post-metadata">

**Author:** ![sxcnmslll](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/sxcnmslll/32/946_2.png) [@sxcnmslll](https://discuss.vllm.ai/u/sxcnmslll)\
**Post date:** [December 5, 2025, 11:11am UTC](https://discuss.vllm.ai/t/vllm-python-qwen3-vl/2083/9 "2025-12-05T11:11:51Z")

</div>

@RunLLM 在vllm的日志里，没有看到图像分辨率相关的配置：  
INFO 12-05 19:05:08 [core.py:77] Initializing a V1 LLM engine (v0.11.0) with config: model=‘/msswift\_tmp/Qwen3-VL-8B-Instruct’, speculative\_config=None, tokenizer=‘/msswift\_tmp/Qwen3-VL-8B-Instruct’, skip\_tokenizer\_init=False, tokenizer\_mode=auto, revision=None, tokenizer\_revision=None, trust\_remote\_code=False, dtype=torch.bfloat16, max\_seq\_len=10240, download\_dir=None, load\_format=safetensors, tensor\_parallel\_size=4, pipeline\_parallel\_size=1, data\_parallel\_size=1, disable\_custom\_all\_reduce=False, quantization=None, enforce\_eager=False, kv\_cache\_dtype=auto, device\_config=cuda, structured\_outputs\_config=StructuredOutputsConfig(backend=‘auto’, disable\_fallback=False, disable\_any\_whitespace=False, disable\_additional\_properties=False, reasoning\_parser=‘’), observability\_config=ObservabilityConfig(show\_hidden\_metrics\_for\_version=None, otlp\_traces\_endpoint=None, collect\_detailed\_traces=None), seed=0, served\_model\_name=/msswift\_tmp/Qwen3-VL-8B-Instruct, enable\_prefix\_caching=True, chunked\_prefill\_enabled=True, pooler\_config=None, compilation\_config={“level”:3,“debug\_dump\_path”:“”,“cache\_dir”:“”,“backend”:“”,“custom\_ops”:,“splitting\_ops”:[“vllm.unified\_attention”,“vllm.unified\_attention\_with\_output”,“vllm.mamba\_mixer2”,“vllm.mamba\_mixer”,“vllm.short\_conv”,“vllm.linear\_attention”,“vllm.plamo2\_mamba\_mixer”,“vllm.gdn\_attention”,“vllm.sparse\_attn\_indexer”],“use\_inductor”:true,“compile\_sizes”:,“inductor\_compile\_config”:{“enable\_auto\_functionalized\_v2”:false},“inductor\_passes”:{},“cudagraph\_mode”:[2,1],“use\_cudagraph”:true,“cudagraph\_num\_of\_warmups”:1,“cudagraph\_capture\_sizes”:[512,504,496,488,480,472,464,456,448,440,432,424,416,408,400,392,384,376,368,360,352,344,336,328,320,312,304,296,288,280,272,264,256,248,240,232,224,216,208,200,192,184,176,168,160,152,144,136,128,120,112,104,96,88,80,72,64,56,48,40,32,24,16,8,4,2,1],“cudagraph\_copy\_inputs”:false,“full\_cuda\_graph”:false,“use\_inductor\_graph\_partition”:false,“pass\_config”:{},“max\_capture\_size”:512,“local\_cache\_dir”:null}

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [December 5, 2025, 11:12am UTC](https://discuss.vllm.ai/t/vllm-python-qwen3-vl/2083/10 "2025-12-05T11:12:19Z")

</div>

vLLM日志不会直接显示图片分辨率或相关配置。图片分辨率的处理和token预算，需通过`mm_processor_kwargs`（如max\_pixels、width、height）等参数在实例化LLM时手动指定，否则vLLM会采用模型默认值。日志中只会显示模型、推理参数等全局配置，不会包含图片分辨率细节。

如需动态适配图片分辨率，请在LLM初始化时设置`mm_processor_kwargs`，但相关信息不会自动出现在日志里。如需确认分辨率配置是否生效，需自行检查代码参数或在推理前打印相关变量。

需要更详细的配置说明吗？

Sources:

- [相关issue说明](https://github.com/vllm-project/vllm/issues/18550)

Hint: Mention @RunLLM in the post for followups.

---

<div class="post-metadata">

**Author:** ![sxcnmslll](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/sxcnmslll/32/946_2.png) [@sxcnmslll](https://discuss.vllm.ai/u/sxcnmslll)\
**Post date:** [December 5, 2025, 11:13am UTC](https://discuss.vllm.ai/t/vllm-python-qwen3-vl/2083/11 "2025-12-05T11:13:46Z")

</div>

@RunLLM 告诉我查看分辨率配置是否生效的完整方法

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [December 5, 2025, 11:14am UTC](https://discuss.vllm.ai/t/vllm-python-qwen3-vl/2083/12 "2025-12-05T11:14:12Z")

</div>

vLLM中，判断图片分辨率（如max\_pixels、width、height）配置是否生效，最直接的方法是： **推理前后分别打印mm\_processor\_kwargs参数和图片实际shape，或在process\_vision\_info处理后检查图片尺寸** 。此外，若配置无效，HuggingFace处理器通常会有warning或报错提示。

官方日志不会显示分辨率参数，需在代码中手动print相关变量，或通过处理器返回的图片shape确认。例如：

```python
print(llm.llm_engine.model_config.mm_processor_kwargs)
print(image_inputs[0].shape) # 检查图片shape

```

需要详细代码示例或逐步操作说明吗？

Sources:

- [相关issue说明](https://github.com/vllm-project/vllm/issues/18550)
- [配置参数说明](https://docs.vllm.ai/en/latest/configuration/conserving_memory/#__span-6-5)

Hint: Mention @RunLLM in the post for followups.

---

<div class="post-metadata">

**Author:** ![sxcnmslll](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/sxcnmslll/32/946_2.png) [@sxcnmslll](https://discuss.vllm.ai/u/sxcnmslll)\
**Post date:** [December 5, 2025, 11:16am UTC](https://discuss.vllm.ai/t/vllm-python-qwen3-vl/2083/13 "2025-12-05T11:16:18Z")

</div>

@RunLLM 我想知道，每一次推理的时候，图像占了多少token

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [December 5, 2025, 11:16am UTC](https://discuss.vllm.ai/t/vllm-python-qwen3-vl/2083/14 "2025-12-05T11:16:50Z")

</div>

vLLM不会在日志中直接显示每次推理时图片占用了多少token，但你可以通过输出`outputs[0].prompt_token_ids`的长度，或统计图片token id出现的次数来获知。例如：

```python
print(outputs[0].prompt_token_ids.count(图片token_id))

```

图片token\_id可通过处理器或模型配置获取。这样可精确统计每次推理图片实际占用的token数。[相关讨论](https://github.com/vllm-project/vllm/issues/15364)

需要更详细的代码示例吗？

Sources:

- [相关issue代码与说明](https://github.com/vllm-project/vllm/issues/15364)

Hint: Mention @RunLLM in the post for followups.