# Zerank - deploying using vllm

**URL:** <https://discuss.vllm.ai/t/zerank-deploying-using-vllm/1793>\
**Category:** Model Support\
**Created:** [October 29, 2025, 8:26am UTC](https://discuss.vllm.ai/t/zerank-deploying-using-vllm/1793 "2025-10-29T08:26:42Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![WarriorOfLiberation](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/warriorofliberation/32/664_2.png) [@WarriorOfLiberation](https://discuss.vllm.ai/u/WarriorOfLiberation)\
**Post date:** [October 29, 2025, 8:26am UTC](https://discuss.vllm.ai/t/zerank-deploying-using-vllm/1793/1 "2025-10-29T08:26:42Z")

</div>

How to host zeroentropy’s zerank-small using vllm?

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [October 29, 2025, 8:26am UTC](https://discuss.vllm.ai/t/zerank-deploying-using-vllm/1793/2 "2025-10-29T08:26:56Z")

</div>

To host zeroentropy’s zerank-small with vLLM, you need to use the OpenAI-compatible server by running the `vllm serve` command and specifying the model path. For example:

```bash
vllm serve zeroentropy/zerank-small

```

This will start the vLLM server and expose OpenAI-compatible endpoints for inference. You can then query the server using the OpenAI API format. For more details, see the [Quickstart guide](https://docs.vllm.ai/en/latest/getting_started/quickstart.html#openai-compatible-server). If you have a local model directory, provide its path instead of the Hugging Face repo name.

Would you like more detail on querying the server or advanced configuration?

Sources:

- [Quickstart guide](https://docs.vllm.ai/en/latest/getting_started/quickstart.html#openai-compatible-server)

Hint: Mention @RunLLM in the post for followups.

---

<div class="post-metadata">

**Author:** ![WarriorOfLiberation](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/warriorofliberation/32/664_2.png) [@WarriorOfLiberation](https://discuss.vllm.ai/u/WarriorOfLiberation)\
**Post date:** [October 29, 2025, 8:28am UTC](https://discuss.vllm.ai/t/zerank-deploying-using-vllm/1793/3 "2025-10-29T08:28:37Z")

</div>

@RunLLM its a reranker are you sure? Is it even supported

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [October 29, 2025, 8:29am UTC](https://discuss.vllm.ai/t/zerank-deploying-using-vllm/1793/4 "2025-10-29T08:29:02Z")

</div>

zerank-small is a reranker model, but as of now, it is not listed among the officially supported reranker/cross-encoder models in vLLM’s documentation or support tables. Only specific architectures like BGE, Qwen2, Qwen3, JinaVL, and others are confirmed as supported rerankers in vLLM. There is no mention of zeroentropy/zerank-small in the [supported models list](https://docs.vllm.ai/en/latest/models/supported_models.html) or [reranker documentation](https://github.com/vllm-project/vllm/blob/main/docs/models/supported_models.md).

Would you like more detail on how to check or request support for new models?

Sources:

- [supported\_models.md](https://github.com/vllm-project/vllm/blob/main/docs/models/supported_models.md)
- [supported models list](https://docs.vllm.ai/en/latest/models/supported_models.html)

Hint: Mention @RunLLM in the post for followups.