# Max\_tokens\_per\_doc support for rerank models
**URL:** https://discuss.vllm.ai/t/max-tokens-per-doc-support-for-rerank-models/2253
**Category:** General
**Created:** [January 21, 2026, 6:39pm UTC](https://discuss.vllm.ai/t/max-tokens-per-doc-support-for-rerank-models/2253 "2026-01-21T18:39:26Z")
**Posts on this page:** 2
**Page:** 1
### Author:  [@hustxiayang](https://discuss.vllm.ai/u/hustxiayang)
#### Post date: [January 21, 2026, 6:39pm UTC](https://discuss.vllm.ai/t/max-tokens-per-doc-support-for-rerank-models/2253/1 "2026-01-21T18:39:26Z")
@RunLLM it seems that in vllm, rerank models do not support `max_tokens_per_doc`, but instead `truncate_prompt_tokens`, so vLLM can not truncate each doc, but truncate the whole context, right?
---
### Author:  [@RunLLM](https://discuss.vllm.ai/u/RunLLM)
#### Post date: [January 21, 2026, 6:39pm UTC](https://discuss.vllm.ai/t/max-tokens-per-doc-support-for-rerank-models/2253/2 "2026-01-21T18:39:50Z")
Correct. In vLLM, there is no `max_tokens_per_doc` parameter for rerank models; instead, you can use `truncate_prompt_tokens`, which truncates the entire input prompt (i.e., the whole context), not each document individually. This means vLLM does not natively support truncating each document within a batch for rerank models—truncation applies to the total input sequence only, not per-document granularity. This limitation is discussed in detail in related issues and code references, confirming that per-document truncation is not currently implemented in vLLM for rerank tasks, and any such logic would need to be handled client-side or by preprocessing inputs before sending to vLLM[^1^][^2^][^3^].
Would you like more detail or example code for handling per-document truncation outside vLLM?
Sources:
- [vllm/issues/20300](https://github.com/vllm-project/vllm/issues/20300)
- [vllm/issues/11568](https://github.com/vllm-project/vllm/issues/11568)
Hint: Mention @RunLLM in the post for followups.