When using a model as a classifier in vllm, I’m curious about the probability of occurrence for each label. How can I check the log probability?
To get the log probability for each label when using a classification model in vLLM, use the LLM.classify method, which returns class probabilities by default. To obtain logits (pre-softmax values), set softmax: False in the override_pooler_config parameter. You can then manually apply log-softmax to these logits to get log probabilities for each label. Example:
from vllm import LLM
import torch
llm = LLM(model="your-model", task="classify", override_pooler_config={"softmax": False})
outputs = llm.classify(["your prompt"])
logits = outputs[0].outputs.probs
log_probs = torch.log_softmax(torch.tensor(logits), dim=-1)
print(log_probs)
According to this GitHub issue, setting softmax: False ensures the output is logits, not probabilities. The documentation confirms that LLM.classify outputs probabilities by default, so you must convert logits to log probabilities yourself if you need them in log-space.
Would you like more detail or a full example?
Sources: