# How to get probs directly **URL:** \ **Category:** General\ **Created:** [July 25, 2025, 11:03am UTC](https://discuss.vllm.ai/t/how-to-get-probs-directly/1175 "2025-07-25T11:03:06Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![Yaawennn](https://avatars.discourse-cdn.com/v4/letter/y/a5b964/32.png) [@Yaawennn](https://discuss.vllm.ai/u/Yaawennn)\ **Post date:** [July 25, 2025, 11:03am UTC](https://discuss.vllm.ai/t/how-to-get-probs-directly/1175/1 "2025-07-25T11:03:06Z")
In VLLM V1, how can I modify it to directly obtain probs and skip logprobs calculation? ---
**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\ **Post date:** [July 25, 2025, 11:03am UTC](https://discuss.vllm.ai/t/how-to-get-probs-directly/1175/2 "2025-07-25T11:03:26Z")
In vLLM V1, logprobs are returned immediately from the model’s raw output, before any post-processing, and there is currently no built-in option to skip logprobs calculation and return only probabilities (“probs”) directly. The logprobs calculation is part of the core output pipeline, and support for post-sampling adjustments or direct probability output is not yet available, but is planned for future updates. You would need to modify the internal code (e.g., in the sampler) to extract and return the softmax probabilities (`probs`) instead of or in addition to logprobs, as there is no user-facing API or parameter for this in V1 as of now. See [V1 User Guide: Logprobs](https://docs.vllm.ai/en/latest/getting_started/v1_user_guide.html#logprobs) and [logprobs API docs](https://docs.vllm.ai/en/latest/api/vllm/vllm.v1.engine.logprobs.html). Would you like more detail on which part of the code to modify for this purpose? Sources: - [V1 User Guide: Logprobs](https://docs.vllm.ai/en/latest/getting_started/v1_user_guide.html#logprobs) - [logprobs API docs](https://docs.vllm.ai/en/latest/api/vllm/vllm.v1.engine.logprobs.html) ---
**Author:** ![Yaawennn](https://avatars.discourse-cdn.com/v4/letter/y/a5b964/32.png) [@Yaawennn](https://discuss.vllm.ai/u/Yaawennn)\ **Post date:** [July 25, 2025, 11:07am UTC](https://discuss.vllm.ai/t/how-to-get-probs-directly/1175/3 "2025-07-25T11:07:31Z")
Yes, I want to know where the code in v1 implements the logprobs calculation? And where can I modify it to calculate probs instead of logprobs. Please tell me! ---
**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\ **Post date:** [July 25, 2025, 11:07am UTC](https://discuss.vllm.ai/t/how-to-get-probs-directly/1175/4 "2025-07-25T11:07:48Z")
In vLLM V1, logprobs calculation is implemented in the sampler, specifically in the `compute_logprobs` method of the `Sampler` class (see [API docs](https://docs.vllm.ai/en/latest/api/vllm/vllm.v1.engine.logprobs.html)). The logprobs are computed from the model’s raw logits using `torch.log_softmax`, and this happens before any post-processing like temperature or top-k/top-p is applied. To modify vLLM V1 to return probabilities (`probs`) instead of logprobs, you should locate the `compute_logprobs` method in `vllm/v1/sample/sampler.py` and change the calculation from `torch.log_softmax(logits, dim=-1)` to `torch.softmax(logits, dim=-1)`. You will also need to update the output structure to return these probabilities instead of logprobs. Would you like a step-by-step example of how to make this code change? Sources: - [logprobs API docs](https://docs.vllm.ai/en/latest/api/vllm/vllm.v1.engine.logprobs.html) - [Sampler logic location confirmation](https://github.com/vllm-project/vllm/issues/11397) ---
**Author:** ![Yaawennn](https://avatars.discourse-cdn.com/v4/letter/y/a5b964/32.png) [@Yaawennn](https://discuss.vllm.ai/u/Yaawennn)\ **Post date:** [July 27, 2025, 11:17am UTC](https://discuss.vllm.ai/t/how-to-get-probs-directly/1175/5 "2025-07-27T11:17:21Z")
I saw this code in the v1 sampler, and I found that the Sampler class can specify the tokens to process by passing the sampling\_metadata: SamplingMetadata parameter; How is this part used? During generation, I only want to get probs for some tokens ---
**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\ **Post date:** [July 27, 2025, 11:17am UTC](https://discuss.vllm.ai/t/how-to-get-probs-directly/1175/6 "2025-07-27T11:17:37Z")
You are correct: in vLLM V1, the `Sampler` class receives a `sampling_metadata: SamplingMetadata` parameter, which contains information about which tokens to process, including allowed token masks and other per-token controls ([Sampler API](https://docs.vllm.ai/en/stable/api/vllm/vllm.v1.sample.sampler.html), [SamplingMetadata API](https://docs.vllm.ai/en/stable/api/vllm/vllm.v1.sample.metadata.html)). To get probabilities (`probs`) for only specific tokens during generation, you can use the `allowed_token_ids_mask` field in `SamplingMetadata` to mask out unwanted tokens, so the softmax is only computed over the allowed tokens. However, there is no built-in user-facing API to return `probs` for arbitrary token subsets; you would need to customize the sampler code to extract and return the probabilities for your desired tokens after applying the mask. This involves modifying the `Sampler.forward` method to gather and output `probs` for the indices specified by your mask in `sampling_metadata.allowed_token_ids_mask`. Would you like a code example for how to do this modification? Sources: - [Sampler API](https://docs.vllm.ai/en/stable/api/vllm/vllm.v1.sample.sampler.html) - [SamplingMetadata API](https://docs.vllm.ai/en/stable/api/vllm/vllm.v1.sample.metadata.html) ---
**Author:** ![Yaawennn](https://avatars.discourse-cdn.com/v4/letter/y/a5b964/32.png) [@Yaawennn](https://discuss.vllm.ai/u/Yaawennn)\ **Post date:** [July 28, 2025, 6:26am UTC](https://discuss.vllm.ai/t/how-to-get-probs-directly/1175/7 "2025-07-28T06:26:07Z")
I want to specify in the Sampler class how the logprobs are calculated based on my custom parameter sampling\_metadata.extra\_args. But in v1, we can’t pass extra\_args as a parameter, as we can in v0. How should I implement it? ---
**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\ **Post date:** [July 28, 2025, 6:26am UTC](https://discuss.vllm.ai/t/how-to-get-probs-directly/1175/8 "2025-07-28T06:26:38Z")
In vLLM V1, you can pass custom arguments for sampling via the `extra_args` field in `SamplingParams`, which are then accessible in `sampling_metadata.extra_args` within the `Sampler` class. This is supported in V1 as of recent updates, and these custom arguments can be set through the REST API using the `vllm_xargs` field in your request ([RFC: Custom sampling params](https://github.com/vllm-project/vllm/issues/17191), [PR: Expose custom args in OpenAI APIs](https://github.com/vllm-project/vllm/pull/16862)). To implement custom logprobs calculation, add your custom parameter to `vllm_xargs` in your API request. Then, in your custom `Sampler` logic, access it via `sampling_metadata.extra_args` and branch your logprobs calculation accordingly. Would you like a code snippet showing how to access and use `extra_args` in the Sampler? Sources: - [RFC: Custom sampling params](https://github.com/vllm-project/vllm/issues/17191) - [PR: Expose custom args in OpenAI APIs](https://github.com/vllm-project/vllm/pull/16862) ---
**Author:** ![Yaawennn](https://avatars.discourse-cdn.com/v4/letter/y/a5b964/32.png) [@Yaawennn](https://discuss.vllm.ai/u/Yaawennn)\ **Post date:** [July 28, 2025, 6:30am UTC](https://discuss.vllm.ai/t/how-to-get-probs-directly/1175/9 "2025-07-28T06:30:53Z")
Please tell me how the parameters in SamplingParams are passed to Metadata in v1, and where is the corresponding code ---
**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\ **Post date:** [July 28, 2025, 6:31am UTC](https://discuss.vllm.ai/t/how-to-get-probs-directly/1175/10 "2025-07-28T06:31:09Z")
In vLLM V1, parameters in `SamplingParams` are passed to `SamplingMetadata` during batch preparation. This mapping occurs in the input batch logic, specifically in the `InputBatch` class (see [InputBatch code reference](https://github.com/vllm-project/vllm/blob/main/vllm/v1/worker/gpu_input_batch.py)), where each request’s `SamplingParams` is used to populate fields in `SamplingMetadata` via the `_make_sampling_metadata()` method. This method collects per-request sampling parameters and constructs the `SamplingMetadata` object, which is then passed to the sampler. For example, in `gpu_input_batch.py`, the `_make_sampling_metadata()` method gathers fields like temperature, top\_k, top\_p, penalties, and any `extra_args` from `SamplingParams` and sets them in the resulting `SamplingMetadata` instance ([SamplingMetadata API](https://docs.vllm.ai/en/stable/api/vllm/vllm.v1.sample.metadata.html)). Would you like a code snippet showing this mapping process? Sources: - [SamplingMetadata API](https://docs.vllm.ai/en/stable/api/vllm/vllm.v1.sample.metadata.html) ---
**Author:** ![Yaawennn](https://avatars.discourse-cdn.com/v4/letter/y/a5b964/32.png) [@Yaawennn](https://discuss.vllm.ai/u/Yaawennn)\ **Post date:** [July 28, 2025, 6:39am UTC](https://discuss.vllm.ai/t/how-to-get-probs-directly/1175/11 "2025-07-28T06:39:06Z")
v1 can also specify allowed sampling tokens right? How to do it, give an example ---
**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\ **Post date:** [July 28, 2025, 6:39am UTC](https://discuss.vllm.ai/t/how-to-get-probs-directly/1175/12 "2025-07-28T06:39:28Z")
Yes, vLLM V1 supports specifying allowed sampling tokens using the `allowed_token_ids` parameter in `SamplingParams`. This restricts generation to only the provided token IDs ([SamplingParams API](https://docs.vllm.ai/en/stable/api/vllm/vllm.sampling_params.html), [Sampler API](https://docs.vllm.ai/en/stable/api/vllm/vllm.v1.sample.sampler.html)). Example usage in Python: ```python from vllm import LLM, SamplingParams llm = LLM(model="facebook/opt-125m") params = SamplingParams(allowed_token_ids=[42, 43, 44], max_tokens=10) outputs = llm.generate("Hello", params) print(outputs[0].outputs[0].text) ``` Would you like more detail on how this works internally? Sources: - [Sampler API](https://docs.vllm.ai/en/stable/api/vllm/vllm.v1.sample.sampler.html) - [SamplingParams API](https://docs.vllm.ai/en/stable/api/vllm/vllm.sampling_params.html)