[Developer Question] How are the position indices updated?

General
1

So I am right now in the process of adding some guide custom model support to VLLM which requires me to keep track of which input_ids belong to which request_id. My assumption is that something similar is happening for the positions. So I tried to track back how they are set and updated.

positions seems to be a slice of self.positions which is updated with self.positions_cpu. But I just can’t figure out how self.positions_cpu is being updated. I string searched the entire code base and it is not a common variable name. So my assumption is it somewhere needs to be updated in place, but I have no idea where.

can someone maybe explain to me how this is being handled?

Thank you for your help in advance!

2

Ah I just found it out myself, it is happening even in the same file in _prepare_inputs. And what I need to correlate the input tokens with the request IDs is req_indices.

3

Nope, so apparently req_indices does not correlate to the request_ids, sad

4

Obviously it was not that it is just a slice of a pre-allocated arrange

5

Oh, self.input_batch has a req_id_to_index mapping dict

Nice