It seems likely that the reason you are seeing all zeros or NaNs when probing the weights in make_layers() is because, at that point in the vLLM initialization, the model weights have not yet been loaded from the checkpoint files. The make_layers() function only constructs the module objects; the actual parameter values are loaded later by the weight loading logic (e.g., via AutoWeightsLoader.load_weights()), which happens after the modules are created and returned from make_layers() (source, source).
To properly inspect or modify the weight tensors, you should do so after the weights have been loaded—typically by overriding or inserting your logic in the model’s load_weights() method, or by using hooks after model initialization is complete. Printing weights in make_layers() will only show uninitialized (default) values, not the actual checkpointed weights (source, source).
Would you like more detail on how to hook into the weight loading process or where to place your inspection code for correct results?
Sources: