Directly setting compilation_config.cudagraph_capture_sizes inside check_and_update_config() is not the recommended way to modify cudagraph capture sizes in a vLLM plugin. The correct approach is to use the init_with_cudagraph_sizes() method, which is designed to update and deduplicate the sizes properly, as shown in your code. Setting the attribute directly may bypass internal checks and deduplication logic, leading to unexpected behavior.
According to the official API and implementation, you should call compilation_config.init_with_cudagraph_sizes(batch_sizes) to update the cudagraph capture sizes safely, rather than assigning to the attribute directly. This ensures the configuration is updated in a way that is consistent with vLLM’s internal logic and future compatibility (source, source).
Would you like a step-by-step breakdown of how init_with_cudagraph_sizes() works and why direct assignment is discouraged?
Sources:
