Features

Ray Reasoning Outputs Quantization LoRA Tool Calling Disaggregated Prefilling Multi-modality KV-Cache Any topics related to KV-cache. For examples: Prefix caching, Hybrid KV cache, KV cache offloading, KV cache transfer, etc Scheduling Any topics relate to scheduling. For examples: Scheduling policies, Scheduler behaviors, Pluggable scheduler, etc Structured Outputs Speculative Decoding