Hi there! I am wondering that besides CustomAllreduce, is there any other custom communication operation in vLLM?
Besides CustomAllreduce, there are no other custom communication operations explicitly documented in vLLM. The main custom communication class is CustomAllreduce, which provides a custom all-reduce operation for device tensors, supporting both CUDA and ROCm backends. Other communication operations in vLLM, such as broadcast, all-gather, reduce-scatter, and gather, are implemented using standard PyTorch distributed primitives and are not described as “custom” in the documentation or API references. The available device communicators are listed, but only all-reduce is marked as custom in both the API and codebase references. No other custom collective (e.g., custom broadcast, custom all-gather) is mentioned or exposed in the official documentation or API index[^1][^2][^3][^4].
Would you like more detail or code examples?
Sources: