# When using large batches, the Ray service crashes.ray.exceptions.RayChannelTimeoutError: System error: Timed out waiting for object available to read

**URL:** <https://discuss.vllm.ai/t/when-using-large-batches-the-ray-service-crashes-ray-exceptions-raychanneltimeouterror-system-error-timed-out-waiting-for-object-available-to-read/1218>\
**Category:** Ray\
**Created:** [July 29, 2025, 9:13am UTC](https://discuss.vllm.ai/t/when-using-large-batches-the-ray-service-crashes-ray-exceptions-raychanneltimeouterror-system-error-timed-out-waiting-for-object-available-to-read/1218 "2025-07-29T09:13:59Z")\
**Posts on this page:** 1\
**Showing post:** 10

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [July 29, 2025, 12:36pm UTC](https://discuss.vllm.ai/t/when-using-large-batches-the-ray-service-crashes-ray-exceptions-raychanneltimeouterror-system-error-timed-out-waiting-for-object-available-to-read/1218/10 "2025-07-29T12:36:16Z")

</div>

两者（CLI serve部署与离线pipeline部署）在精度上理论上应无明显差别，只要模型权重、推理参数完全一致。但实际测试中，API服务和离线推理结果可能会有细微差异，主要原因是数值精度、批处理方式、并发调度等带来的浮点误差和非确定性，尤其是在float16/bfloat16等低精度下，这种差异会被放大，导致生成结果甚至logprobs有较大不同。[官方FAQ](https://docs.vllm.ai/en/stable/getting_started/faq.html)明确说明vLLM不保证完全稳定的输出和logprobs。

此外，vLLM和transformers等实现间也会有类似现象，根本原因是底层算子、批处理、硬件等引入的微小数值不一致会逐步放大，最终影响生成内容。[相关讨论](https://discuss.vllm.ai/t/difference-in-log-probabilities-between-vllm-and-hf-model-in-identical-environment/183)和[官方说明](https://docs.vllm.ai/en/stable/getting_started/faq.html)均有详细解释。

需要更详细的原理分析或复现建议吗？

Sources:

- [vLLM FAQ](https://docs.vllm.ai/en/stable/getting_started/faq.html)
- [Difference in log probabilities between vLLM and HF](https://discuss.vllm.ai/t/difference-in-log-probabilities-between-vllm-and-hf-model-in-identical-environment/183)

---

_[View the full topic](https://discuss.vllm.ai/t/when-using-large-batches-the-ray-service-crashes-ray-exceptions-raychanneltimeouterror-system-error-timed-out-waiting-for-object-available-to-read/1218)._