# RL Integration **URL:** https://discuss.vllm.ai/c/rl-integration/28.md [Latest](https://discuss.vllm.ai/latest.md) · [Categories](https://discuss.vllm.ai/categories.md) --- ## [About the RL Integration category](https://discuss.vllm.ai/t/about-the-rl-integration-category/248)
**Author:** [@youkaichao](https://discuss.vllm.ai/u/youkaichao)\ **Replies:** 0
--- ## [Can the AsyncEngine be compatible with the backend of external\_launcher?](https://discuss.vllm.ai/t/can-the-asyncengine-be-compatible-with-the-backend-of-external-launcher/851)
**Author:** [@hjh](https://discuss.vllm.ai/u/hjh)\ **Replies:** 3\ **Last updated:** [October 8, 2025, 7:18am UTC](https://discuss.vllm.ai/t/can-the-asyncengine-be-compatible-with-the-backend-of-external-launcher/851 "2025-10-08T07:18:21Z")
I attempted to use vLLM AsyncEngine + external\_launcher for inference acceleration under torchrun in the RL training pipeline, but I encountered compatibility issues. The LLMEngine had previously run without any problems… --- ## [Degraded RL training with v0.10.2](https://discuss.vllm.ai/t/degraded-rl-training-with-v0-10-2/1616)
**Author:** [@Jackmin801](https://discuss.vllm.ai/u/Jackmin801)\ **Replies:** 8\ **Last updated:** [September 16, 2025, 8:18am UTC](https://discuss.vllm.ai/t/degraded-rl-training-with-v0-10-2/1616 "2025-09-16T08:18:35Z")
Hey vLLM team. Congratulations on the new release! Was looking forward to it for quite some time (was using torch 2.8 for our RL trainer so needed 2 venvs, one running torch 2.7 until the new release). However, it seems… --- ## [Running vLLM multi-node data parallel with SLURM](https://discuss.vllm.ai/t/running-vllm-multi-node-data-parallel-with-slurm/1362)
**Author:** [@mackereldream](https://discuss.vllm.ai/u/mackereldream)\ **Replies:** 1\ **Last updated:** [August 12, 2025, 3:23pm UTC](https://discuss.vllm.ai/t/running-vllm-multi-node-data-parallel-with-slurm/1362 "2025-08-12T15:23:50Z")
Hello :slight\_smile: I understand the preferred method for multi-node inference is using a ray server. I’ve been trying to manage a training run with vLLM as an inference server on a SLURM cluster. I’ve aiming for some… --- ## [Intergate async-llm to openrlfh, when tensor\_parallel\_size \> 1 will cause ray-error](https://discuss.vllm.ai/t/intergate-async-llm-to-openrlfh-when-tensor-parallel-size-1-will-cause-ray-error/534)
**Author:** [@haotianxu](https://discuss.vllm.ai/u/haotianxu)\ **Replies:** 1\ **Last updated:** [April 27, 2025, 4:21am UTC](https://discuss.vllm.ai/t/intergate-async-llm-to-openrlfh-when-tensor-parallel-size-1-will-cause-ray-error/534 "2025-04-27T04:21:23Z")
Hi, I tried to implement async-llm into openrlhf(v0.6.4). It works for 7/32b when tensor-parallel-size=1. When tensor-parallel-size \> 1, it will cause ray error. Here is the async-vllm import socket, asyncio import ray… --- ## [Is is possible to initialize an AsyncLLMEngine inside the LLM object?](https://discuss.vllm.ai/t/is-is-possible-to-initialize-an-asyncllmengine-inside-the-llm-object/392)
**Author:** [@SparkJiao](https://discuss.vllm.ai/u/SparkJiao)\ **Replies:** 4\ **Last updated:** [April 12, 2025, 4:58pm UTC](https://discuss.vllm.ai/t/is-is-possible-to-initialize-an-asyncllmengine-inside-the-llm-object/392 "2025-04-12T16:58:31Z")
The background is that we are using vllm as the backend for rollout in veRL for RL training. During rollout stage, we need to invoke several function calls inside each trajectory. Currently most implementations simply us… --- ## [No HIP GPUs are available for VeRL](https://discuss.vllm.ai/t/no-hip-gpus-are-available-for-verl/242)
**Author:** [@XueruiSu](https://discuss.vllm.ai/u/XueruiSu)\ **Replies:** 4\ **Last updated:** [April 4, 2025, 12:11pm UTC](https://discuss.vllm.ai/t/no-hip-gpus-are-available-for-verl/242 "2025-04-04T12:11:57Z")
I tested the latest VeRL cloned from github with the following command on the AMD cluster: for data generation: python ~/verl/examples/data\_preprocess/gsm8k.py for LLM training: bash ~/verl/examples/grpo\_trainer/run\_dee… --- ## [Numerical Difference between vLLM logprobs and huggingface logprobs](https://discuss.vllm.ai/t/numerical-difference-between-vllm-logprobs-and-huggingface-logprobs/151)
**Author:** [@Jackmin801](https://discuss.vllm.ai/u/Jackmin801)\ **Replies:** 7\ **Last updated:** [April 4, 2025, 12:19pm UTC](https://discuss.vllm.ai/t/numerical-difference-between-vllm-logprobs-and-huggingface-logprobs/151 "2025-04-04T12:19:30Z")
Hey guys! Big fan of vLLM :rocket: I notice that the logprobs returned by vLLM can be quite far off from the ones calculated by transformers. Logits also very often just round to -inf when theyre perfectly within dynami… --- ## [RL Training with vLLM Rollout: How to Mitigate Load Imbalance from Variable Response Lengths](https://discuss.vllm.ai/t/rl-training-with-vllm-rollout-how-to-mitigate-load-imbalance-from-variable-response-lengths/117)
**Author:** [@uygnef](https://discuss.vllm.ai/u/uygnef)\ **Replies:** 4\ **Last updated:** [April 1, 2025, 9:23am UTC](https://discuss.vllm.ai/t/rl-training-with-vllm-rollout-how-to-mitigate-load-imbalance-from-variable-response-lengths/117 "2025-04-01T09:23:38Z")
We’re using vLLM for RL training with 32xA100 GPUs and a batch size of ~8000. Load Imbalance in SPMD Mode: Prompts are split evenly across workers, but response length variance causes severe load imbalance. Would swit…