# LoRA Adapter enabling with vLLM is not working

**URL:** <https://discuss.vllm.ai/t/lora-adapter-enabling-with-vllm-is-not-working/468>\
**Category:** LoRA\
**Created:** [April 15, 2025, 3:58pm UTC](https://discuss.vllm.ai/t/lora-adapter-enabling-with-vllm-is-not-working/468 "2025-04-15T15:58:19Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![uppadas](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/uppadas/32/221_2.png) [@uppadas](https://discuss.vllm.ai/u/uppadas)\
**Post date:** [April 15, 2025, 3:58pm UTC](https://discuss.vllm.ai/t/lora-adapter-enabling-with-vllm-is-not-working/468/1 "2025-04-15T15:58:19Z")

</div>

I’m trying to enable lora adapter dynamically with vLLM on AMD MI300x GPU’s.

Below are the steps that I followed to enable lora dynamically:

` **VLLM_ALLOW_RUNTIME_LORA_UPDATING=True python3 -m vllm.entrypoints.openai.api_server --model meta-llama/Llama-3.2-3B-Instruct --served-model-name Llama-3.2-3B-Instruct --enable-lora --max-lora-rank 64** `

Below are the logs:  
`INFO 04-14 16:15:31 [**init**.py:239] Automatically detected platform rocm.  
WARNING 04-14 16:15:32 [api\_server.py:759] LoRA dynamic loading & unloading is enabled in the API server. This should ONLY be used for local development!  
INFO 04-14 16:15:32 [api\_server.py:981] vLLM API server version 0.8.3.dev19+g3eb08ed9  
INFO 04-14 16:15:32 [api\_server.py:982] args: Namespace(host=None, port=8000, uvicorn\_log\_level=‘info’, disable\_uvicorn\_access\_log=False, allow\_credentials=False, allowed\_origins=[‘_‘], allowed\_methods=[’_’], allowed\_headers=[‘_’], api\_key=None, lora\_modules=None, prompt\_adapters=None, chat\_template=None, chat\_template\_content\_format=‘auto’, response\_role=‘assistant’, ssl\_keyfile=None, ssl\_certfile=None, ssl\_ca\_certs=None, enable\_ssl\_refresh=False, ssl\_cert\_reqs=0, root\_path=None, middleware=, return\_tokens\_as\_token\_ids=False, disable\_frontend\_multiprocessing=False, enable\_request\_id\_headers=False, enable\_auto\_tool\_choice=False, tool\_call\_parser=None, tool\_parser\_plugin=‘’, model=‘meta-llama/Llama-3.2-3B-Instruct’, task=‘auto’, tokenizer=None, hf\_config\_path=None, skip\_tokenizer\_init=False, revision=None, code\_revision=None, tokenizer\_revision=None, tokenizer\_mode=‘auto’, trust\_remote\_code=False, allowed\_local\_media\_path=None, download\_dir=None, load\_format=‘auto’, config\_format=\<ConfigFormat.AUTO: ‘auto’\>, dtype=‘auto’, kv\_cache\_dtype=‘auto’, max\_model\_len=None, guided\_decoding\_backend=‘xgrammar’, logits\_processor\_pattern=None, model\_impl=‘auto’, distributed\_executor\_backend=None, pipeline\_parallel\_size=1, tensor\_parallel\_size=1, enable\_expert\_parallel=False, max\_parallel\_loading\_workers=None, ray\_workers\_use\_nsight=False, block\_size=None, enable\_prefix\_caching=None, disable\_sliding\_window=False, use\_v2\_block\_manager=True, num\_lookahead\_slots=0, seed=None, swap\_space=4, cpu\_offload\_gb=0, gpu\_memory\_utilization=0.9, num\_gpu\_blocks\_override=None, max\_num\_batched\_tokens=None, max\_num\_partial\_prefills=1, max\_long\_partial\_prefills=1, long\_prefill\_token\_threshold=0, max\_num\_seqs=None, max\_logprobs=20, disable\_log\_stats=False, quantization=None, rope\_scaling=None, rope\_theta=None, hf\_overrides=None, enforce\_eager=False, max\_seq\_len\_to\_capture=8192, disable\_custom\_all\_reduce=False, tokenizer\_pool\_size=0, tokenizer\_pool\_type=‘ray’, tokenizer\_pool\_extra\_config=None, limit\_mm\_per\_prompt=None, mm\_processor\_kwargs=None, disable\_mm\_preprocessor\_cache=False, enable\_lora=True, enable\_lora\_bias=False, max\_loras=1, max\_lora\_rank=64, lora\_extra\_vocab\_size=256, lora\_dtype=‘auto’, long\_lora\_scaling\_factors=None, max\_cpu\_loras=None, fully\_sharded\_loras=False, enable\_prompt\_adapter=False, max\_prompt\_adapters=1, max\_prompt\_adapter\_token=0, device=‘auto’, num\_scheduler\_steps=1, use\_tqdm\_on\_load=True, multi\_step\_stream\_outputs=True, scheduler\_delay\_factor=0.0, enable\_chunked\_prefill=None, speculative\_config=None, speculative\_model=None, speculative\_model\_quantization=None, num\_speculative\_tokens=None, speculative\_disable\_mqa\_scorer=False, speculative\_draft\_tensor\_parallel\_size=None, speculative\_max\_model\_len=None, speculative\_disable\_by\_batch\_size=None, ngram\_prompt\_lookup\_max=None, ngram\_prompt\_lookup\_min=None, spec\_decoding\_acceptance\_method=‘rejection\_sampler’, typical\_acceptance\_sampler\_posterior\_threshold=None, typical\_acceptance\_sampler\_posterior\_alpha=None, disable\_logprobs\_during\_spec\_decoding=None, model\_loader\_extra\_config=None, ignore\_patterns=, preemption\_mode=None, served\_model\_name=[‘Llama-3.2-3B-Instruct’], qlora\_adapter\_name\_or\_path=None, show\_hidden\_metrics\_for\_version=None, otlp\_traces\_endpoint=None, collect\_detailed\_traces=None, disable\_async\_output\_proc=False, scheduling\_policy=‘fcfs’, scheduler\_cls=‘vllm.core.scheduler.Scheduler’, override\_neuron\_config=None, override\_pooler\_config=None, compilation\_config=None, kv\_transfer\_config=None, worker\_cls=‘auto’, worker\_extension\_cls=‘’, generation\_config=‘auto’, override\_generation\_config=None, enable\_sleep\_mode=False, calculate\_kv\_scales=False, additional\_config=None, enable\_reasoning=False, reasoning\_parser=None, disable\_cascade\_attn=False, disable\_log\_requests=False, max\_log\_len=None, disable\_fastapi\_docs=False, enable\_prompt\_tokens\_details=False, enable\_server\_load\_tracking=False)  
INFO 04-14 16:15:44 [config.py:585] This model supports multiple tasks: {‘embed’, ‘generate’, ‘classify’, ‘score’, ‘reward’}. Defaulting to ‘generate’.  
INFO 04-14 16:15:44 [arg\_utils.py:1868] LORA is experimental on VLLM\_USE\_V1=1. Falling back to V0 Engine.  
WARNING 04-14 16:15:44 [arg\_utils.py:1744] The model has a long context length (131072). This may causeOOM during the initial memory profiling phase, or result in low performance due to small KV cache size. Consider setting --max-model-len to a smaller value.  
INFO 04-14 16:15:44 [config.py:1552] Disabled the custom all-reduce kernel because it is not supported on AMD GPUs.  
INFO 04-14 16:15:44 [api\_server.py:241] Started engine process with PID 161  
INFO 04-14 16:15:47 [**init**.py:239] Automatically detected platform rocm.  
WARNING 04-14 16:15:48 [api\_server.py:759] LoRA dynamic loading & unloading is enabled in the API server. This should ONLY be used for local development!  
INFO 04-14 16:15:48 [llm\_engine.py:241] Initializing a V0 LLM engine (v0.8.3.dev19+g3eb08ed9) with config: model=‘meta-llama/Llama-3.2-3B-Instruct’, speculative\_config=None, tokenizer=‘meta-llama/Llama-3.2-3B-Instruct’, skip\_tokenizer\_init=False, tokenizer\_mode=auto, revision=None, override\_neuron\_config=None, tokenizer\_revision=None, trust\_remote\_code=False, dtype=torch.bfloat16, max\_seq\_len=131072, download\_dir=None, load\_format=LoadFormat.AUTO, tensor\_parallel\_size=1, pipeline\_parallel\_size=1, disable\_custom\_all\_reduce=True, quantization=None, enforce\_eager=False, kv\_cache\_dtype=auto, device\_config=cuda, decoding\_config=DecodingConfig(guided\_decoding\_backend=‘xgrammar’, reasoning\_backend=None), observability\_config=ObservabilityConfig(show\_hidden\_metrics=False, otlp\_traces\_endpoint=None, collect\_model\_forward\_time=False, collect\_model\_execute\_time=False), seed=None, served\_model\_name=Llama-3.2-3B-Instruct, num\_scheduler\_steps=1, multi\_step\_stream\_outputs=True, enable\_prefix\_caching=None, chunked\_prefill\_enabled=False, use\_async\_output\_proc=True, disable\_mm\_preprocessor\_cache=False, mm\_processor\_kwargs=None, pooler\_config=None, compilation\_config={“splitting\_ops”:,“compile\_sizes”:,“cudagraph\_capture\_sizes”:[256,248,240,232,224,216,208,200,192,184,176,168,160,152,144,136,128,120,112,104,96,88,80,72,64,56,48,40,32,24,16,8,4,2,1],“max\_capture\_size”:256}, use\_cached\_outputs=True,  
INFO 04-14 16:15:54 [rocm.py:131] None is not supported in AMD GPUs.  
INFO 04-14 16:15:54 [rocm.py:132] Using ROCmFlashAttention backend.  
INFO 04-14 16:15:54 [parallel\_state.py:954] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, TP rank 0  
INFO 04-14 16:15:54 [model\_runner.py:1110] Starting to load model meta-llama/Llama-3.2-3B-Instruct…  
INFO 04-14 16:15:55 [weight\_utils.py:265] Using model weights format [‘_.safetensors’]  
Loading safetensors checkpoint shards: 0% Completed | 0/2 [00:00\<?, ?it/s]  
Loading safetensors checkpoint shards: 50% Completed | 1/2 [00:00\<00:00, 1.51it/s]  
Loading safetensors checkpoint shards: 100% Completed | 2/2 [00:02\<00:00, 1.55s/it]  
Loading safetensors checkpoint shards: 100% Completed | 2/2 [00:02\<00:00, 1.42s/it]

INFO 04-14 16:15:58 [loader.py:447] Loading weights took 3.02 seconds  
INFO 04-14 16:15:58 [punica\_selector.py:18] Using PunicaWrapperGPU.  
INFO 04-14 16:15:58 [model\_runner.py:1146] Model loading took 7.2754 GB and 3.961958 seconds  
ERROR 04-14 16:15:59 [engine.py:448] ‘Keyword argument maxnreg was specified but unrecognised’  
ERROR 04-14 16:15:59 [engine.py:448] Traceback (most recent call last):  
ERROR 04-14 16:15:59 [engine.py:448] File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/engine/multiprocessing/engine.py”, line 436, in run\_mp\_engine  
ERROR 04-14 16:15:59 [engine.py:448] engine = MQLLMEngine.from\_vllm\_config(  
ERROR 04-14 16:15:59 [engine.py:448] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
ERROR 04-14 16:15:59 [engine.py:448] File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/engine/multiprocessing/engine.py”, line 128, in from\_vllm\_config  
ERROR 04-14 16:15:59 [engine.py:448] return cls(  
ERROR 04-14 16:15:59 [engine.py:448] ^^^^  
ERROR 04-14 16:15:59 [engine.py:448] File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/engine/multiprocessing/engine.py”, line 82, in **init**  
ERROR 04-14 16:15:59 [engine.py:448] self.engine = LLMEngine(\*args, \*\*kwargs)  
ERROR 04-14 16:15:59 [engine.py:448] ^^^^^^^^^^^^^^^^^^^^^^^^^^  
ERROR 04-14 16:15:59 [engine.py:448] File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/engine/llm\_engine.py”, line 283, in **init**  
ERROR 04-14 16:15:59 [engine.py:448] self.\_initialize\_kv\_caches()  
ERROR 04-14 16:15:59 [engine.py:448] File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/engine/llm\_engine.py”, line 432, in \_initialize\_kv\_caches  
ERROR 04-14 16:15:59 [engine.py:448] self.model\_executor.determine\_num\_available\_blocks())  
ERROR 04-14 16:15:59 [engine.py:448] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
ERROR 04-14 16:15:59 [engine.py:448] File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/executor/executor\_base.py”, line 102, in determine\_num\_available\_blocks  
ERROR 04-14 16:15:59 [engine.py:448] results = self.collective\_rpc(“determine\_num\_available\_blocks”)  
ERROR 04-14 16:15:59 [engine.py:448] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
ERROR 04-14 16:15:59 [engine.py:448] File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/executor/uniproc\_executor.py”, line 56, in collective\_rpc  
ERROR 04-14 16:15:59 [engine.py:448] answer = run\_method(self.driver\_worker, method, args, kwargs)  
ERROR 04-14 16:15:59 [engine.py:448] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
ERROR 04-14 16:15:59 [engine.py:448] File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/utils.py”, line 2255, in run\_method  
ERROR 04-14 16:15:59 [engine.py:448] return func(\*args, \*\*kwargs)  
ERROR 04-14 16:15:59 [engine.py:448] ^^^^^^^^^^^^^^^^^^^^^  
ERROR 04-14 16:15:59 [engine.py:448] File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/torch/utils/\_contextlib.py”, line 116, in decorate\_context  
ERROR 04-14 16:15:59 [engine.py:448] return func(\*args, \*\*kwargs)  
ERROR 04-14 16:15:59 [engine.py:448] ^^^^^^^^^^^^^^^^^^^^^  
ERROR 04-14 16:15:59 [engine.py:448] File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/worker/worker.py”, line 229, in determine\_num\_available\_blocks  
ERROR 04-14 16:15:59 [engine.py:448] self.model\_runner.profile\_run()  
ERROR 04-14 16:15:59 [engine.py:448] File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/torch/utils/\_contextlib.py”, line 116, in decorate\_context  
ERROR 04-14 16:15:59 [engine.py:448] return func(\*args, \*\*kwargs)  
ERROR 04-14 16:15:59 [engine.py:448] ^^^^^^^^^^^^^^^^^^^^^  
ERROR 04-14 16:15:59 [engine.py:448] File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/worker/model\_runner.py”, line 1243, in profile\_run  
ERROR 04-14 16:15:59 [engine.py:448] self.\_dummy\_run(max\_num\_batched\_tokens, max\_num\_seqs)  
ERROR 04-14 16:15:59 [engine.py:448] File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/worker/model\_runner.py”, line 1354, in \_dummy\_run  
ERROR 04-14 16:15:59 [engine.py:448] self.execute\_model(model\_input, kv\_caches, intermediate\_tensors)  
ERROR 04-14 16:15:59 [engine.py:448] File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/torch/utils/\_contextlib.py”, line 116, in decorate\_context  
ERROR 04-14 16:15:59 [engine.py:448] return func(\*args, \*\*kwargs)  
ERROR 04-14 16:15:59 [engine.py:448] ^^^^^^^^^^^^^^^^^^^^^  
ERROR 04-14 16:15:59 [engine.py:448] File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/worker/model\_runner.py”, line 1742, in execute\_model  
ERROR 04-14 16:15:59 [engine.py:448] hidden\_or\_intermediate\_states = model\_executable(  
ERROR 04-14 16:15:59 [engine.py:448] ^^^^^^^^^^^^^^^^^  
ERROR 04-14 16:15:59 [engine.py:448] File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/torch/nn/modules/module.py”, line 1751, in \_wrapped\_call\_impl  
ERROR 04-14 16:15:59 [engine.py:448] return self.\_call\_impl(\*args, \*\*kwargs)  
ERROR 04-14 16:15:59 [engine.py:448] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
ERROR 04-14 16:15:59 [engine.py:448] File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/torch/nn/modules/module.py”, line 1762, in \_call\_impl  
ERROR 04-14 16:15:59 [engine.py:448] return forward\_call(\*args, \*\*kwargs)  
ERROR 04-14 16:15:59 [engine.py:448] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
ERROR 04-14 16:15:59 [engine.py:448] File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/model\_executor/models/llama.py”, line 529, in forward  
ERROR 04-14 16:15:59 [engine.py:448] model\_output = self.model(input\_ids, positions, intermediate\_tensors,  
ERROR 04-14 16:15:59 [engine.py:448] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
ERROR 04-14 16:15:59 [engine.py:448] File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/compilation/decorators.py”, line 172, in **call**  
ERROR 04-14 16:15:59 [engine.py:448] return self.forward(\*args, \*\*kwargs)  
ERROR 04-14 16:15:59 [engine.py:448] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
ERROR 04-14 16:15:59 [engine.py:448] File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/model\_executor/models/llama.py”, line 350, in forward  
ERROR 04-14 16:15:59 [engine.py:448] hidden\_states = self.get\_input\_embeddings(input\_ids)  
ERROR 04-14 16:15:59 [engine.py:448] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
ERROR 04-14 16:15:59 [engine.py:448] File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/model\_executor/models/llama.py”, line 337, in get\_input\_embeddings  
ERROR 04-14 16:15:59 [engine.py:448] return self.embed\_tokens(input\_ids)  
ERROR 04-14 16:15:59 [engine.py:448] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
ERROR 04-14 16:15:59 [engine.py:448] File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/torch/nn/modules/module.py”, line 1751, in \_wrapped\_call\_impl  
ERROR 04-14 16:15:59 [engine.py:448] return self.\_call\_impl(\*args, \*\*kwargs)  
ERROR 04-14 16:15:59 [engine.py:448] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
ERROR 04-14 16:15:59 [engine.py:448] File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/torch/nn/modules/module.py”, line 1762, in \_call\_impl  
ERROR 04-14 16:15:59 [engine.py:448] return forward\_call(\*args, \*\*kwargs)  
ERROR 04-14 16:15:59 [engine.py:448] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
ERROR 04-14 16:15:59 [engine.py:448] File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/lora/layers.py”, line 264, in forward  
ERROR 04-14 16:15:59 [engine.py:448] self.punica\_wrapper.add\_lora\_embedding(full\_output,  
ERROR 04-14 16:15:59 [engine.py:448] File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/lora/punica\_wrapper/punica\_gpu.py”, line 176, in add\_lora\_embedding  
ERROR 04-14 16:15:59 [engine.py:448] lora\_expand(  
ERROR 04-14 16:15:59 [engine.py:448] File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/torch/\_ops.py”, line 1158, in **call**  
ERROR 04-14 16:15:59 [engine.py:448] return self.\_op(\*args, \*\*(kwargs or {}))  
ERROR 04-14 16:15:59 [engine.py:448] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
ERROR 04-14 16:15:59 [engine.py:448] File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/torch/utils/\_contextlib.py”, line 116, in decorate\_context  
ERROR 04-14 16:15:59 [engine.py:448] return func(\*args, \*\*kwargs)  
ERROR 04-14 16:15:59 [engine.py:448] ^^^^^^^^^^^^^^^^^^^^^  
ERROR 04-14 16:15:59 [engine.py:448] File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/lora/ops/triton\_ops/lora\_expand.py”, line 219, in \_lora\_expand  
ERROR 04-14 16:15:59 [engine.py:448] \_lora\_expand\_kernel[grid](  
ERROR 04-14 16:15:59 [engine.py:448] File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/triton/runtime/jit.py”, line 368, in  
ERROR 04-14 16:15:59 [engine.py:448] return lambda \*args, \*\*kwargs: self.run(grid=grid, warmup=False, \*args, \*\*kwargs)  
ERROR 04-14 16:15:59 [engine.py:448] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
ERROR 04-14 16:15:59 [engine.py:448] File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/triton/runtime/jit.py”, line 596, in run  
ERROR 04-14 16:15:59 [engine.py:448] raise KeyError(“Keyword argument %s was specified but unrecognised” % k)  
ERROR 04-14 16:15:59 [engine.py:448] KeyError: ‘Keyword argument maxnreg was specified but unrecognised’  
Process SpawnProcess-1:  
Traceback (most recent call last):  
File “/opt/conda/envs/py\_3.12/lib/python3.12/multiprocessing/process.py”, line 314, in \_bootstrap  
self.run()  
File “/opt/conda/envs/py\_3.12/lib/python3.12/multiprocessing/process.py”, line 108, in run  
self.\_target(\*self.\_args, \*\*self.\_kwargs)  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/engine/multiprocessing/engine.py”, line 450, in run\_mp\_engine  
raise e  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/engine/multiprocessing/engine.py”, line 436, in run\_mp\_engine  
engine = MQLLMEngine.from\_vllm\_config(  
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/engine/multiprocessing/engine.py”, line 128, in from\_vllm\_config  
return cls(  
^^^^  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/engine/multiprocessing/engine.py”, line 82, in **init**  
self.engine = LLMEngine(\*args, \*\*kwargs)  
^^^^^^^^^^^^^^^^^^^^^^^^^^  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/engine/llm\_engine.py”, line 283, in **init**  
self.\_initialize\_kv\_caches()  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/engine/llm\_engine.py”, line 432, in \_initialize\_kv\_caches  
self.model\_executor.determine\_num\_available\_blocks())  
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/executor/executor\_base.py”, line 102, in determine\_num\_available\_blocks  
results = self.collective\_rpc(“determine\_num\_available\_blocks”)  
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/executor/uniproc\_executor.py”, line 56, in collective\_rpc  
answer = run\_method(self.driver\_worker, method, args, kwargs)  
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/utils.py”, line 2255, in run\_method  
return func(\*args, \*\*kwargs)  
^^^^^^^^^^^^^^^^^^^^^  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/torch/utils/\_contextlib.py”, line 116, in decorate\_context  
return func(\*args, \*\*kwargs)  
^^^^^^^^^^^^^^^^^^^^^  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/worker/worker.py”, line 229, in determine\_num\_available\_blocks  
self.model\_runner.profile\_run()  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/torch/utils/\_contextlib.py”, line 116, in decorate\_context  
return func(\*args, \*\*kwargs)  
^^^^^^^^^^^^^^^^^^^^^  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/worker/model\_runner.py”, line 1243, in profile\_run  
self.\_dummy\_run(max\_num\_batched\_tokens, max\_num\_seqs)  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/worker/model\_runner.py”, line 1354, in \_dummy\_run  
self.execute\_model(model\_input, kv\_caches, intermediate\_tensors)  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/torch/utils/\_contextlib.py”, line 116, in decorate\_context  
return func(\*args, \*\*kwargs)  
^^^^^^^^^^^^^^^^^^^^^  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/worker/model\_runner.py”, line 1742, in execute\_model  
hidden\_or\_intermediate\_states = model\_executable(  
^^^^^^^^^^^^^^^^^  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/torch/nn/modules/module.py”, line 1751, in \_wrapped\_call\_impl  
return self.\_call\_impl(\*args, \*\*kwargs)  
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/torch/nn/modules/module.py”, line 1762, in \_call\_impl  
return forward\_call(\*args, \*\*kwargs)  
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/model\_executor/models/llama.py”, line 529, in forward  
model\_output = self.model(input\_ids, positions, intermediate\_tensors,  
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/compilation/decorators.py”, line 172, in **call**  
return self.forward(\*args, \*\*kwargs)  
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/model\_executor/models/llama.py”, line 350, in forward  
hidden\_states = self.get\_input\_embeddings(input\_ids)  
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/model\_executor/models/llama.py”, line 337, in get\_input\_embeddings  
return self.embed\_tokens(input\_ids)  
^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/torch/nn/modules/module.py”, line 1751, in \_wrapped\_call\_impl  
return self.\_call\_impl(\*args, \*\*kwargs)  
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/torch/nn/modules/module.py”, line 1762, in \_call\_impl  
return forward\_call(\*args, \*\*kwargs)  
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/lora/layers.py”, line 264, in forward  
self.punica\_wrapper.add\_lora\_embedding(full\_output,  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/lora/punica\_wrapper/punica\_gpu.py”, line 176, in add\_lora\_embedding  
lora\_expand(  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/torch/\_ops.py”, line 1158, in **call**  
return self.\_op(\*args, \*\*(kwargs or {}))  
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/torch/utils/\_contextlib.py”, line 116, in decorate\_context  
return func(\*args, \*\*kwargs)  
^^^^^^^^^^^^^^^^^^^^^  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/lora/ops/triton\_ops/lora\_expand.py”, line 219, in \_lora\_expand  
\_lora\_expand\_kernel[grid](  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/triton/runtime/jit.py”, line 368, in  
return lambda \*args, \*\*kwargs: self.run(grid=grid, warmup=False, \*args, \*\*kwargs)  
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/triton/runtime/jit.py”, line 596, in run  
raise KeyError(“Keyword argument %s was specified but unrecognised” % k)  
KeyError: ‘Keyword argument maxnreg was specified but unrecognised’  
[rank0]:[W414 16:15:59.861372715 ProcessGroupNCCL.cpp:1477] Warning: WARNING: destroy\_process\_group() was not called before program exit, which can leak resources. For more info, please see [Distributed communication package - torch.distributed — PyTorch 2.6 documentation](https://pytorch.org/docs/stable/distributed.html#shutdown) (function operator())  
Traceback (most recent call last):  
File “”, line 198, in \_run\_module\_as\_main  
File “”, line 88, in \_run\_code  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/entrypoints/openai/api\_server.py”, line 1066, in  
uvloop.run(run\_server(args))  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/uvloop/ **init**.py”, line 109, in run  
return \_\_asyncio.run(  
^^^^^^^^^^^^^^  
File “/opt/conda/envs/py\_3.12/lib/python3.12/asyncio/runners.py”, line 195, in run  
return runner.run(main)  
^^^^^^^^^^^^^^^^  
File “/opt/conda/envs/py\_3.12/lib/python3.12/asyncio/runners.py”, line 118, in run  
return self.\_loop.run\_until\_complete(task)  
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
File “uvloop/loop.pyx”, line 1518, in uvloop.loop.Loop.run\_until\_complete  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/uvloop/ **init**.py”, line 61, in wrapper  
return await main  
^^^^^^^^^^  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/entrypoints/openai/api\_server.py”, line 1016, in run\_server  
async with build\_async\_engine\_client(args) as engine\_client:  
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
File “/opt/conda/envs/py\_3.12/lib/python3.12/contextlib.py”, line 210, in **aenter**  
return await anext(self.gen)  
^^^^^^^^^^^^^^^^^^^^^  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/entrypoints/openai/api\_server.py”, line 141, in build\_async\_engine\_client  
async with build\_async\_engine\_client\_from\_engine\_args(  
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
File “/opt/conda/envs/py\_3.12/lib/python3.12/contextlib.py”, line 210, in **aenter**  
return await anext(self.gen)  
^^^^^^^^^^^^^^^^^^^^^  
File “/opt/conda/envs/py\_3.12/lib/python3.12/site-packages/vllm/entrypoints/openai/api\_server.py”, line 264, in build\_async\_engine\_client\_from\_engine\_args  
raise RuntimeError(  
`

**vLLM version: 0.8.3.dev19+g3eb08ed9.rocm634**

Please let me know If I’m missing anything here?

---

<div class="post-metadata">

**Author:** ![jeejeelee](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/jeejeelee/32/23_2.png) [@jeejeelee](https://discuss.vllm.ai/u/jeejeelee)\
**Post date:** [April 16, 2025, 1:32am UTC](https://discuss.vllm.ai/t/lora-adapter-enabling-with-vllm-is-not-working/468/2 "2025-04-16T01:32:28Z")

</div>

Which version of triton are you using?

---

<div class="post-metadata">

**Author:** ![uppadas](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/uppadas/32/221_2.png) [@uppadas](https://discuss.vllm.ai/u/uppadas)\
**Post date:** [April 16, 2025, 5:06pm UTC](https://discuss.vllm.ai/t/lora-adapter-enabling-with-vllm-is-not-working/468/3 "2025-04-16T17:06:11Z")

</div>

I’m using below version:  
`triton - 3.3.0+gitb962e444`  
`vllm - 0.8.3.dev19+g3eb08ed9.rocm634`

---

<div class="post-metadata">

**Author:** ![uppadas](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/uppadas/32/221_2.png) [@uppadas](https://discuss.vllm.ai/u/uppadas)\
**Post date:** [April 21, 2025, 8:34pm UTC](https://discuss.vllm.ai/t/lora-adapter-enabling-with-vllm-is-not-working/468/4 "2025-04-21T20:34:50Z")

</div>

@jeejeelee, can you please provide update on this issue?

---

<div class="post-metadata">

**Author:** ![RunLLM](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.vllm.ai/runllm/32/233_2.png) [@RunLLM](https://discuss.vllm.ai/u/RunLLM)\
**Post date:** [April 21, 2025, 8:35pm UTC](https://discuss.vllm.ai/t/lora-adapter-enabling-with-vllm-is-not-working/468/5 "2025-04-21T20:35:17Z")

</div>

Could you please specify which issue you are referring to? There are multiple issues and discussions involving @jeejeelee in the provided data sources. If you can provide more context or details, I can assist you better.

Sources:

- [https://github.com/vllm-project/vllm/issues/12829](https://github.com/vllm-project/vllm/issues/12829)
- [https://vllm-dev.slack.com/archives/C07QP347J4D/p1744035479.551079](https://vllm-dev.slack.com/archives/C07QP347J4D/p1744035479.551079)