How use profile file to optimize the vllm inference as a model?

General
1

Hi,

I have a openguade-text model to inference use vllm.But it’s slow,I can’t to use it for work.So get the profile file

Can you give me some advices with the profile file?

2

To profile vLLM inference and generate a profile file, launch your vLLM server with the PyTorch profiler enabled using the --profiler-config flag (v0.13.0+). Then, use the /start_profile and /stop_profile HTTP endpoints to control profiling. The traces will be saved to the directory you specify and can be analyzed with tools like Perfetto. For example:

vllm serve <your-model> --profiler-config '{"profiler": "torch", "torch_profiler_dir": "./vllm_profile"}'
# Start profiling
curl -X POST http://localhost:8000/start_profile
# Run your inference requests
# Stop profiling
curl -X POST http://localhost:8000/stop_profile

You can then visualize the trace files at https://ui.perfetto.dev/. For more details on interpreting the results and optimizing performance, see the official profiling guide. Would you like step-by-step guidance on analyzing the profile file or tips for common bottlenecks?

Sources:

Hint: Mention @Herald in the post for followups.