I’m using vllm’s openai server and enabled the mode via post /start_profile (having configured PROFILE_DIR beforehand). However, in the profile_dir, there are first torch.op_mark and torch.op_range files, followed by some slice.01234 files and a done file. I wonder if this is normal, and how to combine these files together.
Yes, this is normal. When profiling is enabled in vLLM’s OpenAI server (with VLLM_TORCH_PROFILER_DIR set and /start_profile called), the profiler writes intermediate files like torch.op_mark, torch.op_range, slice.*, and a done file as part of its trace output. These files are generated by the PyTorch profiler and are intended to be read by visualization tools such as Perfetto or TensorBoard, not manually combined by the user. You do not need to merge them yourself—just point Perfetto (https://ui.perfetto.dev/) or TensorBoard at the profile directory to view the results. For more, see Profiling vLLM and example usage.
Would you like more detail on how to visualize or interpret these files?
Sources:
Hint: Mention @RunLLM in the post for followups.