Is this normal for vllm serve output waiting 1 hour b´before startup

Hardware Support NVIDIA GPU Support
1

i just started vllm serve

is this the normal output of vLLM is it stuck because cannot reach it on localhost:8000

no INFO Server Started or serving at??

is this ok output?? everything seems ok

$ vllm serve unsloth/Qwen3.8-27B-NVFP4
(APIServer pid=10951) INFO 09-08 08:23:37 [api_utils.py:333]
(APIServer pid=10951) INFO 09-08 08:23:37 [api_utils.py:333] █ █ █▄ ▄█
(APIServer pid=10951) INFO 09-08 08:23:37 [api_utils.py:333] ▄▄ ▄█ █ █ █ ▀▄▀ █ version 0.28.0
(APIServer pid=10951) INFO 09-08 08:23:37 [api_utils.py:333] █▄█▀ █ █ █ █ model unsloth/Qwen3.8-27B-NVFP4
(APIServer pid=10951) INFO 09-08 08:23:37 [api_utils.py:333] ▀▀ ▀▀▀▀▀ ▀▀▀▀▀ ▀ ▀
(APIServer pid=10951) INFO 09-08 08:23:37 [api_utils.py:333]
(APIServer pid=10951) INFO 09-08 08:23:37 [api_utils.py:272] non-default args: {‘model_tag’: ‘unsloth/Qwen3.8-27B-NVFP4’, ‘model’: ‘unsloth/Qwen3.8-27B-NVFP4’}
(APIServer pid=10951) INFO 09-08 08:23:39 [model.py:672] Resolved architecture: Qwen3_5ForConditionalGeneration
(APIServer pid=10951) INFO 09-08 08:23:39 [model.py:1965] Using max model len 262144
(APIServer pid=10951) INFO 09-08 08:23:42 [config.py:605] Mamba cache mode is set to ‘align’ for Qwen3_5ForConditionalGeneration by default when prefix caching is enabled
(APIServer pid=10951) INFO 09-08 08:23:42 [kernel.py:308] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=[‘native’], fused_add_rms_norm=[‘native’])
(APIServer pid=10951) [transformers] The use_fast parameter is deprecated and will be removed in a future version. Use backend="torchvision" instead of use_fast=True, or backend="pil" instead of use_fast=False.
(EngineCore pid=11092) INFO 09-08 08:23:56 [core.py:122] Initializing a V1 LLM engine (v0.28.0) with config: model=‘unsloth/Qwen3.8-27B-NVFP4’, speculative_config=None, tokenizer=‘unsloth/Qwen3.8-27B-NVFP4’, skip_tokenizer_init=False, tokenizer_mode=auto, revision=main, tokenizer_revision=main, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=262144, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, decode_context_parallel_size=1, dcp_comm_backend=ag_rs, disable_custom_all_reduce=False, quantization=compressed-tensors, quantization_config=None, enforce_eager=False, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend=‘auto’, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser=‘’, reasoning_parser_plugin=‘’, enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False, jit_monitor_mode=‘warn’, jit_monitor_verbose=False), seed=0, served_model_name=unsloth/Qwen3.8-27B-NVFP4, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={‘mode’: <CompilationMode.VLLM_COMPILE: 3>, ‘debug_dump_path’: None, ‘cache_dir’: ‘’, ‘compile_cache_save_format’: ‘binary’, ‘backend’: ‘inductor’, ‘custom_ops’: [‘none’], ‘ir_enable_torch_wrap’: True, ‘splitting_ops’: [‘vllm::unified_attention_with_output’, ‘vllm::unified_mla_attention_with_output’, ‘vllm::mamba_mixer2’, ‘vllm::mamba_mixer’, ‘vllm::short_conv’, ‘vllm::linear_attention’, ‘vllm::qwen_gdn_attention_core’, ‘vllm::qwen_gdn_attention_core_fused_norm_packed’, ‘vllm::gdn_attention_core_xpu’, ‘vllm::olmo_hybrid_gdn_full_forward’, ‘vllm::sparse_attn_indexer’, ‘vllm::rocm_aiter_sparse_attn_indexer’, ‘vllm::deepseek_v4_attention’, ‘vllm::hpc_rope_norm_forward’, ‘vllm::unified_kv_cache_update’, ‘vllm::unified_mla_kv_cache_update’], ‘compile_mm_encoder’: False, ‘cudagraph_mm_encoder’: False, ‘encoder_cudagraph_token_budgets’: , ‘encoder_cudagraph_max_vision_items_per_batch’: 0, ‘encoder_cudagraph_max_frames_per_batch’: None, ‘compile_sizes’: , ‘compile_ranges_endpoints’: [2048], ‘inductor_compile_config’: {‘enable_auto_functionalized_v2’: False, ‘combo_kernels’: True, ‘benchmark_combo_kernel’: True}, ‘inductor_passes’: {}, ‘cudagraph_mode’: <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, ‘cudagraph_num_of_warmups’: 1, ‘cudagraph_capture_sizes’: [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256, 272, 288, 304, 320, 336, 352, 368, 384, 400, 416, 432, 448, 464, 480, 496, 512], ‘cudagraph_copy_inputs’: False, ‘cudagraph_specialize_lora’: True, ‘use_inductor_graph_partition’: False, ‘pass_config’: {‘fuse_norm_quant’: False, ‘fuse_act_quant’: False, ‘fuse_attn_quant’: False, ‘enable_sp’: False, ‘fuse_gemm_comms’: False, ‘fuse_allreduce_rms’: False, ‘enable_qk_norm_rope_fusion’: False, ‘fuse_rope_kvcache_cat_mla’: False, ‘fuse_act_padding’: False, ‘fuse_qk_norm_rope_kvcache’: False}, ‘max_cudagraph_capture_size’: 512, ‘dynamic_shapes_config’: {‘type’: <DynamicShapesType.BACKED: ‘backed’>, ‘evaluate_guards’: False, ‘assume_32_bit_indexing’: False}, ‘local_cache_dir’: None, ‘fast_moe_cold_start’: False, ‘static_all_moe_layers’: }, kernel_config=KernelConfig(ir_op_priority=IrOpPriorityConfig(rms_norm=[‘native’], fused_add_rms_norm=[‘native’]), enable_flashinfer_autotune=True, enable_cutedsl_warmup=True, enable_jit_warmup=True, enable_bf16x3_router_gemm=False, moe_backend=‘auto’, linear_backend=‘auto’)
(EngineCore pid=11092) INFO 09-08 08:23:59 [parallel_state.py:1638] world_size=1 rank=0 local_rank=0 distributed_init_method=file:///tmp/vllm_dist_04dfb5e63cb34c4ba2337b64d943df33 backend=nccl
(EngineCore pid=11092) INFO 09-08 08:23:59 [parallel_state.py:1982] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A, EPLB rank N/A
(EngineCore pid=11092) INFO 09-08 08:24:01 [topk_topp_sampler.py:62] Using FlashInfer for top-p & top-k sampling.
(EngineCore pid=11092) [transformers] The use_fast parameter is deprecated and will be removed in a future version. Use backend="torchvision" instead of use_fast=True, or backend="pil" instead of use_fast=False.
(EngineCore pid=11092) INFO 09-08 08:24:10 [gpu_model_runner.py:5419] Starting to load model unsloth/Qwen3.8-27B-NVFP4…
(EngineCore pid=11092) INFO 09-08 08:24:10 [cuda.py:545] Using backend AttentionBackendEnum.FLASH_ATTN for vit attention
(EngineCore pid=11092) INFO 09-08 08:24:10 [mm_encoder_attention.py:375] Using AttentionBackendEnum.FLASH_ATTN for MMEncoderAttention.
(EngineCore pid=11092) INFO 09-08 08:24:10 [init.py:720] Selected CutlassFP8ScaledMMLinearKernel for CompressedTensorsW8A8Fp8
(EngineCore pid=11092) INFO 09-08 08:24:10 [deep_gemm.py:186] deep_gemm not found in site-packages, trying vendored vllm.third_party.deep_gemm
(EngineCore pid=11092) INFO 09-08 08:24:10 [deep_gemm.py:213] DeepGEMM PDL enabled on vllm.third_party.deep_gemm.
(EngineCore pid=11092) INFO 09-08 08:24:10 [deep_gemm.py:131] DeepGEMM E8M0 enabled on current platform.
(EngineCore pid=11092) INFO 09-08 08:24:10 [qwen_gdn_linear_attn.py:158] Using Triton/FLA GDN prefill kernel (requested=auto, head_k_dim=128).
(EngineCore pid=11092) INFO 09-08 08:24:10 [qwen_gdn_linear_attn.py:505] GDN decode kernel: cuda
(EngineCore pid=11092) INFO 09-08 08:24:10 [init.py:1100] Using FlashInferCutlassNvFp4LinearKernel for NVFP4 GEMM
(EngineCore pid=11092) INFO 09-08 08:24:10 [cuda.py:486] Using FLASHINFER attention backend out of potential backends: [‘FLASHINFER’, ‘TRITON_ATTN’].
(EngineCore pid=11092) INFO 09-08 08:24:12 [weight_utils.py:858] Filesystem type for checkpoints: EXT4. Checkpoint size: 21.81 GiB. Available RAM: 25.72 GiB.
(EngineCore pid=11092) INFO 09-08 08:24:12 [weight_utils.py:881] Auto-prefetch is disabled because the filesystem (EXT4) is not a recognized network FS (NFS/Lustre). If you want to force prefetching, start vLLM with --safetensors-load-strategy=prefetch.
Loading safetensors checkpoint shards: 0% Completed | 0/2 [00:00<?, ?it/s]
Loading safetensors checkpoint shards: 50% Completed | 1/2 [00:05<00:05, 5.46s/it]
Loading safetensors checkpoint shards: 100% Completed | 2/2 [00:05<00:00, 2.73s/it]
(EngineCore pid=11092)
(EngineCore pid=11092) INFO 09-08 08:24:17 [default_loader.py:430] Loading weights took 5.46 seconds
(EngineCore pid=11092) INFO 09-08 08:24:18 [gpu_model_runner.py:5515] Model loading took 21.34 GiB memory and 7.582766 seconds
(EngineCore pid=11092) INFO 09-08 08:24:18 [interface.py:911] Setting attention block size to 1568 tokens to ensure that attention page size is >= mamba page size.
(EngineCore pid=11092) INFO 09-08 08:24:18 [interface.py:935] Padding mamba page size by 0.13% to ensure that mamba page size and attention page size are exactly equal.
(EngineCore pid=11092) INFO 09-08 08:24:18 [gpu_model_runner.py:6590] Encoder cache will be initialized with a budget of 16384 tokens, and profiled with 1 image items of the maximum feature size.

2

Your output shows that vLLM is still in the process of loading and initializing the model; this is normal for large models and can take several minutes. You should wait until you see a log line similar to “Starting vLLM server on http://0.0.0.0:8000” or “Application startup complete” before the server is ready to accept requests. If you do not see such a message and cannot reach localhost:8000, the server is not yet ready or may have encountered an error after the last log line you posted. This is expected behavior for large models, and the absence of a “serving” or “server started” message means it is not yet ready for requests. See examples of expected startup logs and readiness in this issue and the docs.

Would you like more detail on how to troubleshoot if it never reaches the “server started” state?

Sources:

Hint: Mention @Herald in the post for followups.

3

@Herald how log does it take to start allready waiting 40 min

4

@Herald how log does it take to start allready waiting 40 min

5

Startup time for large models in vLLM can vary widely. For 70B parameter models, loading from local disk can take anywhere from a few minutes to over 40 minutes, depending on your storage speed (SSD vs HDD), filesystem (local vs network), and available CPU memory. Users have reported 3–40 minutes for 67B–70B models, with slow speeds often due to network/distributed filesystems or HDDs rather than vLLM itself. Using a fast local SSD and ensuring sufficient CPU RAM can significantly reduce load times. If you are waiting over 40 minutes, check your disk speed and storage setup first, as this is the most common bottleneck—not vLLM code or GPU performance. See details in this issue and this issue.

If you want more detail on how to diagnose or speed up startup, let me know.

Sources:

Hint: Mention @Herald in the post for followups.

6

@harald edibi@Vector:~$ source ./unsloth-nvfp4-env/bin/activate
(unsloth-nvfp4-env) edibi@Vector:~$ vllm serve unsloth/Qwen3.8-27B-NVFP4
–host 0.0.0.0 --port 1773
–speculative-config ‘{“method”: “mtp”, “num_speculative_tokens”: 2}’
(APIServer pid=1385) INFO 09-08 09:41:31 [api_utils.py:333]
(APIServer pid=1385) INFO 09-08 09:41:31 [api_utils.py:333] █ █ █▄ ▄█
(APIServer pid=1385) INFO 09-08 09:41:31 [api_utils.py:333] ▄▄ ▄█ █ █ █ ▀▄▀ █ version 0.28.0
(APIServer pid=1385) INFO 09-08 09:41:31 [api_utils.py:333] █▄█▀ █ █ █ █ model unsloth/Qwen3.8-27B-NVFP4
(APIServer pid=1385) INFO 09-08 09:41:31 [api_utils.py:333] ▀▀ ▀▀▀▀▀ ▀▀▀▀▀ ▀ ▀
(APIServer pid=1385) INFO 09-08 09:41:31 [api_utils.py:333]
(APIServer pid=1385) INFO 09-08 09:41:31 [api_utils.py:272] non-default args: {‘model_tag’: ‘unsloth/Qwen3.8-27B-NVFP4’, ‘host’: ‘0.0.0.0’, ‘port’: 1773, ‘model’: ‘unsloth/Qwen3.8-27B-NVFP4’, ‘speculative_config’: {‘method’: ‘mtp’, ‘num_speculative_tokens’: 2}}
(APIServer pid=1385) INFO 09-08 09:41:33 [model.py:672] Resolved architecture: Qwen3_5ForConditionalGeneration
(APIServer pid=1385) INFO 09-08 09:41:33 [model.py:1965] Using max model len 262144
(APIServer pid=1385) INFO 09-08 09:41:36 [model.py:672] Resolved architecture: Qwen3_5MTP
(APIServer pid=1385) INFO 09-08 09:41:36 [model.py:1965] Using max model len 262144
(APIServer pid=1385) INFO 09-08 09:41:36 [model.py:875] Disabled mm_prefix attention mode because multimodal inputs are configuration-disabled. Attention backends without mm_prefix support may now be selected.
(APIServer pid=1385) WARNING 09-08 09:41:36 [speculative.py:971] Enabling num_speculative_tokens > 1 will run multiple times of forward on same MTP layer,which may result in lower acceptance rate
(APIServer pid=1385) INFO 09-08 09:41:36 [config.py:605] Mamba cache mode is set to ‘align’ for Qwen3_5ForConditionalGeneration by default when prefix caching is enabled
(APIServer pid=1385) INFO 09-08 09:41:36 [kernel.py:308] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=[‘native’], fused_add_rms_norm=[‘native’])
(APIServer pid=1385) WARNING 09-08 09:41:36 [vllm.py:1862] max_num_scheduled_tokens is set to 2048 based on the speculative decoding settings. This may lead to suboptimal performance. Consider increasing max_num_batched_tokens to accommodate the additional draft token slots, or decrease num_speculative_tokens.
(APIServer pid=1385) [transformers] The use_fast parameter is deprecated and will be removed in a future version. Use backend="torchvision" instead of use_fast=True, or backend="pil" instead of use_fast=False.
(EngineCore pid=1502) INFO 09-08 09:41:51 [core.py:122] Initializing a V1 LLM engine (v0.28.0) with config: model=‘unsloth/Qwen3.8-27B-NVFP4’, speculative_config=SpeculativeConfig(method=‘mtp’, model=‘unsloth/Qwen3.8-27B-NVFP4’, num_spec_tokens=2), tokenizer=‘unsloth/Qwen3.8-27B-NVFP4’, skip_tokenizer_init=False, tokenizer_mode=auto, revision=main, tokenizer_revision=main, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=262144, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, decode_context_parallel_size=1, dcp_comm_backend=ag_rs, disable_custom_all_reduce=False, quantization=compressed-tensors, quantization_config=None, enforce_eager=False, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend=‘auto’, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser=‘’, reasoning_parser_plugin=‘’, enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False, jit_monitor_mode=‘warn’, jit_monitor_verbose=False), seed=0, served_model_name=unsloth/Qwen3.8-27B-NVFP4, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={‘mode’: <CompilationMode.VLLM_COMPILE: 3>, ‘debug_dump_path’: None, ‘cache_dir’: ‘’, ‘compile_cache_save_format’: ‘binary’, ‘backend’: ‘inductor’, ‘custom_ops’: [‘none’], ‘ir_enable_torch_wrap’: True, ‘splitting_ops’: [‘vllm::unified_attention_with_output’, ‘vllm::unified_mla_attention_with_output’, ‘vllm::mamba_mixer2’, ‘vllm::mamba_mixer’, ‘vllm::short_conv’, ‘vllm::linear_attention’, ‘vllm::qwen_gdn_attention_core’, ‘vllm::qwen_gdn_attention_core_fused_norm_packed’, ‘vllm::gdn_attention_core_xpu’, ‘vllm::olmo_hybrid_gdn_full_forward’, ‘vllm::sparse_attn_indexer’, ‘vllm::rocm_aiter_sparse_attn_indexer’, ‘vllm::deepseek_v4_attention’, ‘vllm::hpc_rope_norm_forward’, ‘vllm::unified_kv_cache_update’, ‘vllm::unified_mla_kv_cache_update’], ‘compile_mm_encoder’: False, ‘cudagraph_mm_encoder’: False, ‘encoder_cudagraph_token_budgets’: , ‘encoder_cudagraph_max_vision_items_per_batch’: 0, ‘encoder_cudagraph_max_frames_per_batch’: None, ‘compile_sizes’: , ‘compile_ranges_endpoints’: [2048], ‘inductor_compile_config’: {‘enable_auto_functionalized_v2’: False, ‘combo_kernels’: True, ‘benchmark_combo_kernel’: True}, ‘inductor_passes’: {}, ‘cudagraph_mode’: <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, ‘cudagraph_num_of_warmups’: 1, ‘cudagraph_capture_sizes’: [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256, 272, 288, 304, 320, 336, 352, 368, 384, 400, 416, 432, 448, 464, 480, 496, 512], ‘cudagraph_copy_inputs’: False, ‘cudagraph_specialize_lora’: True, ‘use_inductor_graph_partition’: False, ‘pass_config’: {‘fuse_norm_quant’: False, ‘fuse_act_quant’: False, ‘fuse_attn_quant’: False, ‘enable_sp’: False, ‘fuse_gemm_comms’: False, ‘fuse_allreduce_rms’: False, ‘enable_qk_norm_rope_fusion’: False, ‘fuse_rope_kvcache_cat_mla’: False, ‘fuse_act_padding’: False, ‘fuse_qk_norm_rope_kvcache’: False}, ‘max_cudagraph_capture_size’: 512, ‘dynamic_shapes_config’: {‘type’: <DynamicShapesType.BACKED: ‘backed’>, ‘evaluate_guards’: False, ‘assume_32_bit_indexing’: False}, ‘local_cache_dir’: None, ‘fast_moe_cold_start’: False, ‘static_all_moe_layers’: }, kernel_config=KernelConfig(ir_op_priority=IrOpPriorityConfig(rms_norm=[‘native’], fused_add_rms_norm=[‘native’]), enable_flashinfer_autotune=True, enable_cutedsl_warmup=True, enable_jit_warmup=True, enable_bf16x3_router_gemm=False, moe_backend=‘auto’, linear_backend=‘auto’)
(EngineCore pid=1502) INFO 09-08 09:41:55 [parallel_state.py:1638] world_size=1 rank=0 local_rank=0 distributed_init_method=file:///tmp/vllm_dist_c5e069ef94d940e598f9fee8f4b68d22 backend=nccl
(EngineCore pid=1502) INFO 09-08 09:41:55 [parallel_state.py:1982] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A, EPLB rank N/A
(EngineCore pid=1502) INFO 09-08 09:41:56 [topk_topp_sampler.py:62] Using FlashInfer for top-p & top-k sampling.
(EngineCore pid=1502) WARNING 09-08 09:41:56 [init.py:205] min_p and logit_bias parameters won’t work with speculative decoding.
(EngineCore pid=1502) [transformers] The use_fast parameter is deprecated and will be removed in a future version. Use backend="torchvision" instead of use_fast=True, or backend="pil" instead of use_fast=False.
(EngineCore pid=1502) INFO 09-08 09:42:05 [gpu_model_runner.py:5419] Starting to load model unsloth/Qwen3.8-27B-NVFP4…
(EngineCore pid=1502) INFO 09-08 09:42:06 [cuda.py:545] Using backend AttentionBackendEnum.FLASH_ATTN for vit attention
(EngineCore pid=1502) INFO 09-08 09:42:06 [mm_encoder_attention.py:375] Using AttentionBackendEnum.FLASH_ATTN for MMEncoderAttention.
(EngineCore pid=1502) INFO 09-08 09:42:06 [init.py:720] Selected CutlassFP8ScaledMMLinearKernel for CompressedTensorsW8A8Fp8
(EngineCore pid=1502) INFO 09-08 09:42:06 [deep_gemm.py:186] deep_gemm not found in site-packages, trying vendored vllm.third_party.deep_gemm
(EngineCore pid=1502) INFO 09-08 09:42:06 [deep_gemm.py:213] DeepGEMM PDL enabled on vllm.third_party.deep_gemm.
(EngineCore pid=1502) INFO 09-08 09:42:06 [deep_gemm.py:131] DeepGEMM E8M0 enabled on current platform.
(EngineCore pid=1502) INFO 09-08 09:42:06 [qwen_gdn_linear_attn.py:158] Using Triton/FLA GDN prefill kernel (requested=auto, head_k_dim=128).
(EngineCore pid=1502) INFO 09-08 09:42:06 [qwen_gdn_linear_attn.py:505] GDN decode kernel: cuda
(EngineCore pid=1502) INFO 09-08 09:42:06 [init.py:1100] Using FlashInferCutlassNvFp4LinearKernel for NVFP4 GEMM
(EngineCore pid=1502) INFO 09-08 09:42:06 [cuda.py:486] Using FLASHINFER attention backend out of potential backends: [‘FLASHINFER’, ‘TRITON_ATTN’].
(EngineCore pid=1502) INFO 09-08 09:42:08 [weight_utils.py:858] Filesystem type for checkpoints: EXT4. Checkpoint size: 21.81 GiB. Available RAM: 25.68 GiB.
(EngineCore pid=1502) INFO 09-08 09:42:08 [weight_utils.py:881] Auto-prefetch is disabled because the filesystem (EXT4) is not a recognized network FS (NFS/Lustre). If you want to force prefetching, start vLLM with --safetensors-load-strategy=prefetch.
Loading safetensors checkpoint shards: 0% Completed | 0/2 [00:00<?, ?it/s]
Loading safetensors checkpoint shards: 50% Completed | 1/2 [00:12<00:12, 12.99s/it]
Loading safetensors checkpoint shards: 100% Completed | 2/2 [00:13<00:00, 6.52s/it]
(EngineCore pid=1502)
(EngineCore pid=1502) INFO 09-08 09:42:21 [default_loader.py:430] Loading weights took 13.04 seconds
(EngineCore pid=1502) INFO 09-08 09:42:21 [gpu_model_runner.py:5443] Loading drafter model…
(EngineCore pid=1502) INFO 09-08 09:42:21 [kernel.py:308] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=[‘native’], fused_add_rms_norm=[‘native’])
(EngineCore pid=1502) WARNING 09-08 09:42:21 [vllm.py:1862] max_num_scheduled_tokens is set to 2048 based on the speculative decoding settings. This may lead to suboptimal performance. Consider increasing max_num_batched_tokens to accommodate the additional draft token slots, or decrease num_speculative_tokens.
(EngineCore pid=1502) INFO 09-08 09:42:22 [weight_utils.py:858] Filesystem type for checkpoints: EXT4. Checkpoint size: 21.81 GiB. Available RAM: 25.87 GiB.
Loading safetensors checkpoint shards: 0% Completed | 0/2 [00:00<?, ?it/s]
Loading safetensors checkpoint shards: 50% Completed | 1/2 [00:01<00:01, 1.42s/it]
Loading safetensors checkpoint shards: 100% Completed | 2/2 [00:01<00:00, 1.21it/s]
Loading safetensors checkpoint shards: 100% Completed | 2/2 [00:01<00:00, 1.09it/s]
(EngineCore pid=1502)
(EngineCore pid=1502) INFO 09-08 09:42:24 [default_loader.py:430] Loading weights took 1.84 seconds
(EngineCore pid=1502) INFO 09-08 09:42:24 [llm_base_proposer.py:1484] Detected MTP model. Sharing target model embedding weights with the draft model.
(EngineCore pid=1502) INFO 09-08 09:42:24 [llm_base_proposer.py:1564] Detected MTP model. Sharing target model lm_head weights with the draft model.
(EngineCore pid=1502) INFO 09-08 09:42:25 [gpu_model_runner.py:5515] Model loading took 22.13 GiB memory and 18.180170 seconds
(EngineCore pid=1502) INFO 09-08 09:42:25 [interface.py:911] Setting attention block size to 1600 tokens to ensure that attention page size is >= mamba page size.
(EngineCore pid=1502) INFO 09-08 09:42:25 [interface.py:935] Padding mamba page size by 0.88% to ensure that mamba page size and attention page size are exactly equal.
(EngineCore pid=1502) INFO 09-08 09:42:25 [gpu_model_runner.py:6590] Encoder cache will be initialized with a budget of 16384 tokens, and profiled with 1 image items of the maximum feature size.
(EngineCore pid=1502) INFO 09-08 09:44:14 [backends.py:1094] Using cache directory: /home/edibi/.cache/vllm/torch_compile_cache/661ecbe72a/rank_0_0/backbone for vLLM’s torch.compile
(EngineCore pid=1502) INFO 09-08 09:44:14 [backends.py:1155] Dynamo bytecode transform time: 16.54 s
(EngineCore pid=1502) INFO 09-08 10:25:58 [backends.py:393] Compiling a graph for compile range (1, 2048) takes 2503.84 s
(EngineCore pid=1502) INFO 09-08 10:26:04 [backends.py:920] collected artifacts: 65 entries, 9 artifacts, 31502124 bytes total
(EngineCore pid=1502) INFO 09-08 10:26:04 [decorators.py:708] saved AOT compiled function to /home/edibi/.cache/vllm/torch_compile_cache/torch_aot_compile/ac0d54773ec6032202a176cfb6113db6f3e55390127662f5e037c6936061bb60/rank_0_0/model
(EngineCore pid=1502) INFO 09-08 10:26:04 [monitor.py:53] torch.compile took 2526.89 s in total
(EngineCore pid=1502) /home/edibi/unsloth-nvfp4-env/lib/python3.13/site-packages/triton/language/core.py:2284: UserWarning: tl.make_block_ptr is deprecated. Use TensorDescriptor or tl.make_tensor_descriptor instead.
(EngineCore pid=1502) warn(“tl.make_block_ptr is deprecated. Use TensorDescriptor or tl.make_tensor_descriptor instead.”)
PS C:\Users\edibi> wsl

what does it mean any progress cant start the engine? what to do next

7

@herald server stops at

(EngineCore pid=1516) INFO 09-08 11:03:23 [gpu_model_runner.py:6590] Encoder cache will be initialized with a budget of 16384 tokens, and profiled with 1 image items of the maximum feature size.

no progress bar nothing?

8

Hey, saw your vLLM server has been sitting for about an hour and still isn’t reachable.

I’m building Badgr for exactly this kind of problem, getting GPU workloads/models actually running instead of spending hours debugging the setup.

If you send me the exact vllm serve command + GPU you’re using, I can take a look and see where it’s getting stuck for you.