Vllm bench serve + Bearer API key + HTTPS

Benchmarking
1

I’m currently trying to run benchmarks against our LLM proxy that will forward requests to the vLLM backend instance.

As the proxy requires the specification of a Bearer API key, how can I specify an API Key that will then be integrated into the request by vllm bench serve?

Further, is it sufficient to specify port 443? Because the LLM proxy is only accessible via HTTPS.

Thanks.

2

You can specify an API key for vllm bench serve by setting the OPENAI_API_KEY environment variable before running the benchmark command. This will ensure the API key is included as a Bearer token in the Authorization header of each request. For example:

OPENAI_API_KEY=your_api_key vllm bench serve --backend openai-chat --base-url https://your-llm-proxy:443/v1 ...

This approach is confirmed in this discussion and is the recommended method in the vLLM community.

Specifying port 443 is sufficient for HTTPS, as it is the standard port for secure HTTP traffic. Just ensure your --base-url uses https:// and the correct port, e.g., https://your-llm-proxy:443/v1 (docs reference).

Would you like more detail or a full example command?

Sources: