vLLM - 1 Click Deployment on Cloudron

General
1

vLLM packaged for Cloudron: one-click self-hosted deployment on the OpenAI API

Cloudron is a self-hosting platform: you point it at a domain and it installs, updates, backs up and issues TLS for applications, largely with one click and largely unattended. There is now an unofficial community package that puts vLLM on it.

Disclosure: this package and post were produced with AI assistance , with human review and real-hardware verification throughout, in the spirit of vLLM’s AI-assisted contribution guidance.

The one-click experience

Installing from the package’s versions URL gives you vLLM 0.26.0’s OpenAI-compatible server on your own domain: a generated bearer key protecting /v1, the model cache on a persistent path deliberately excluded from platform backups (weights are reproducible; a 30 GB model should never ride a nightly backup), streaming verified through the platform’s reverse proxy, and honest CPU-only framing (the platform cannot pass GPUs to app containers yet).

cloudron install \
  --versions-url https://raw.githubusercontent.com/OrcVole/vllm-cloudron/main/CloudronVersions.json \
  --location vllm.example.com

The package is a thin wrapper by design: your prebuilt +cpu release wheel installed into a venv on Cloudron’s base image, an nginx front end for health and streaming, and a start script. vLLM itself ships unmodified.

Everything user-facing was proven on a real box with an acceptance ladder: install and first run, auth boundaries, streamed flows through the proxy, backup and restore survival, and memory sizing from cgroup measurement (the shipped limit is 10 GB because the default small model idles at ~7.4 GB with its preallocated KV cache).

Observations from packaging, offered gratefully

  • A liveness endpoint that answers before model load would remove a whole class of packaging work. /health today answers only after engine initialisation, and on a platform that health-checks during install, a first boot that downloads a model gets killed mid-download. The package fronts vLLM with nginx purely to answer liveness immediately; a native early-binding /health (with readiness split out) would make that shim unnecessary for every orchestrated deployment, not just Cloudron.

  • A model’s declared context can abort engine start on CPU defaults. Qwen3-0.6B declares a 40960-token context, which alone needs 4.4 GiB of KV cache against the CPU backend’s 4 GiB default, and the engine exits. The error message is excellent; a CPU-docs note that --max-model-len is effectively mandatory for small-RAM deployments would save the next packager an evening.

  • The VLLM_* environment namespace warning is a small trap for wrappers. Anything wrapping the server naturally reaches for VLLM_-prefixed settings and then triggers the unknown-variable warning at every boot. A documented statement that the namespace is reserved (it is a good warning!) would push wrappers to their own prefix from day one.

  • The prebuilt CPU wheels are excellent, with one resolver wrinkle. The exact setuptools pin in the release wheel is unsatisfiable for uv’s first-index strategy against the PyTorch CPU extra index; plain pip resolves it. Worth a line in the CPU install docs for uv users.

  • Official CPU sizing guidance would help self-hosters. Rough tokens-per-second expectations by instruction set tier (AVX2 versus AVX-512 versus AMX) and a RAM rule of thumb (weights + KV cache + ~2.5 GB runtime overhead was our measurement) would let people size before installing rather than after.

Links

vLLM developers and users are warmly welcome in the Cloudron forum thread. Corrections to anything above are equally welcome. The package is unofficial and unaffiliated; vLLM is shipped unmodified under Apache-2.0, and nothing here speaks for the vLLM project.

If you are interested in improving vLLM for packagers, there are some notes in the docs in the repo.

2

Yes, there is an unofficial community package for vLLM on Cloudron, enabling one-click self-hosted deployment of vLLM’s OpenAI-compatible API server. The package installs vLLM 0.26.0 (CPU-only, as Cloudron does not support GPU passthrough), sets up a secure endpoint with a generated bearer key, manages model weights outside backups, and uses nginx for health checks and streaming. Installation is via the Cloudron dashboard or CLI with the provided versions URL, and the package is a thin wrapper around the official vLLM release wheel, leaving vLLM itself unmodified. For details, see the package repo and Cloudron app directory.

Key packaging notes include: the need for a liveness endpoint before model load, context length and KV cache sizing for small-RAM deployments, reserved environment variable namespaces, and CPU sizing guidance. The package has been tested on real hardware and is suitable for private, unattended LLM serving on x86 CPUs. Would you like more detail on installation steps or configuration options?

Sources:

Hint: Mention @Herald in the post for followups.