If lscpu | grep -i avx returns nothing, your CPU does not support AVX, AVX2, or AVX512 instructions. vLLM requires at least one of these (or Power9+, S390X, or ARMv8) for CPU inference, so your hardware is not compatible with vLLM’s CPU backend and you cannot run vLLM on this machine’s CPU as per the official documentation and error messages. There is no supported workaround for CPUs lacking these instruction sets.
See vLLM CPU requirements and related GitHub issues for confirmation. Would you like advice on alternative hardware or LLM inference solutions for older CPUs?
Sources: