https://docs.vllm.ai/en/latest/getting_started/installation/cpu/#apple-silicon

Installing on Macbook Pro M1

After a successful build according to the docs, the server fails to start. This error is the issue:

WARNING 02-01 16:22:28 [interface.py:219] Failed to import from vllm._C: ImportError('dlopen(/Users/erikvw/vllm/vllm/_C.abi3.so, 0x0002): Symbol not found: __ZN3c1013MessageLoggerC1EPKcii\n  Referenced from: /Users/erikvw/vllm/vllm/_C.abi3.so\n  Expected in:     <8D97C466-D00B-36C7-8E36-A2E591A516BD> /Users/erikvw/vllm/.venv/lib/python3.12/site-packages/torch/lib/libc10.dylib')

Made some slight changes to the build to resolve the above error

remove old .venv and clear uv cache

rm -rf .venv
uv clean --force

clone repo

git clone https://github.com/vllm-project/vllm.git
cd vllm

create venv and activate

uv venv --python 3.12 --seed
source .venv/bin/activate

install requirements

# 2 lines below not mentioned in docs
export VLLM_TARGET_DEVICE=cpu
export CMAKE_BUILD_TYPE=Release

uv pip install -r requirements/cpu.txt --index-strategy unsafe-best-match

# line below not mentioned in docs
uv pip install cmake ninja "setuptools>=61" "setuptools-scm>=8"

build vllm

# added --no-build-isolation
uv pip install -e . --no-build-isolation

start up server

export VLLM_CPU_KVCACHE_SPACE=40
export VLLM_CPU_NUM_OF_RESERVED_CPU=1
vllm serve facebook/opt-125m --dtype=bfloat16