Vllm serve… huggingface gives run instructions for every model with vllm on thei... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

Der_Einzige on April 20, 2025 | parent | context | favorite | on: Gemma 3 QAT Models: Bringing AI to Consumer GPUs

Vllm serve… huggingface gives run instructions for every model with vllm on their website.

Zambyte on April 20, 2025 [–]

How do I serve multiple models? I can pick from dozens of models that I have downloaded through Open WebUI.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact