hf-vllm-docker

Docker image to serve a LLM on vLLM using Huggingface inference endpoints DockerHub image

Go to deploy a model on huggingface

Use the image under advanced settings: docker.io/tsunkaraneni/vllm-openai-hf:latest

Optionally set any vLLm env variables