You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently the 'https://hub.docker.com/r/vllm/vllm-openai/' image uses Cuda 12.1 - this runs into a lot of cuda versioning issues depending on the drivers used on the underlying GPU.
This makes the image an inconsistent starting point for running on services like vast ai or runpod.
Could the docker image be updated to more dynamically support cuda versions from 11.8 and up?
The text was updated successfully, but these errors were encountered:
Currently the 'https://hub.docker.com/r/vllm/vllm-openai/' image uses Cuda 12.1 - this runs into a lot of cuda versioning issues depending on the drivers used on the underlying GPU.
This makes the image an inconsistent starting point for running on services like vast ai or runpod.
Could the docker image be updated to more dynamically support cuda versions from 11.8 and up?
The text was updated successfully, but these errors were encountered: