-
Notifications
You must be signed in to change notification settings - Fork 30
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feature(ApplicationManager): handle podman connections #1528
feature(ApplicationManager): handle podman connections #1528
Conversation
Short demoThis PR does not expose UI, however using the dev tools, we can make some experiments On my Linux fedora 40, I have a Inside AI Lab I can open the dev tools and check those connections Let's store them inside a Let's start a code generation recipe on the qemu machine studioClient.requestPullApplication({
'modelId': 'hf.ibm-granite.granite-8b-code-instruct',
'recipeId': 'codegeneration',
connection: connections[2]
}); And now, let's start a text_to_audio recipe on the native connection studioClient.requestPullApplication({
'modelId': 'hf.ggerganov.whisper.cpp',
'recipeId': 'audio_to_text',
connection: connections[1]
}); Finally we have two pod running on two different machines Let's look at the console
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Signed-off-by: axel7083 <42176370+axel7083@users.noreply.github.com>
Signed-off-by: axel7083 <42176370+axel7083@users.noreply.github.com>
Signed-off-by: axel7083 <42176370+axel7083@users.noreply.github.com>
Signed-off-by: axel7083 <42176370+axel7083@users.noreply.github.com>
Signed-off-by: axel7083 <42176370+axel7083@users.noreply.github.com>
Signed-off-by: axel7083 <42176370+axel7083@users.noreply.github.com>
Signed-off-by: axel7083 <42176370+axel7083@users.noreply.github.com>
Signed-off-by: axel7083 <42176370+axel7083@users.noreply.github.com>
Signed-off-by: axel7083 <42176370+axel7083@users.noreply.github.com>
Signed-off-by: axel7083 <42176370+axel7083@users.noreply.github.com>
Signed-off-by: axel7083 <42176370+axel7083@users.noreply.github.com>
Signed-off-by: axel7083 <42176370+axel7083@users.noreply.github.com>
26c4f3c
to
622fb7d
Compare
What does this PR do?
This PR makes the application manager requires a
ContainerProviderConnection
argument when pulling a recipe. This connection will be propagated to the RecipeManager and the BuilderManager, to respect the connection to use.This allow us to target a specific connection if multiple exists. For now we keep the existing behaviour of taking the first running.
Screenshot / video of UI
N/A
What issues does this PR fix or reference?
Fixes #1527
Requires for #1462
How to test this PR?
Manuallyy