Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OpenAI API: new parameter to change model list behavior #6358

Open
wants to merge 7 commits into
base: dev
Choose a base branch
from

Conversation

jsboige
Copy link

@jsboige jsboige commented Sep 1, 2024

Checklist:

This is an attempt at moving on with the following PR, which got stalled because there isn't one single behavior for API's model list composition that fits all uses:
Some cases require the dummy OpenAI models exposed, others local models instead, and in the latter case some may need to list all models available whereas others will want to only expose the currently loaded local model.

This PR introduced an additional parameter in the form of a flag allowing to expose dummy models and/or (all local models or single loaded local model)

@Touch-Night
Copy link
Contributor

Can this pr enable user to switch model (Unload the currently loaded model and load the selected model) through api?

@jsboige
Copy link
Author

jsboige commented Sep 2, 2024

Can this pr enable user to switch model (Unload the currently loaded model and load the selected model) through api?

@Touch-Night, this PR is only about controlling what's returned by the model list API handle.

Switching model is available with this handle, which calls this method, which in turn calls that method.

Now with this PR, if you choose to return the currently loaded model (value 6 or 7 for the new parameter) with the request for the model list, if you switch model on the fly, through ooba's web ui or through the corresponding handle, the response for the list handle will update accordingly. I was able to test that in Open-Webui: just refresh the conversation page and the drop down list now displays the new model.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants