Add support for remote models (OpenAI, ...)

[cztomsik]

• Add OPEN_AI_KEY, etc. in Settings
• Update <ModelSelect, to check if this is filled and to include OpenAI models
• If any OpenAI model is selected make it visually distinctive, that you are using remote model
• Hide "partial" completion in <EditMessage if remote model is selected
• call remote endpoints
  • these are different to what we are doing right now, so maybe we can first add /api/chat/completions endpoint which will just wrap what we do in client-side and then if we are using remote endpoint, we can just proxy

[prabirshrestha]

Would be great if Ava supported remote openai api. This would allow us to reuse the server and avoid loading the model multiple times if we are using it in a different app.

• Ollama recently added openai compatibility layer. https://ollama.com/blog/openai-compatibility
• https://edgen.co is another one that adds openai compatibility layer.

[sammcj]

Would be really great if Ava supported using an already running Ollama instance via its API!

[cztomsik]

Would be really great if Ava supported using an already running Ollama instance via its API!

Yes, this is in the works, but not finished yet.

[cztomsik]

Just a small update, the UI part has been rewritten and we now have /api/chat/completions endpoint which is mostly compatible with openai, so hopefully, we are really close to closing this.

What's missing:

• add a new field to the Settings page for OpenAI API key
• if this is filled in, we should offer gpt models in the <ModelSelect (it's not yet clear which ones, and how to configure that)
• decide if the real api request is going to happen from the browser (simple) or from our /api endpoint (more work), because the first option would make the api key visible in the web browser devtools panel
• disable some features if openai model is selected