Python: #6761 Onnx Connector

[nmoeller]

Motivation and Context

1. Why is this changed required ?
   To enable Onnx Models with Semantic Kernel, there was the issue Python: Add support for local models via ONNX #6761 in the Backlog to add a Onnx Connector
2. What problem does it solve ?
   It solves the problem, that semantic kernel is not yet integrated with Onnx Gen Ai Runtime
3. What scenario does it contribute to?
   The scenario is to use different connector than HF,OpenAI or AzureOpenAI. When User's want to use Onnx they can easliy integrate it now
4. If it fixes an open issue, please link to the issue here.
   Python: Add support for local models via ONNX #6761

Description

The changes made are designed by my own based on other connectors, i tried to stay as close as possible to the structure.
For the integration i installed the mistral python package in the repository.

I added the following Classes :

• OnnxCompletionBase --> Responsible to control the inference
• OnnxTextCompletion --> Inherits from OnnxCompletionBase
  • Support for Text Completion with and without Images
  • Ready for Multimodal Inference
  • Ready for Text Only Inference
  • Supports all Models on onnxruntime-genai
• OnnxChatCompletion -->Inherits from OnnxCompletionBase
  • Support for Text Completion with and without Images
  • The user needs to provide the corresponding chat template to use this class
  • Ready for Multimodal Inference
  • Ready for Text Only Inference
  • Supports all Models on onnxruntime-genai

What is integrated yet :

• OnnxCompletionBase Class
• OnnxChatCompletionBase Class with Dynamic Template Support
• OnnxTextCompletionBase Class
• Sample Multimodal Inference with Phi3-Vision
• Sample of OnnxChatCompletions with Phi3
• Sample of OnnxTextCompletions with Phi3
• Integration Tests
• Unit Tests

Some Notes

Contribution Checklist

• The code builds clean without any errors or warnings
• The PR follows the SK Contribution Guidelines and the pre-submission formatting script raises no violations
• All unit tests pass, and I have added new tests where possible
• I didn't break anyone 😄

[TaoChenOSU]

Regarding our offline conversation on the prompt template, is using a prompt template to parse the chat history to some format an overkill? Prompt template can do much more that substituting arguments. Is it possible to override the _prepare_chat_history_for_request method to get what Onnx wants?

[moonbox3]

Thanks for working on this! Some questions for you.

[TaoChenOSU]

What I meant here is that self.tokenizer is an object. We probably should not call an object directly. Please verify.

Eduard is currently oof, and this change request could be blocking.

[moonbox3]

A couple of typos we'll need to fix:

Warning: "interogate" should be "interrogate".
Warning: "choosen" should be "chosen".

[moonbox3]

Also, this is failing on the MacOS unit tests:

Using CPython 3.10.11 interpreter at: /Library/Frameworks/Python.framework/Versions/3.10/bin/python
Creating virtual environment at: .venv
Resolved 288 packages in 4.62s
error: distribution onnxruntime-genai==0.4.0 @ registry+https://pypi.org/simple can't be 
installed because it doesn't have a source distribution or wheel for the current platform