-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feature(dspy): added compatibility with ANY llamaindex LLM #1225
base: main
Are you sure you want to change the base?
Conversation
@jerryjliu I'm sure this could use a little polishing but what's your thought on this |
@@ -0,0 +1,335 @@ | |||
""" | |||
this code implements a wrapper around the llama_index library to emulate a dspy llm |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
feel free to add these comments into documentation for the LlamaIndexWrapper LM
from llama_index.core.llms import LLM | ||
|
||
|
||
def LlamaIndexOpenAIClientWrapper(llm: LLM): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To clarify, does this support compatibility with all Llamaindex-LLM integrations? It seems to me like the definition here is only for OpenAI clients, which would leave this change redundant to just using DSPy.OpenAI. If it is not just isolated to OpenAI, lets refactor this function to LlamaIndexLLMWrapper
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is supposed to support all llamaindex llms. However the way i use this is by wrapping the llamaindex llm to look like an openai
client object, then i pass this to a dspy azure openai llm.
Llamaindex llm.chat and llm.complete are used for the actual calls
) | ||
|
||
|
||
class DspyLlamaIndexWrapper(LM): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
refactor name to LlamaIndexWrapper. and please add to imports in lm.py as done with other modules
|
||
|
||
class DspyLlamaIndexWrapper(LM): | ||
"""Wrapper around Azure's API for OpenAI. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is the azure openai documentation still relevant here? please refactor if not
|
||
self.history: list[dict[str, Any]] = [] | ||
|
||
def _openai_client(self): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
refactor if the scope of this LM is not limited to just OpenAI
|
||
## ======== test ========= | ||
|
||
if __name__ == '__main__': |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remove and add to documentation for the LM
Hi @FarisHijazi , left some comments here. I mainly want to clarify whether this PR supports all LLM integrations with LlamaIndex as specified or only limited to OpenAI? seems like one could just use DSPy.OpenAI instead? Additionally, can you add the comments and example code to documentation for this LM as done with other LMs in our documentation here? |
this is dope passing it over internally to help take a look very soon! |
I've responded to this in one of the comments: supports all llamaindex llms not just openai I'll get onto addressing the comments within a few days hopefully, mind the messy code it was a quick hack, but now that i have your initial approvals I can invest time in polishing (also waiting for @jerryjliu 's feedback before I start coding) |
hey guys anything happened to this? I just wanna know if I should work on it or if it's no longer needed/already implemented @jerryjliu @arnavsinghvi11 |
hello and thank you for the lovely tool
I've been struggling to get it to work with different LLM providers like Anthropic or some unorthodox LLMs that need to use the huggingface transformers modules, and I saw that dspy implements most of these but not all of them, while llama-index already has everything I've ever needed implemented.
this code implements a wrapper around the llama_index library to emulate a dspy llm
this allows the llama_index library to be used in the dspy framework since dspy has limited support for LLMs
This code is a slightly modified copy of dspy/dsp/modules/azure_openai.py
The way this works is simply by creating a dummy openai client that wraps around any llama_index LLM object and implements llm.complete and llm.chat
tested with python 3.12
dspy==0.1.4
dspy-ai==2.4.9
llama-index==0.10.35
llama-index-llms-openai==0.1.18