Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache-control headers are not set when polling agent status #1673

Open
1 task done
robbyt opened this issue Aug 25, 2024 · 1 comment
Open
1 task done

Cache-control headers are not set when polling agent status #1673

robbyt opened this issue Aug 25, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@robbyt
Copy link

robbyt commented Aug 25, 2024

Confirm this is an issue with the Python library and not an underlying OpenAI API

  • This is an issue with the Python library

Describe the bug

When using client.beta.threads.runs.create_and_poll, I would expect the Cache-Control header to be set to no-cache, so that an upstream caching proxy does not store the agent response while polling the same API URL.

To Reproduce

  1. Setup a caching proxy, e.g., llm_proxy
$ git clone git@github.com:Proxati/llm_proxy.git
$ cd llm_proxy
  1. Start the proxy in cache mode
$ go run main.go cache --debug
  1. Use the Agent API to make a request
$ cd llm_proxy/examples/python
$ poetry run agent/agent.py
  1. The agent will poll status forever, because the cache will store the status response because the request doesn't have a Cache-Control header set.

Code snippets

thread = client.beta.threads.create()

message = client.beta.threads.messages.create(
    thread_id=thread.id,
    role="user",
    content="Can you help me? How does AI work?",
)

run = client.beta.threads.runs.create_and_poll(
    thread_id=thread.id,
    assistant_id=assistant.id,
    instructions="When using a caching proxy, you will never return a 'completed' status",
)


### OS

any

### Python version

any

### Library version

openai-1.42.0
@robbyt robbyt added the bug Something isn't working label Aug 25, 2024
@RobertCraigie
Copy link
Collaborator

Thanks for reporting, in the meantime you can explicitly set the cache control yourself

run = client.beta.threads.runs.create_and_poll(
    thread_id=thread.id,
    assistant_id=assistant.id,
    instructions="When using a caching proxy, you will never return a 'completed' status",
    extra_headers={'Cache-Control': 'no-cache'},
)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants