generator: Ollama #876
Open
generator: Ollama #876
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The tests are skipped when no Ollama server can be found.
|
DCO Assistant Lite bot All contributors have signed the DCO ✍️ ✅ |
|
I have read the DCO Document and I hereby sign the DCO |
|
recheck |
leondz
added
generators
|
Thanks for this! Will take a look |
jmartin-tech
requested changes
Sep 4, 2024
Co-authored-by: Jeffrey Martin <jmartin@Op3n4M3.dev> Signed-off-by: Martin <31307962+martinebl@users.noreply.github.com>
…predicate instead
|
|
||
| @backoff.on_exception( | ||
| backoff.fibo, | ||
| (TimeoutError, ollama.ResponseError), |
There was a problem hiding this comment.
Testing shows TimeoutError may not be the exception that should be handled here.
This exception here is an OSError not a request timeout.
class TimeoutError(OSError): ...Example failure:
File "/home/jemartin/miniconda3/envs/garak/lib/python3.12/site-packages/httpx/_transports/default.py", line 69, in map_httpcore_exceptions
yield
File "/home/jemartin/miniconda3/envs/garak/lib/python3.12/site-packages/httpx/_transports/default.py", line 233, in handle_request
resp = self._pool.handle_request(req)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/jemartin/miniconda3/envs/garak/lib/python3.12/site-packages/httpcore/_sync/connection_pool.py", line 216, in handle_request
raise exc from None
File "/home/jemartin/miniconda3/envs/garak/lib/python3.12/site-packages/httpcore/_sync/connection_pool.py", line 196, in handle_request
response = connection.handle_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/jemartin/miniconda3/envs/garak/lib/python3.12/site-packages/httpcore/_sync/connection.py", line 101, in handle_request
return self._connection.handle_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/jemartin/miniconda3/envs/garak/lib/python3.12/site-packages/httpcore/_sync/http11.py", line 143, in handle_request
raise exc
File "/home/jemartin/miniconda3/envs/garak/lib/python3.12/site-packages/httpcore/_sync/http11.py", line 113, in handle_request
) = self._receive_response_headers(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/jemartin/miniconda3/envs/garak/lib/python3.12/site-packages/httpcore/_sync/http11.py", line 186, in _receive_response_headers
event = self._receive_event(timeout=timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/jemartin/miniconda3/envs/garak/lib/python3.12/site-packages/httpcore/_sync/http11.py", line 224, in _receive_event
data = self._network_stream.read(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/jemartin/miniconda3/envs/garak/lib/python3.12/site-packages/httpcore/_backends/sync.py", line 124, in read
with map_exceptions(exc_map):
File "/home/jemartin/miniconda3/envs/garak/lib/python3.12/contextlib.py", line 158, in __exit__
self.gen.throw(value)
File "/home/jemartin/miniconda3/envs/garak/lib/python3.12/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
raise to_exc(exc) from exc
httpcore.ReadTimeout: timed out
The expected timeout class would be httpx.TimeoutException which would cover ConnectTimeout, ReadTimeout, WriteTimeout, & PoolTimeout:
from httpx import TimeoutException
Suggested change
| (TimeoutError, ollama.ResponseError), | |
| (TimeoutException, ollama.ResponseError), |
This would apply to both implementation of _call_model().
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add two Ollama generators, using either the chat or generate functions from the Ollama package.
Add tests, that are skipped when no Ollama server is running.