Skip to content

Commit

Permalink
fix: check all tokens for output_tokens_before_timeout
Browse files Browse the repository at this point in the history
  • Loading branch information
sjmonson committed Sep 26, 2024
1 parent 3e559a2 commit f551d51
Showing 1 changed file with 2 additions and 5 deletions.
7 changes: 2 additions & 5 deletions plugins/openai_plugin.py
Original file line number Diff line number Diff line change
Expand Up @@ -297,11 +297,8 @@ def streaming_request_http(self, query: dict, user_id: int, test_end_time: float

# If the current token time is outside the test duration, record the total tokens received before
# the current token.
result.output_tokens_before_timeout = 0
for i, token in enumerate(tokens, 1):
if token['time'] > test_end_time:
break
result.output_tokens_before_timeout = i
tokens_before_timeout = [t for t in tokens if t['time'] <= test_end_time]
result.output_tokens_before_timeout = len(tokens_before_timeout)

# Last token comes with finish_reason set.
result.stop_reason = deepget(resps[-1], "choices", 0, "finish_reason")
Expand Down

0 comments on commit f551d51

Please sign in to comment.