Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

server : avoid antiprompt in probabilities of final response #2849

Merged
merged 1 commit into from
Sep 2, 2023

Conversation

jhen0409
Copy link
Sponsor Collaborator

@jhen0409 jhen0409 commented Aug 28, 2023

This fix the probabilities of stopping_word are included in the final response of /completion.

To test response without stream mode:

curl --url http://localhost:8080/completion --header "Content-Type: application/json" \
  --data '{ "n_probs": 1, "prompt": "Hello my name is", "stop": ["I"] }' | json_pp

With stream mode (see the last event):

curl -N --url http://localhost:8080/completion --header "Content-Type: application/json" \
  --data '{ "stream": true, "n_probs": 1, "prompt": "Hello my name is", "stop": ["I"] }'

Or see the console output in the web UI.

@ggerganov ggerganov changed the title server : avoid aniprompt in probabilities of final response server : avoid antiprompt in probabilities of final response Aug 28, 2023
Copy link
Sponsor Collaborator

@SlyEcho SlyEcho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, that does the trick.

@jhen0409 jhen0409 merged commit 571083f into ggerganov:master Sep 2, 2023
25 checks passed
@jhen0409 jhen0409 deleted the fix-server-final-probs branch September 2, 2023 00:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants