You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
If you work on domain corpora, collecting additional training data to improve your reader or retriever models is very helpful.
For collecting training data, there are two main options:
a) Manual Labelling
b) User feedback for "live predictions"
b) is particularly promising if you don't have enough time or resources for a) OR if you want to continuously improve your model once it is production.
We already have the API for storing user feedback and used it successfully in some deployments. In these deployments, we used proprietary user interfaces to collect feedback. Let's add those feedback buttons also to our Haystack Demo UI based on streamlit so that everybody can experiment with it more easily and understand the mechanics.
Describe the solution you'd like
Add 👍 / 👎 buttons in the streamlit UI that allow capturing user feedback per result
Optionally: When clicking 👎 we could distinguish between "wrong answer and wrong passage" and "wrong answer, but correct passage" (this distinction helps for the training of retriever models)
Is your feature request related to a problem? Please describe.
If you work on domain corpora, collecting additional training data to improve your reader or retriever models is very helpful.
For collecting training data, there are two main options:
a) Manual Labelling
b) User feedback for "live predictions"
b) is particularly promising if you don't have enough time or resources for a) OR if you want to continuously improve your model once it is production.
We already have the API for storing user feedback and used it successfully in some deployments. In these deployments, we used proprietary user interfaces to collect feedback. Let's add those feedback buttons also to our Haystack Demo UI based on streamlit so that everybody can experiment with it more easily and understand the mechanics.
Describe the solution you'd like
Additional context
https://haystack.deepset.ai/docs/latest/domain_adaptationmd#User-Feedback
The text was updated successfully, but these errors were encountered: