Support conjunctive queries in sparse retrieval

[tstadel]

Currently we only support disjunctive queries in sparse retrieval. I.e. only one query term needs to be present in a document in order to be retrieved. This is a highly recall-favouring approach. In some use-cases (e.g. document retrieval) a more precision-focused approach might be better, however. Conjunctive queries ensure that all query terms must be present in a document in order to be retrieved. Within elasticsearch/OpenSearch this could easily be achieved by setting the operator param of the match clause to AND.

Proposed changes:

• add all_terms_must_match to ElasticsearchRetriever's query() method
• set operator of match clause in elasticsearch and OpenSearch queries according to all_terms_must_match
• delegate all_terms_must_match to DeepsetCloudDocumentStore

Status (please check what you already did):

• First draft (up for discussions & feedback)
• Final code
• Added tests
• Updated documentation

[tstadel]

@bogdankostic it would be great if you could take a look at this and especially the naming of the param all_terms_must_match. I also thought about naming this param operator or query_operator and restricting the values to AND and OR as in elasticsearch. But ultimately I opted for the current approach as this is more generic and less coupled to the concrete elasticsearch API.

[tstadel]

@Timoeller I'd also like to hear your opinion on the naming of the param all_terms_must_match. Is this comprehensive for our users?

[bogdankostic]

Looking already pretty good to me!
I think the naming of all_terms_must_match is okay, as this param is well explained in the docstrings. I am happy to approve this PR once the test test_DeepsetCloudDocumentStore_query is fixed and labels are added :)

[Timoeller]

I think the term is a bit long but fine, it describes what is happening pretty well.

Although I like this feature I thought we had a discussion about conjunctive matching between terms and filters. Will this come in a later PR?

[tstadel]

I think the term is a bit long but fine, it describes what is happening pretty well.

Although I like this feature I thought we had a discussion about conjunctive matching between terms and filters. Will this come in a later PR?

Alright @Timoeller, conjunctive matching between terms and filters is already handled in #2359.

[tstadel]

@bogdankostic test is fixed. Currently there is a test failing due to some hf downtime I guess.

[bogdankostic]

LGTM