[Search] Consistent support for track_total_hits

[timroes]

Currently, the search service defaults to sending track_total_hits=true to provide the exact hit count for Elasticsearch searches (resulting in slower queries). Before returning the response to the consumer of the service, we modify the output from something like {total: {value: 1, relation: 'eq'}} to {total: 1}.

If a consumer of the service doesn't want to track the total number of hits, it can specify track_total_hits=false to override this default behavior.

There are, however, a few problems with this approach:

• There isn't any way for applications using SearchSource (or esaggs for that matter) to override this parameter.
• The track_total_hits Elasticsearch parameter also allows numbers. When specifying as a number, if the total number of hits in the response is greater than the number, the response from Elasticsearch looks like {total: {value: 1, relation: 'gte'}}, yet we are still returning {total: 1} to the consumer.

We need to provide a path for applications using our search service to migrate from the old behavior (specifying total as a number) to the new behavior (getting a value and relation).

I propose that from the search service, we always return the raw Elasticsearch response. Then, we can add an additional parameter in SearchSource that allows the consumer to specify which response behavior it receives (something like legacyHitsTotal).

A couple of open questions:

• Is there any usage of our search service directly that utilizes the current total format we'd need to update?
• What should the default value be for the new parameter in SearchSource?

See also: #26356

[elasticmachine]

Pinging @elastic/kibana-app-arch (Team:AppArch)

[lukasolson]

Removed "blocker" label since we are no longer sending the deprecated rest_total_hits_as_int parameter, and instead are sending track_total_hits and shimming the response to look as it did before https://www.elastic.co/guide/en/elasticsearch/reference/current/breaking-changes-7.0.html.