Improved search timeout checking capabilities:

[markharwood]

• Now accounts for time from creation of SearchContext
• new “isTimedOut” helper method on SearchContext for use by known hotspots
• IncludeExclude is a known hotspot and so added checks for timeouts
• Fixed ThreadPool which was looking for a timer config setting name that its own validation code prohibited

Closes #9156

[clintongormley]

@s1monw and @rmuir could you take a look at this scaled down PR for the improved timeouts please?

[markharwood]

I took a look at rebasing this recently.
The problem areas this PR was tackling have undergone changes which made it difficult to merge and may have invalidated the purpose of the PR.

• Configuration settings had undergone a significant rework which means that the timeout interval settings were renamed (and now may actually honour any custom settings in the config). If there is a residual issue this should be tackled with a more targeted issue/PR.
• The IncludeExclude regex hotspot had fundamentally been rewritten to use Lucene automaton and as such hit a new error ("too many states") pretty quickly before it got into any real death-spins (see Terms agg fails with medium-large "exclude" clause lists #11176 )

[clintongormley]

thanks @markharwood - sounds like we can close this one then? (and #4586 as well?)

[nemobis]

Configuration settings had undergone a significant rework which means that the timeout interval settings were renamed (and now may actually honour any custom settings in the config).

Thanks. In what release does this happen and could you be so kind as to link the docs for it?

[roykachouh]

+1 I'd like to add this feature as well, but I'm having a tough discerning if it was actually implemented and in which version?

[sameerr25]

+1 Which PR improved the search timeout capabilities, can someone provide link to it?

[markharwood]

Which PR improved the search timeout capabilities,

Improving timeout capabilities is a process of weaving checks into all the "hot loops".
The main hot loop is the the collecting logic for processing each doc and that is handled at a low level in Lucene's TimeLimitingCollector class.
We had a hot-loop in the regex handling of terms agg include clauses but that logic was rewritten to be much faster using Lucene-level classes here: aecd9ac