Cancellation of search requests are not reliably forwarded to the server-side search strategy

[weltenwort]

Kibana version: master as of dbd14ad

Browser version: Chromium 87.0.4280.88

Original install method (e.g. download page, yum, from source, etc.): source

Describe the bug:

Async searches started via the data.search.search() API can be cancelled by passing an AbortSignal. When the AbortSignal is triggered while the search logic is polling the async result, the search is supposed be canceled by issuing a DELETE request to a specific data plugin route:

kibana/x-pack/plugins/data_enhanced/public/search/search_interceptor.ts

Lines 89 to 92 in 051bbf0

	catchError((e: AbortError) => {
	if (id && !isSavedToBackground) this.deps.http.delete(`/internal/search/${strategy}/${id}`);
	return throwError(this.handleSearchError(e, timeoutSignal, options));
	}),

Whether that DELETE request is actually sent depends on the timing of the abort signalling and the unsubscription from the response observable. If the unsubscription happens too early the error representing the cancellation never reaches the catchError operator. That means that the Kibana server and therefore the Elasticsearch server never learn about the cancellation.

Steps to reproduce:

1. Start a browser-side async search request with an AbortSignal via data.search.search(), which runs for several polling cycles.
2. Unsubscribe from the response Observable.
3. Trigger the AbortSignal.
4. Observe that the DELETE request to cancel the search on the server side is not sent and the search continues to run.

Expected behavior:

The DELETE request is sent regardless of the order of unsubscription or AbortSignal triggering.

Additional context from a conversation with @lizozom:

This interacts with the background sessions functionality, so care has to be taken not to delete the search too eagerly. Something like the following tapUnsubscribe operator might be used to implement more robust cancellation semantics (if guarded correctly).

const tapUnsubscribe = (onUnsubscribe: () => void) => <T>(source$: Observable<T>) => {
  return new Observable<T>((subscriber) => {
    const subscription = source$.subscribe({
      next: (value) => subscriber.next(value),
      error: (error) => subscriber.error(error),
      complete: () => subscriber.complete(),
    });

    return () => {
      onUnsubscribe();
      subscription.unsubscribe();
    };
  });
};

[elasticmachine]

Pinging @elastic/kibana-app-services (Team:AppServices)