Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Search] fix cancelation related memory leaks #81996

Merged
merged 4 commits into from
Oct 30, 2020

Conversation

Dosant
Copy link
Contributor

@Dosant Dosant commented Oct 29, 2020

Summary

Attempts to fix #65051
See explanation: #65051 (comment)

For my experiments with branch the leak is fixed.

Checklist

Delete any items that are not applicable to this PR.

@Dosant Dosant added Feature:Search Querying infrastructure in Kibana v8.0.0 Team:AppArch release_note:skip Skip the PR/issue when compiling release notes v7.11.0 labels Oct 29, 2020
@Dosant Dosant changed the title [Search] fix cancelation related memory leaks on a search layer [Search] fix cancelation related memory leaks Oct 29, 2020
@Dosant Dosant requested review from lizozom and lukasolson and removed request for lizozom October 29, 2020 12:19
@@ -59,7 +59,7 @@ describe('ExecutionContract', () => {

test('can cancel execution', () => {
const execution = createExecution('foo bar=123');
const spy = jest.spyOn(execution, 'cancel');
const spy = jest.spyOn(execution, 'cancel').mockImplementation(() => {});
Copy link
Contributor Author

@Dosant Dosant Oct 29, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed that test were silently failing tests (on master)

(node:57027) UnhandledPromiseRejectionWarning: AbortError: Aborted
(node:57027) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:57027) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

I assume that it warns is that cancel is called without start and there is no one subscribed to handled inner AbortRejection promise of the execution.
I mocked it, since this unit test for execution_contract and we are not testing an execution here.

Copy link
Contributor Author

@Dosant Dosant Oct 29, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just fixed that particular higher level test, but really, probably we should also handle and test edge case:

const ex = new Execution();
ex.cancel() // ?? what happens? 
ex.start() // ?? what happens? 

@Dosant Dosant marked this pull request as ready for review October 29, 2020 14:22
@Dosant Dosant requested a review from a team as a code owner October 29, 2020 14:22
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-app-arch (Team:AppArch)

Copy link
Member

@lukasolson lukasolson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked over these changes and while I'm not a huge fan of adding properties to built-in classes, I can't really think of much of a better way to accomplish this (other than what's suggested below, which I'm also not a huge fan of). Feel free to move forward with this approach unless you think the suggested approach feels cleaner.

return new Promise((resolve, reject) => {
if (signal.aborted) reject(new AbortError());
const abortHandler = () => {
export function toPromise(signal: AbortSignal): Promise<never> & { cleanup: () => void } {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I'm not a huge fan of adding a method to a built-in class (Promise). We might consider returning an object instead (like { promise, cleanup }). I think it will also help to enforce that any consumers of this function will handle cleaning up and not just use the promise returned.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't mind initial approach I took mostly because it is clearly typed that it has a cleanup method, nevertheless:

I think it will also help to enforce that any consumers of this function will handle cleaning up and not just use the promise returned.

I agree with this 👍 changed {promise, cleanup}

one tiny caveat is that now we can't just use:

Promise.race(signals.map(toPromise))

but have to:

Promise.race(signals.map(toPromise).map(pr => pr.promise))

const cleanup = () => {
subscription.unsubscribe();
combinedController.signal.removeEventListener('abort', cleanup);
combinedController.abort();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the reason we need to abort here because we need to trigger the internal cleanup from getCombinedController? If so, maybe we can do similarly to what I've suggested above (instead of returning just the signal from getCombinedSignal, return something like {signal, cleanup}). Thoughts?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the reason we need to abort here because we need to trigger the internal cleanup from getCombinedController

Yes, exactly.

If so, maybe we can do similarly to what I've suggested above (instead of returning just the signal from getCombinedSignal, return something like {signal, cleanup})

Agree. Done!

@botelastic botelastic bot added the Feature:ExpressionLanguage Interpreter expression language (aka canvas pipeline) label Oct 29, 2020
@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

page load bundle size

id before after diff
data 950.3KB 950.8KB +503.0B
dataEnhanced 27.0KB 27.1KB +45.0B
expressions 182.1KB 182.2KB +53.0B
total +601.0B

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@Dosant Dosant merged commit 7833fb3 into elastic:master Oct 30, 2020
Dosant added a commit to Dosant/kibana that referenced this pull request Oct 30, 2020
gmmorris added a commit to gmmorris/kibana that referenced this pull request Nov 2, 2020
* master:
  Add derivative function (elastic#81178)
  [Discover] Deangularize context_app.html, part 3 (elastic#81838)
  [Visualize] Vis listing page breaks on unknown vis type (elastic#82018)
  Rename `batchSize` parameter to `batch_size` to be consisten with the API namings guidelines. (elastic#82123)
  Minor edits in Single Metric Viewer (elastic#82159)
  [Actions] Fix type contract (elastic#82168)
  Upgrade EUI to v30.1.1 (elastic#81499)
  Skip failing ES snapshot test (elastic#82207)
  Skip ES snapshot failing suite (elastic#82206)
  [Alerting UI] Grouped list of alert types using producers in Types filter of Alerts tab (elastic#81876)
  [Maps] convert vector style component to typescript round 1 (elastic#81961)
  Fix link to upgrade assistant (elastic#82138)
  Rename "service overview" to "service inventory" (elastic#81933)
  adjust policy test to drop test for server addresses (elastic#82120)
  Cleanup/codeowners (elastic#82146)
  [DOCS] Updates add data content (elastic#81093)
  [DOCS] Remove index mgmt docs (elastic#82099)
  [Search] fix cancelation related memory leaks (elastic#81996)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:ExpressionLanguage Interpreter expression language (aka canvas pipeline) Feature:Search Querying infrastructure in Kibana release_note:skip Skip the PR/issue when compiling release notes v7.11.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Search] Memory leaks caused by AbortController
4 participants