Local test failures when running make -j5 test

[gabrielschulhof]

=== release test-benchmark-misc ===                                            
Path: parallel/test-benchmark-misc
assert.js:80
  throw new AssertionError(obj);
  ^

AssertionError [ERR_ASSERTION]: Input A expected to strictly equal input B:
+ expected - actual

- 1
+ 0
    at ChildProcess.child.on (/home/nix/node/node2/test/common/benchmark.js:32:12)
    at ChildProcess.emit (events.js:182:13)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:237:12)
Command: out/Release/node /home/nix/node/node2/test/parallel/test-benchmark-misc.js
=== release test-debugger-debug-brk ===                                        
Path: sequential/test-debugger-debug-brk
assert.js:80
  throw new AssertionError(obj);
  ^

AssertionError [ERR_ASSERTION]: '/home/nix/node/node2/out/Release/node --inspect --debug-brk /home/nix/node/node2/test/fixtures/empty.js' should not quit
    at ChildProcess.fail (/home/nix/node/node2/test/sequential/test-debugger-debug-brk.js:19:29)
    at ChildProcess.emit (events.js:182:13)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:237:12)
Command: out/Release/node /home/nix/node/node2/test/sequential/test-debugger-debug-brk.js

This is master Fedora 26 on X86_64.

[Trott]

Is it a consistent failure or did it fail one time and now it's succeeding?

[Trott]

Also, if it's a consistent failure on the benchmark test, what's the output of this?

./node benchmark/run.js --set concat=0 --set dur=0.1 --set method= --set n=1 --set type= --set val=magyarország.icom.museum misc

[gabrielschulhof]

@Trott I'll check again because I had a stale benchmark/misc/function_call from when function_call was under misc, not napi.

[gabrielschulhof]

Indeed the benchmark test is now OK, but the inspector tests are still not:

=== release test-debugger-debug-brk ===                                        
Path: sequential/test-debugger-debug-brk
assert.js:80
  throw new AssertionError(obj);
  ^

AssertionError [ERR_ASSERTION]: '/home/nix/node/node2/out/Release/node --inspect --debug-brk /home/nix/node/node2/test/fixtures/empty.js' should not quit
    at ChildProcess.fail (/home/nix/node/node2/test/sequential/test-debugger-debug-brk.js:19:29)
    at ChildProcess.emit (events.js:182:13)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:237:12)
Command: out/Release/node /home/nix/node/node2/test/sequential/test-debugger-debug-brk.js
=== release test-inspector-port-cluster ===                                    
Path: sequential/test-inspector-port-cluster
Starting inspector on 127.0.0.1:9229 failed: address already in use
node: ../deps/uv/src/unix/core.c:117: uv_close: Assertion `!uv__is_closing(handle)' failed.
Debugger listening on ws://127.0.0.1:65534/12376357-0bc8-4065-8205-4117d9974f5b
For help, see: https://nodejs.org/en/docs/inspector
Debugger listening on ws://127.0.0.1:12346/8755e4ea-b904-4c05-8810-6275f24a4de4
For help, see: https://nodejs.org/en/docs/inspector
Debugger listening on ws://127.0.0.1:12351/2a8ef499-9843-41b0-9485-992ede9884f2
For help, see: https://nodejs.org/en/docs/inspector
Debugger listening on ws://127.0.0.1:12356/d4e300db-bc7d-4b5d-aa9c-d5de90c55793
For help, see: https://nodejs.org/en/docs/inspector
Debugger listening on ws://0.0.0.0:12361/228ab6d1-437b-4dea-8d08-9242a22a0fde
For help, see: https://nodejs.org/en/docs/inspector
Debugger listening on ws://[::]:12371/9e6e28e2-1475-41ba-ab93-8d1f622640e1
For help, see: https://nodejs.org/en/docs/inspector
Debugger listening on ws://[::1]:12376/39690f17-cbf6-4b17-9a94-7d50d41e16c6
For help, see: https://nodejs.org/en/docs/inspector
Debugger listening on ws://127.0.0.1:12366/8dab6cde-fef9-46a9-8bc9-aaae69810569
For help, see: https://nodejs.org/en/docs/inspector
Debugger listening on ws://127.0.0.1:12381/6dcdc53b-279a-43c3-adf3-f66de6b10450
For help, see: https://nodejs.org/en/docs/inspector
Debugger listening on ws://127.0.0.1:12386/32b23e85-be44-4eb1-a8d7-d51e46318ff0
For help, see: https://nodejs.org/en/docs/inspector
Debugger listening on ws://127.0.0.1:12391/b3a97d18-0bcb-4d20-84c2-b67d1e9da2ca
For help, see: https://nodejs.org/en/docs/inspector
Debugger listening on ws://127.0.0.1:12396/78b21319-0995-4b67-ba34-1e47f4f80951
For help, see: https://nodejs.org/en/docs/inspector
Debugger listening on ws://127.0.0.1:12411/b8cf716e-54da-4c03-b08b-080cb5bfaa7d
For help, see: https://nodejs.org/en/docs/inspector
Debugger listening on ws://127.0.0.1:12416/199082d2-b496-40ad-8a65-b4947d3e5a0f
For help, see: https://nodejs.org/en/docs/inspector
Debugger listening on ws://127.0.0.1:12401/c18629ec-ce65-4ea5-aba5-e6edb4d1ddd6
For help, see: https://nodejs.org/en/docs/inspector
Debugger listening on ws://127.0.0.1:12406/8787dcb9-5854-44d5-a141-e81053203212
For help, see: https://nodejs.org/en/docs/inspector
Debugger listening on ws://127.0.0.1:12426/b442b1b3-c284-4342-98bd-8aa4c59412c6
For help, see: https://nodejs.org/en/docs/inspector
assert.js:80
  throw new AssertionError(obj);
  ^

AssertionError [ERR_ASSERTION]: Input A expected to strictly equal input B:
+ expected - actual

- null
+ 0
    at checkExitCode (/home/nix/node/node2/test/sequential/test-inspector-port-cluster.js:340:10)
    at ChildProcess.childProcess.fork.on.common.mustCall (/home/nix/node/node2/test/sequential/test-inspector-port-cluster.js:333:7)
    at ChildProcess.<anonymous> (/home/nix/node/node2/test/common/index.js:467:15)
    at ChildProcess.emit (events.js:182:13)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:237:12)
Debugger listening on ws://127.0.0.1:12347/c55f32f8-ad2f-4cf0-8058-e4f641f4f38c
For help, see: https://nodejs.org/en/docs/inspector
Debugger listening on ws://127.0.0.1:12348/5ab0b2f8-5ccd-4257-84b8-5b45d9a2f883
For help, see: https://nodejs.org/en/docs/inspector
Debugger listening on ws://127.0.0.1:12349/7f97f900-5272-408c-9ebf-7342c20e98fd
For help, see: https://nodejs.org/en/docs/inspector
Command: out/Release/node /home/nix/node/node2/test/sequential/test-inspector-port-cluster.js

[gabrielschulhof]

I'll run the tests several more times to see if the two remaining failures are also reproducible.

[gabrielschulhof]

1. identical failure
2. identical failure

Seems pretty consistent.

[Trott]

@gabrielschulhof Any chance you have a stale node test process from a prior run hanging around and occupying port 9229, or a debugging session open somewhere doing the same?

[gabrielschulhof]

@Trott after making sure that every instance of node is killed, the tests pass. Gotta keep this mind. Thanks!

[devsnek]

we might want to add better checks about orphaned processes on test end

[Trott]

we might want to add better checks about orphaned processes on test end

It's in the test-ci task but maybe it should be added to the test task.

[gabrielschulhof]

TBH ideally a test run should be orthogonal to any open inspectors. I suspect though that distinguishing a process that was part of a test run from a process that was me doing something else while the tests were running is not easy.