[turborepo] Task failures not detected

[colbysadams]

What version of Turborepo are you using?

1.10.3

What package manager are you using / does the bug impact?

npm

What operating system are you using?

Mac

Describe the Bug

I replaced the api test case with exit 1 but running npm run test still shows all the tasks as successful

api:test: cache miss, executing dc2014f1938bcca1
...
api:test: 
api:test: > api@0.0.0 test
api:test: > exit 1
api:test: 
api:test: npm ERR! Lifecycle script `test` failed with error: 
api:test: npm ERR! Error: command failed 
api:test: npm ERR!   in workspace: api@0.0.0 
api:test: npm ERR!   at location: /Users/cadams/projects/kitchen-sink-test/apps/api 

 Tasks:    3 successful, 3 total
Cached:    2 cached, 3 total
  Time:    1.078s 

Expected Behavior

turbo should report 2 successful, 1 failure,

To Reproduce

using nodejs version: 20.2

clone https://github.com/colbysadams/kitchen-sink-test

npm i
npm run test

Reproduction Repo

https://github.com/colbysadams/kitchen-sink-test

[mehulkar]

I'm pretty sure you're running into npm/cli#6399, I had tracked this down in #5098 (comment)

[buggedcom]

If anyone arrives here due to the same issue, for me at least it was because the version of npm that ran the command was different that the npm that was running from inside the package scripts.

For example:

ollie@MacBook-Pro turbo-npm-mac-bug % nvm use 20.5.1 && echo "shell:" && which npm && node -v && npm -v && npm run test:turbo -- --force
Now using node v20.5.1 (npm v9.8.0)
shell:
/Users/ollie/.nvm/versions/node/v20.5.1/bin/npm
v20.5.1
9.8.0

> test@1.0.0 test:turbo
> echo "root:" && which npm && npm -v && node -v && turbo run test --force

root:
/Users/ollie/node_modules/.bin/npm
8.15.0
v20.5.1
• Packages in scope: @package/test
• Running test in 1 packages
• Remote caching disabled
@package/test:test: cache bypass, force executing af75f7c17d2b65a3
@package/test:test: 
@package/test:test: > @package/test@1.0.0 test
@package/test:test: > echo "package:" && which npm && npm -v && node -v && exit 1
@package/test:test: 
@package/test:test: package:
@package/test:test: /Users/ollie/node_modules/.bin/npm
@package/test:test: 8.15.0
@package/test:test: v20.5.1
@package/test:test: npm ERR! Lifecycle script `test` failed with error: 
@package/test:test: npm ERR! Error: command failed 
@package/test:test: npm ERR!   in workspace: @package/test@1.0.0 
@package/test:test: npm ERR!   at location: /Users/ollie/Repos/turbo-npm-mac-bug/packages/test 

 Tasks:    1 successful, 1 total
Cached:    0 cached, 1 total
  Time:    427ms 

Notice that the location and versions of npm are different.

Once I uninstalled all of node, npm and nvm, then re-installed just npm it worked as expected.

ollie@MacBook-Pro turbo-npm-mac-bug % nvm use 20.5.1 && echo "shell:" && which npm && node -v && npm -v && npm run test:turbo -- --force
Now using node v20.5.1 (npm v9.8.0)
shell:
/Users/ollie/.nvm/versions/node/v20.5.1/bin/npm
v20.5.1
9.8.0

> test@1.0.0 test:turbo
> echo "root:" && which npm && npm -v && node -v && turbo run test --force

root:
/Users/ollie/.nvm/versions/node/v20.5.1/bin/npm
9.8.0
v20.5.1
• Packages in scope: @package/test
• Running test in 1 packages
• Remote caching disabled
@package/test:test: cache bypass, force executing af75f7c17d2b65a3
@package/test:test: 
@package/test:test: > @package/test@1.0.0 test
@package/test:test: > echo "package:" && which npm && npm -v && node -v && exit 1
@package/test:test: 
@package/test:test: package:
@package/test:test: /Users/ollie/.nvm/versions/node/v20.5.1/bin/npm
@package/test:test: 9.8.0
@package/test:test: v20.5.1
@package/test:test: npm ERR! Lifecycle script `test` failed with error: 
@package/test:test: npm ERR! Error: command failed 
@package/test:test: npm ERR!   in workspace: @package/test@1.0.0 
@package/test:test: npm ERR!   at location: /Users/ollie/Repos/turbo-npm-mac-bug/packages/test 
@package/test:test: ERROR: command finished with error: command (/Users/ollie/Repos/turbo-npm-mac-bug/packages/test) /Users/ollie/.nvm/versions/node/v20.5.1/bin/npm run test exited (1)
@package/test#test: command (/Users/ollie/Repos/turbo-npm-mac-bug/packages/test) /Users/ollie/.nvm/versions/node/v20.5.1/bin/npm run test exited (1)

 Tasks:    0 successful, 1 total
Cached:    0 cached, 1 total
  Time:    413ms 
Failed:    @package/test#test

 ERROR  run failed: command  exited (1)