Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NPM 9 update in 18.14.0 broke npm ci with local dependencies #46542

Closed
MattIPv4 opened this issue Feb 7, 2023 · 12 comments
Closed

NPM 9 update in 18.14.0 broke npm ci with local dependencies #46542

MattIPv4 opened this issue Feb 7, 2023 · 12 comments
Labels
npm Issues and PRs related to the npm client dependency or the npm registry.

Comments

@MattIPv4
Copy link
Member

MattIPv4 commented Feb 7, 2023

Version

18.14.0

Platform

Darwin HMTV-Cowley.local 22.2.0 Darwin Kernel Version 22.2.0: Fri Nov 11 02:08:47 PST 2022; root:xnu-8792.61.2~4/RELEASE_X86_64 x86_64

Subsystem

No response

What steps will reproduce the bug?

Creating package a:

mkdir a
cd a
fnm use 18.13.0 # Or any other tooling to switch Node/NPM version
npm -v # 8.19.3
npm init -y
npm i random
npm ci # Works
fnm use 18.14.0 # Or any other tooling to switch Node/NPM version
npm -v # 9.3.1
npm ci # Works

Creating package b:

mkdir b
cd b
fnm use 18.13.0 # Or any other tooling to switch Node/NPM version
npm -v # 8.19.3
npm init -y
npm i file:../a
npm ci # Works

Triggering npm ci failure:

cd b
fnm use 18.14.0 # Or any other tooling to switch Node/NPM version
npm -v # 9.3.1
npm ci # Fails

How often does it reproduce? Is there a required condition?

Always

What is the expected behavior?

npm ci doesn't fail

What do you see instead?

npm ERR! code EUSAGE
npm ERR! 
npm ERR! `npm ci` can only install packages when your package.json and package-lock.json or npm-shrinkwrap.json are in sync. Please update your lock file with `npm install` before continuing.
npm ERR! 
npm ERR! Missing: random@4.1.0 from lock file
npm ERR! Missing: seedrandom@3.0.5 from lock file
npm ERR! 
npm ERR! Clean install a project
npm ERR! 
npm ERR! Usage:
npm ERR! npm ci
npm ERR! 
npm ERR! Options:
npm ERR! [-S|--save|--no-save|--save-prod|--save-dev|--save-optional|--save-peer|--save-bundle]
npm ERR! [-E|--save-exact] [-g|--global] [--install-strategy <hoisted|nested|shallow>]
npm ERR! [--legacy-bundling] [--global-style]
npm ERR! [--omit <dev|optional|peer> [--omit <dev|optional|peer> ...]]
npm ERR! [--strict-peer-deps] [--no-package-lock] [--foreground-scripts]
npm ERR! [--ignore-scripts] [--no-audit] [--no-bin-links] [--no-fund] [--dry-run]
npm ERR! [-w|--workspace <workspace-name> [-w|--workspace <workspace-name> ...]]
npm ERR! [-ws|--workspaces] [--include-workspace-root] [--no-install-links]
npm ERR! 
npm ERR! aliases: clean-install, ic, install-clean, isntall-clean
npm ERR! 
npm ERR! Run "npm help ci" for more info

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/mattcowley/.npm/_logs/2023-02-07T13_37_40_247Z-debug-0.log

Additional information

No response

@targos
Copy link
Member

targos commented Feb 7, 2023

@nodejs/npm

@richardlau richardlau added the npm Issues and PRs related to the npm client dependency or the npm registry. label Feb 7, 2023
@MattIPv4
Copy link
Member Author

MattIPv4 commented Feb 7, 2023

After doing a bit of digging into the actual changes in NPM 9, it seems like major changes occurred to how local dependencies are installed. They are now actually installed rather than symlinked, which I think it the source of this issue.

If I run npm ci --install-links=false on the final step that fails above, it works without issue.

The 18.14.0 release notes claim that this new install-links behaviour wouldn't impact existing projects with a lock file, but that does not seem to be the case.

Explanation: install-links is the only config or command in the list that has an effect on package installs. We fixed a number of issues that came up during prereleases with this change. It will also only be applied to new package trees created without a package-lock.json file. Any install with an existing lock file will not be changed.

(This seems to impact all local dependencies, not isolated to peer dependencies [that was just how I encountered it]. I've updated the title and initial report to use a standard dependency for reproduction, not a peer dependency)

@MattIPv4 MattIPv4 changed the title NPM 9 update in 18.14.0 broke npm ci with peer dependencies NPM 9 update in 18.14.0 broke npm ci with local dependencies Feb 7, 2023
@MylesBorins
Copy link
Contributor

Thanks for sharing @MattIPv4. I'm surfacing this to the npm CLI team now and will get back to you soon.

nlf added a commit to npm/cli that referenced this issue Feb 7, 2023
this new default was unintentionally breaking so we are reverting it as a bug fix, see nodejs/node#46542
@nlf
Copy link
Contributor

nlf commented Feb 7, 2023

thank you for identifying and raising this concern! it seems we had some miscommunication about how this feature would impact existing lock files. in the interest of keeping as many builds as possible from breaking, we plan to ship a patch release of npm today reverting install-links back to false as a bug fix. we will submit this patch release to node to be shipped in future 18.x releases as well.

you can follow the patch here: npm/cli#6142

lukekarrys pushed a commit to npm/cli that referenced this issue Feb 7, 2023
this new default was unintentionally breaking so we are reverting it as a bug fix, see nodejs/node#46542
@saquibkhan
Copy link
Contributor

#46549

@threema-danilo
Copy link

The change also had a side effect on tools that parse the lockfile to pre-cache all dependencies, in order to allow offline-installation (like flatpak-node-generator, see flatpak/flatpak-builder-tools#308). With Node 18.14, the dev dependencies of local dependencies don't end up in any lockfile anymore, so they cannot be extracted and cached (and npm install --offline fails).

@dardaiin
Copy link

Having the same issue, having a local dependency the dependency gets removed from my package-lock.json file which causes the same issues.

@MylesBorins
Copy link
Contributor

This is fixed with npm 9.4.0 and higher. Node.js 18.14.2 was just released with npm 9.5.0

@TotallyInformation
Copy link

For future note - please don't change this default. Or at least get the word out and provide an equivalent command. There are a lot of us that rely on the way npm install works with local folders for our development workflows. Thanks.

@MylesBorins
Copy link
Contributor

@TotallyInformation fwiw that change impacting projects with an existing package-lock was not intentional. We definitely commit to not breaking npm install in non-major versions of Node.js

@dardaiin
Copy link

Having the same issue, having a local dependency the dependency gets removed from my package-lock.json file which causes the same issues.

Can confirm that everything is working for us with version npm 9.4.0 and higher 🙌

@TotallyInformation
Copy link

TotallyInformation commented Mar 11, 2023

@TotallyInformation fwiw that change impacting projects with an existing package-lock was not intentional. We definitely commit to not breaking npm install in non-major versions of Node.js

I think we would all be rather grateful if you didn't break it in major versions either! 😁

Thanks though, I realise how hard it is to maintain something like npm.

cruessler added a commit to cruessler/exrss that referenced this issue Apr 7, 2023
`npm ci` fails on CI with an error message:

```
npm ERR! code EUSAGE
npm ERR!
npm ERR! `npm ci` can only install packages when your package.json and package-lock.json or npm-shrinkwrap.json are in sync. Please update your lock file with `npm install` before continuing.
npm ERR!
npm ERR! Missing: phoenix@1.7.2 from lock file
npm ERR! Missing: phoenix_html@3.3.1 from lock file
```

nodejs/node#46542
cruessler added a commit to cruessler/exrss that referenced this issue Apr 7, 2023
`npm ci` fails on CI with an error message:

```
npm ERR! code EUSAGE
npm ERR!
npm ERR! `npm ci` can only install packages when your package.json and package-lock.json or npm-shrinkwrap.json are in sync. Please update your lock file with `npm install` before continuing.
npm ERR!
npm ERR! Missing: phoenix@1.7.2 from lock file
npm ERR! Missing: phoenix_html@3.3.1 from lock file
```

nodejs/node#46542
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
npm Issues and PRs related to the npm client dependency or the npm registry.
Projects
None yet
Development

No branches or pull requests

9 participants