[BUG] failed optional dependency builds should not prevent global install of packages

[rigor789]

Is there an existing issue for this?

• I have searched the existing issues

This issue exists in the latest npm version

• I am using the latest npm

Current Behavior

With the inclusion of #7126 in v10.4 global installs of packages that have transient dependencies on packages like fsevents started failing.

Example Logs

npm ERR! code 1
npm ERR! path /Users/rigor789/.nvm/versions/node/v21.7.2/lib/node_modules/nativescript/node_modules/fsevents
npm ERR! command failed
npm ERR! command sh -c node-gyp rebuild
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using node-gyp@10.1.0
npm ERR! gyp info using node@21.7.2 | darwin | arm64
npm ERR! gyp info find Python using Python version 3.9.6 found at "/Applications/Xcode-15.2.0.app/Contents/Developer/usr/bin/python3"
npm ERR! gyp info spawn /Applications/Xcode-15.2.0.app/Contents/Developer/usr/bin/python3
npm ERR! gyp info spawn args [
npm ERR! gyp info spawn args '/Users/rigor789/.nvm/versions/node/v21.7.2/lib/node_modules/nativescript/node_modules/node-gyp/gyp/gyp_main.py',
npm ERR! gyp info spawn args 'binding.gyp',
npm ERR! gyp info spawn args '-f',
npm ERR! gyp info spawn args 'make',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args '/Users/rigor789/.nvm/versions/node/v21.7.2/lib/node_modules/nativescript/node_modules/fsevents/build/config.gypi',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args '/Users/rigor789/.nvm/versions/node/v21.7.2/lib/node_modules/nativescript/node_modules/node-gyp/addon.gypi',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args '/Users/rigor789/Library/Caches/node-gyp/21.7.2/include/node/common.gypi',
npm ERR! gyp info spawn args '-Dlibrary=shared_library',
npm ERR! gyp info spawn args '-Dvisibility=default',
npm ERR! gyp info spawn args '-Dnode_root_dir=/Users/rigor789/Library/Caches/node-gyp/21.7.2',
npm ERR! gyp info spawn args '-Dnode_gyp_dir=/Users/rigor789/.nvm/versions/node/v21.7.2/lib/node_modules/nativescript/node_modules/node-gyp',
npm ERR! gyp info spawn args '-Dnode_lib_file=/Users/rigor789/Library/Caches/node-gyp/21.7.2/<(target_arch)/node.lib',
npm ERR! gyp info spawn args '-Dmodule_root_dir=/Users/rigor789/.nvm/versions/node/v21.7.2/lib/node_modules/nativescript/node_modules/fsevents',
npm ERR! gyp info spawn args '-Dnode_engine=v8',
npm ERR! gyp info spawn args '--depth=.',
npm ERR! gyp info spawn args '--no-parallel',
npm ERR! gyp info spawn args '--generator-output',
npm ERR! gyp info spawn args 'build',
npm ERR! gyp info spawn args '-Goutput_dir=.'
npm ERR! gyp info spawn args ]
npm ERR! gyp: binding.gyp not found (cwd: /Users/rigor789/.nvm/versions/node/v21.7.2/lib/node_modules/nativescript/node_modules/fsevents) while trying to load binding.gyp
npm ERR! gyp ERR! configure error
npm ERR! gyp ERR! stack Error: `gyp` failed with exit code: 1
npm ERR! gyp ERR! stack at ChildProcess.<anonymous> (/Users/rigor789/.nvm/versions/node/v21.7.2/lib/node_modules/nativescript/node_modules/node-gyp/lib/configure.js:297:18)
npm ERR! gyp ERR! stack at ChildProcess.emit (node:events:519:28)
npm ERR! gyp ERR! stack at ChildProcess._handle.onexit (node:internal/child_process:294:12)
npm ERR! gyp ERR! System Darwin 23.4.0
npm ERR! gyp ERR! command "/Users/rigor789/.nvm/versions/node/v21.7.2/bin/node" "/Users/rigor789/.nvm/versions/node/v21.7.2/lib/node_modules/nativescript/node_modules/.bin/node-gyp" "rebuild"
npm ERR! gyp ERR! cwd /Users/rigor789/.nvm/versions/node/v21.7.2/lib/node_modules/nativescript/node_modules/fsevents
npm ERR! gyp ERR! node -v v21.7.2
npm ERR! gyp ERR! node-gyp -v v10.1.0
npm ERR! gyp ERR! not ok

npm ERR! A complete log of this run can be found in: /Users/rigor789/.npm/_logs/2024-04-09T09_05_41_833Z-debug-0.log

Expected Behavior

Global installs should not fail. fsevents is an optional peerDependency, and installing it should not trigger a rebuild, nor fail on unsupported systems (like Linux/Windows).

Steps To Reproduce

1. install npm 10.4 or newer
2. run npm i -g nativescript
3. the install will fail

To confirm the linked change breaks it:

1. open node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/build-ideal-tree.js of your global packages
2. edit line 1235 and comment out // fullMetadata: true,
3. re-run: npm i -g nativescript
4. the install works as expected.

Environment

• npm: 10.4.0 (or newer)
• Node.js: v21.7.2
• OS Name: macOS Sonoma 14.4.1
• System Model Name: MacBook Pro
• npm config:

; "user" config from /Users/rigor789/.npmrc

//registry.npmjs.org/:_authToken = (protected)

; node bin location = /Users/rigor789/.nvm/versions/node/v21.7.2/bin/node
; node version = v21.7.2
; npm local prefix = /Users/rigor789/Code/
; npm version = 10.4.0
; cwd = /Users/rigor789/Code/
; HOME = /Users/rigor789
; Run `npm config ls -l` to show all defaults.

[rigor789]

If it's helpful, here are the manifests before/after including fullMetadata: true

Without `fullMetadata: true`

{
  name: 'fsevents',
  version: '2.3.3',
  devDependencies: { 'node-gyp': '^9.4.0' },
  dist: {
    integrity: 'sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==',
    shasum: 'cac6407785d03675a2a5e1a5305c697b347d90d6',
    tarball: 'https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz',
    fileCount: 6,
    unpackedSize: 173224,
    signatures: [ [Object] ]
  },
  engines: { node: '^8.16.0 || ^10.6.0 || >=11.0.0' },
  os: [ 'darwin' ],
  hasInstallScript: true,
  _id: 'fsevents@2.3.3',
  _resolved: 'https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz',
  _from: 'fsevents@~2.3.2',
  _integrity: 'sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==',
  _signatures: [
    {
      keyid: 'SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA',
      sig: 'MEUCIBqKmzRJwFJcSLXc/SdNTdFsUQvsS/tX+pFHr2JkniDbAiEAiTjoxV3W2IK2g8Udbes/ZP46+TtL40PX3IX/bxosfWs='
    }
  ]
}

With `fullMetadata: true`

{
  name: 'fsevents',
  version: '2.3.3',
  description: 'Native Access to MacOS FSEvents',
  main: 'fsevents.js',
  types: 'fsevents.d.ts',
  os: [ 'darwin' ],
  engines: { node: '^8.16.0 || ^10.6.0 || >=11.0.0' },
  scripts: {
    clean: 'node-gyp clean && rm -f fsevents.node',
    build: 'node-gyp clean && rm -f fsevents.node && node-gyp rebuild && node-gyp clean',
    test: '/bin/bash ./test.sh 2>/dev/null',
    prepublishOnly: 'npm run build',
    install: 'node-gyp rebuild'
  },
  repository: { type: 'git', url: 'git+https://github.com/fsevents/fsevents.git' },
  keywords: [ 'fsevents', 'mac' ],
  contributors: [
    { name: 'Philipp Dunkel', email: 'pip@pipobscure.com' },
    { name: 'Ben Noordhuis', email: 'info@bnoordhuis.nl' },
    { name: 'Elan Shankar', email: 'elan.shanker@gmail.com' },
    { name: 'Miroslav Bajtoš', email: 'mbajtoss@gmail.com' },
    { name: 'Paul Miller', url: 'https://paulmillr.com' }
  ],
  bugs: { url: 'https://github.com/fsevents/fsevents/issues' },
  homepage: 'https://github.com/fsevents/fsevents',
  devDependencies: { 'node-gyp': '^9.4.0' },
  gypfile: true,
  gitHead: '2db891e51aa0f2975c5eaaf6aa30f13d720a830a',
  _id: 'fsevents@2.3.3',
  _nodeVersion: '18.17.1',
  _npmVersion: '9.6.7',
  dist: {
    integrity: 'sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==',
    shasum: 'cac6407785d03675a2a5e1a5305c697b347d90d6',
    tarball: 'https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz',
    fileCount: 6,
    unpackedSize: 173224,
    signatures: [ [Object] ]
  },
  _npmUser: { name: 'pipobscure', email: 'pip@pipobscure.com' },
  directories: {},
  maintainers: [
    { name: 'pipobscure', email: 'pip@pipobscure.com' },
    { name: 'paulmillr', email: 'paul@paulmillr.com' }
  ],
  _npmOperationalInternal: {
    host: 's3://npm-registry-packages',
    tmp: 'tmp/fsevents_2.3.3_1692635062642_0.01197687980904627'
  },
  _hasShrinkwrap: false,
  _time: '2023-08-21T16:24:22.854Z',
  _resolved: 'https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz',
  _from: 'fsevents@~2.3.2',
  _integrity: 'sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==',
  _signatures: [
    {
      keyid: 'SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA',
      sig: 'MEUCIBqKmzRJwFJcSLXc/SdNTdFsUQvsS/tX+pFHr2JkniDbAiEAiTjoxV3W2IK2g8Udbes/ZP46+TtL40PX3IX/bxosfWs='
    }
  ]
}

[wraithgar]

This appears to have been working by accident before. The minified packument does not have scripts which likely means that any package that required a build step would not work globally if we reverted that line.

This is still a bug, but the bug is that this should not be failing in global mode. Local mode gracefully catches this:

npm info run fsevents@2.3.3 install node_modules/fsevents node-gyp rebuild
npm info run fsevents@2.3.3 install { code: 1, signal: null }
npm verb reify failed optional dependency /Users/wraithgar/Development/npm/scratch/optional/node_modules/fsevents
npm info run nativescript@8.7.0 postinstall node_modules/nativescript node postinstall.js
npm info run nativescript@8.7.0 postinstall { code: 0, signal: null }

[rigor789]

Thanks for confirming, indeed that sounds more reasonable as an explanation - I wouldn't expect optional dependencies to fail a global install.

I do wonder when that install: 'node-gyp rebuild' is being added however, because it's not in the published package.json of fsevents:

"scripts": {
  "clean": "node-gyp clean && rm -f fsevents.node",
  "build": "node-gyp clean && rm -f fsevents.node && node-gyp rebuild && node-gyp clean",
  "test": "/bin/bash ./test.sh 2>/dev/null",
  "prepublishOnly": "npm run build"
},

(from https://www.npmjs.com/package/fsevents?activeTab=code)

[wraithgar]

It's added automatically by npm during publish because it detects no existing install script but it finds a binding.gyp file. You can see an old issue where folks had similar problems here: #5234

[koiker]

Got the same issue.
The fix was to downgrade npm:
npm i -g npm@~10.3

[restjohn]

This is probably related to #7622.

[Xunnamius]

I'm also running into this when attempting to globally install a package that has bundled dependencies. When the bundleDependencies field is removed (and no dependencies are bundled), attempting to install the package globally works fine. Local installs and installs as dependencies of other locally installed packages also work fine.

My workaround was to install with --ignore-scripts, or potentially --no-optional, but both have downsides.