Bug 1565256 - NPM permanently breaks after "npm update -g"
Summary: NPM permanently breaks after "npm update -g"
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: nodejs
Version: 31
Hardware: All
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: NodeJS Packaging SIG
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-04-09 18:32 UTC by nik
Modified: 2020-01-05 00:07 UTC (History)
10 users (show)

Fixed In Version: nodejs-12.13.1-1.fc31 nodejs-12-3020191106195518.a5b0195c nodejs-12-3120191202171146.f636be4b
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-12-06 05:44:07 UTC
Type: Bug


Attachments (Terms of Use)

Description nik 2018-04-09 18:32:41 UTC
Description of problem:
Running "npm update -g" on a fresh Fedora box, npm will complete the update successfully but will break permanently upon future invocations because all its node_modules will be gone. The only way to fix it is to reinstall the package from scratch back to 5.6.

Version-Release number of selected component (if applicable):
Affects npm 5.6 on Fedora 27 & 28

How reproducible:
Consistent

Steps to Reproduce:
1. Start with a fresh Fedora: docker run -it fedora:28 /bin/bash
2. Install npm: dnf install npm
3. Update npm: npm update -g
4. Run npm again, see error

Actual results:
module.js:549
    throw err;
    ^

Error: Cannot find module 'npmlog'
    at Function.Module._resolveFilename (module.js:547:15)
    at Function.Module._load (module.js:474:25)
    ....

Expected results:
npm successfully upgraded from npm 5.6 to 5.8 (at the time of writing)


Additional info:

The bug is caused by the way Fedora packages npm.

We can see in https://src.fedoraproject.org/rpms/nodejs/blob/cc0d5eb2e643873f8dc448769370c274be00de1d/f/nodejs.spec#_370 that the node_modules that come with npm are actually in /lib/node_modules/npm/node_modules.bundled and are symlinked from /lib/node_modules/npm/node_modules.

npm wants to upgrade packages independent of their order, so it has strange logic that can remove a package while keeping its node_modules, this way the node_modules can be dealt with separately. The code that does it is here: https://github.com/npm/npm/blob/ee147fbbca6f2707d3b16f4fa78f4c4606b2d9b1/lib/install/action/remove.js#L13

When npm tries to upgrade itself it first removes itself, (which means that it deletes everything in /usr/lib/node_modules/npm except node_modules) and then installs the new version in. As part of this action node_modules.bundled gets removed as well and all symlinks in /lib/node_modules/npm/node_modules break. 

After this npm is broken forever...

We probably need another way to package npm, because putting node_modules.bundled near the real node_modules breaks upgrade. Maybe we should put it in node_modules/.bundled?

Comment 1 Tom Hughes 2018-04-09 18:45:15 UTC
As a general rule you should never use npm --global with the Fedora packaged npm/node as it will overwrite things in /usr/lib/node_modules and cause general chaos.

As I understand it upstream regard --global as a misfeature that shouldn't be used and also don't support putting user installed global modules in a different location to system installed global modules like some other languages.

Comment 2 Stephen Gallagher 2018-04-09 18:58:59 UTC
The reason behind node_modules.bundled in our package was to provide us an escape-hatch if we ever decided to split out the npm subpackage from the nodejs SRPM again. It's a preventative workaround for https://fedoraproject.org/wiki/Packaging:Directory_Replacement

That said, we've been largely successful with just using the version bundled by Node.js's tarball and I don't really see us wanting to go through the hassle of unbundling it again (especially since its dependency chain has been growing constantly). So we *could* just bite the bullet and decide that this is a permanent state of being.

Options: 
1) We could opt to move the node_modules.bundled folder to somewhere that it wouldn't get clobbered. If we did this, I think /usr/lib/node/.bundled/npm is probably the right place, as its out of the path of the deletion and still sufficiently unique that it can be managed easily.

2) We can accept that we'll never try to unbundle npm again and just drop the symlink emergency hatch entirely.


That all being said, Tom is right: using `npm --global` is a recipe for disaster on an RPM-managed system (its changes WILL get overwritten by the next RPM update). It's far better to make per-user changes. But people are people and if they decide to do this (which is recommended by NPM itself, unfortunately...), I think we need to try really hard not to break on it.

Tom, which approach would you prefer?

Comment 3 Tom Hughes 2018-04-09 20:18:47 UTC
I'm not sure I have any particular preferences...

Another option of course is to patch npm to disable global operations ;-)

We should probably at least patch it to stop it suggesting that people update npm that way.

Comment 4 Stephen Gallagher 2018-04-09 21:07:34 UTC
(In reply to Tom Hughes from comment #3)
> I'm not sure I have any particular preferences...
> 
> Another option of course is to patch npm to disable global operations ;-)
> 
> We should probably at least patch it to stop it suggesting that people
> update npm that way.

I've got it patched to disable the update message now. I think I'm going to go with option 1) for the time being, simply because it gives us more flexibility to change our minds later.

Comment 5 nik 2018-04-09 21:21:42 UTC
I'm not sure if Tom was joking, but I think patching npm to disable global operations or at least print a huge red warning is actually a good idea. The average user (me!) doesn't know that "using `npm --global` is a recipe for disaster on an RPM-managed system (its changes WILL get overwritten by the next RPM update)".

I would have really appreciated a message like "You installed npm through the package manger and global operations might break you in strange ways. Please install and update global npm packages through the package manager". Especially since a lot of places on the internet recommend installing global packages.

Comment 6 Ben Cotton 2019-05-02 21:17:06 UTC
This message is a reminder that Fedora 28 is nearing its end of life.
On 2019-May-28 Fedora will stop maintaining and issuing updates for
Fedora 28. It is Fedora's policy to close all bug reports from releases
that are no longer maintained. At that time this bug will be closed as
EOL if it remains open with a Fedora 'version' of '28'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 28 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 7 Ben Cotton 2019-05-28 21:53:57 UTC
Fedora 28 changed to end-of-life (EOL) status on 2019-05-28. Fedora 28 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.

Comment 8 Stephen Gallagher 2019-10-01 14:55:56 UTC
The underlying problem is still an issue in F31. Sorry I didn't notice the auto-close.

Comment 9 Fedora Update System 2019-12-02 20:31:14 UTC
FEDORA-2019-7443ebda4b has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2019-7443ebda4b

Comment 10 Fedora Update System 2019-12-02 20:31:14 UTC
FEDORA-2019-1686ae9b59 has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2019-1686ae9b59

Comment 11 Fedora Update System 2019-12-02 20:31:15 UTC
FEDORA-MODULAR-2019-72885eafd5 has been submitted as an update to Fedora 30 Modular. https://bodhi.fedoraproject.org/updates/FEDORA-MODULAR-2019-72885eafd5

Comment 12 Stephen Gallagher 2019-12-02 20:32:24 UTC
As of 12.13.0, `npm -g` will now install global Node.js packages into /usr/local instead of /usr, which should ensure that they don't conflict with the RPM-provided ones.

Comment 13 Fedora Update System 2019-12-03 00:54:02 UTC
libuv-1.33.1-1.fc31, nodejs-12.13.1-1.fc31 has been pushed to the Fedora 31 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-7443ebda4b

Comment 14 Fedora Update System 2019-12-03 01:28:21 UTC
nodejs-12-3020191106195518.a5b0195c has been pushed to the Fedora 30 Modular testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-MODULAR-2019-72885eafd5

Comment 15 Fedora Update System 2019-12-03 01:33:51 UTC
libuv-1.33.1-1.fc30 has been pushed to the Fedora 30 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-1686ae9b59

Comment 16 Fedora Update System 2019-12-03 03:01:30 UTC
libuv-1.33.1-1.el7 has been pushed to the Fedora EPEL 7 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2019-b48135c789

Comment 17 Fedora Update System 2019-12-03 03:03:35 UTC
nodejs-12-3120191202171146.f636be4b has been pushed to the Fedora 31 Modular testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-MODULAR-2019-4a18b47487

Comment 18 Fedora Update System 2019-12-06 05:44:07 UTC
libuv-1.33.1-1.fc31, nodejs-12.13.1-1.fc31 has been pushed to the Fedora 31 stable repository. If problems still persist, please make note of it in this bug report.

Comment 19 Fedora Update System 2019-12-10 03:04:22 UTC
libuv-1.33.1-1.fc30 has been pushed to the Fedora 30 stable repository. If problems still persist, please make note of it in this bug report.

Comment 20 Fedora Update System 2019-12-10 17:59:23 UTC
nodejs-12-3020191106195518.a5b0195c has been pushed to the Fedora 30 Modular stable repository. If problems still persist, please make note of it in this bug report.

Comment 21 Fedora Update System 2019-12-11 17:24:51 UTC
nodejs-12-3120191202171146.f636be4b has been pushed to the Fedora 31 Modular stable repository. If problems still persist, please make note of it in this bug report.

Comment 22 Fedora Update System 2019-12-12 13:38:23 UTC
FEDORA-EPEL-2019-39eb4afe6e has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2019-39eb4afe6e

Comment 23 Fedora Update System 2019-12-13 02:01:42 UTC
libuv-1.34.0-1.el7 has been pushed to the Fedora EPEL 7 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2019-39eb4afe6e

Comment 24 Fedora Update System 2020-01-05 00:07:28 UTC
libuv-1.34.0-1.el7 has been pushed to the Fedora EPEL 7 stable repository. If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.