Bug 1290781

Summary: Npm pulls in a lot of dependencies
Product: [Fedora] Fedora Reporter: Franco Bugnano <fri8k>
Component: node-gypAssignee: Tom Hughes <tom>
Status: CLOSED NEXTRELEASE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: low Docs Contact:
Priority: unspecified    
Version: 23CC: db, jamielinux, sgallagh, tchollingsworth, tom, zsvetlik
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-04-04 08:20:58 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Franco Bugnano 2015-12-11 12:52:43 UTC
Description of problem:
When installing npm there are more than 100 dependencies installed, including a C compiler.
It would be nice to reduce those dependencies to the minimum.
Let me explain:
- When installing python-pip, only the python-pip package gets installed, and everything works fine.
- When installing rubygems, only the rubygems package gets installed, and everything works fine.
- When installing npm, more than 100 packages get installed, including a C compiler, kernel headers, and a lot of -devel packages

Why not split npm into 2 packages:
- an npm-lite which installs only npm, and no C compiler dependencies (much like python-pyp and rubygems)
- an npm-full which installs everything and the kitchen sink, like today.

I understand the rationale behind using a C compiler as a dependency for npm, but why is it a dependency for npm and not for pip, for example?
Why do I have to install hundreds of packages if I only want to install js-based npm packages? 

Version-Release number of selected component (if applicable):
Every version of npm since it was introduced in the Fedora repositories

How reproducible:
Always

Steps to Reproduce:
1. Open a terminal 
2. Get root privileges (either by using su -, or by using sudo, it doesn't matter)
3. Issue the command:
dnf install npm

Actual results:
More than 100 packages get installed as dependencies.

Expected results:
A little less packages as hard dependencies.

Additional info:
As I understand that as npm is packaged now, everything works, I suppose that having a minimal npm package would work as well for js-only packages.

Comment 1 Zuzana Svetlikova 2015-12-11 13:18:15 UTC
npm itself has around 50 direct js dependencies, which have another dependencies etc, so consequently you have around 100 nodejs-* dependencies. npm also depends on packages like nodejs, node-gyp and v8, which require another packages like C compiler and *-devel packages.

Comment 2 Stephen Gallagher 2016-01-15 18:29:24 UTC
Realistically, this is not going to change. It doesn't make any sense for Fedora to have an NPM that doesn't match upstream behavior.

Comment 3 Dennis Björklund 2016-01-25 19:45:11 UTC
Still, if we install the binaries from https://nodejs.org/en/download/stable/ we don't need a C compiler and what else.

Why do the fedora installation need all these dependencies when the upstream version does not?

Comment 4 Stephen Gallagher 2016-01-25 20:28:20 UTC
(In reply to Dennis Björklund from comment #3)
> Still, if we install the binaries from
> https://nodejs.org/en/download/stable/ we don't need a C compiler and what
> else.
> 
> Why do the fedora installation need all these dependencies when the upstream
> version does not?

So, the upstream version is basically "cheating". Node's package.json requires that npm includes 'node-gyp', which is a cross-platform tool for building natively-compiled modules. This module is not functional *in any way*, without a compiler on the system, so Fedora's guidelines require that this becomes a strict dependency (so people aren't installing it then immediately filing bugs because it "doesn't work").

npm relies on it because if the user wants to install a native module (or something that depends on a native module somewhere in its dependency chain), it needs to be able to compile it natively on that user's system. Since this is part of a major piece of the npm functionality, we left it in there because (again) we don't want users installing npm, trying to install other modules and hitting what to them will be a bug.

We sometimes refer to this as "the principle of least surprise": what choice can we make so that the user experiences the fewest number of surprises. In this case, we make the assumption that the user expects that when they install 'npm', they will be able to use the npm command for any upstream module (to the best of our ability). This does result in a default case where there are more packages pulled into the installation than some users (such as you, Dennis) are expecting, but the net result is that the package works as expected in hopefully all cases.

Upstream Node.js on the other hand, chose a different path: they choose to bundle a copy of npm with Node.js that behaves differently based on the state of the user's computer. If they have a C/C++ compiler on the system, then npm will be able to install binary packages. If they don't, it will fail in a way that may confuse a non-technical user.

In Fedora, however, we *do* have one other option that we didn't have when npm first landed in the distribution. We now have the ability to specify weak dependencies, not just strict ones. I still think we need to *default* to providing all functionality (thereby maintaining the principle of least surprise), but we *could* modify the node-gyp package such that it only `Recommends: gcc-c++` rather than `Requires: gcc-c++`. The difference here means that one should be able to do `dnf install npm --exclude=gcc-c++` and then you would get the behavior originally requested: an npm on the system that only works for native modules and has none of the compiler chain present.

Does that sound like a reasonable approach?

Comment 5 Tom Hughes 2016-01-25 22:20:25 UTC
Shouldn't the weak dependency be from npm to node-gyp? node-gyp is pretty much useless without a compiler but npm is perfectly useful without node-gyp - it only becomes an issue if you want to install a binary modyle.

Comment 6 Dennis Björklund 2016-01-26 08:40:56 UTC
I wont pretend that I know that much about nodejs and npm but my understanding is that node-gyp is a separate tool that you usually install using npm itself (for the non-fedora users) and that npm work fine without it.

So having node-gyp as a weak dependency from npm sound right to me and in line with the upstream intentions.

How bad is the error message you get from npm if you don't have node-gyp installed and you do something where npm need node-gyp?

I also agree that node-gyp should have a strict dependecy on the compiler since it doesn't work at all without it.

Comment 7 Stephen Gallagher 2016-01-26 13:07:34 UTC
(In reply to Tom Hughes from comment #5)
> Shouldn't the weak dependency be from npm to node-gyp? node-gyp is pretty
> much useless without a compiler but npm is perfectly useful without node-gyp
> - it only becomes an issue if you want to install a binary modyle.

The problem with that is that the node-gyp dependency is handled by package.json and auto-detected. Yes, we can remove and add it as a Recommends: manually, but that will be error-prone if we forget to update the dependency when the requirements change. I suppose it would be more correct, though.

(In reply to Dennis Björklund from comment #6)
> I wont pretend that I know that much about nodejs and npm but my
> understanding is that node-gyp is a separate tool that you usually install
> using npm itself (for the non-fedora users) and that npm work fine without
> it.
> 

You don't install it separately; it's actually present in the binary package as a bundled dependency, it's just shipped non-functional.

> So having node-gyp as a weak dependency from npm sound right to me and in
> line with the upstream intentions.
> 

See above.

> How bad is the error message you get from npm if you don't have node-gyp
> installed and you do something where npm need node-gyp?
> 

That's a good question; I haven't tried it but I suspect that it will essentially look like a compilation error.

> I also agree that node-gyp should have a strict dependecy on the compiler
> since it doesn't work at all without it.

Comment 8 Tom Hughes 2016-01-26 13:09:25 UTC
(In reply to Stephen Gallagher from comment #7)
> (In reply to Tom Hughes from comment #5)
> > Shouldn't the weak dependency be from npm to node-gyp? node-gyp is pretty
> > much useless without a compiler but npm is perfectly useful without node-gyp
> > - it only becomes an issue if you want to install a binary modyle.
> 
> The problem with that is that the node-gyp dependency is handled by
> package.json and auto-detected. Yes, we can remove and add it as a
> Recommends: manually, but that will be error-prone if we forget to update
> the dependency when the requirements change. I suppose it would be more
> correct, though.

Another use for my (currently mythical) fixdep --convert-to-opt switch, combined with handling optionalDependencies in the requirement generator ;-)

Comment 9 Stephen Gallagher 2016-01-26 13:15:07 UTC
(In reply to Tom Hughes from comment #8)
> Another use for my (currently mythical) fixdep --convert-to-opt switch,
> combined with handling optionalDependencies in the requirement generator ;-)

Thanks for volunteering to take this one :)

Comment 10 Tom Hughes 2016-04-04 08:20:58 UTC
This should be fixed in f24 as we are now using the bundled npm.