Bug 2158986

(In reply to postmodern from comment #0)
> Description of problem:
> 
> The stock version of ruby-3.1.3 ships with the net/ftp gem, however Fedora
> 36's package of ruby-3.1.3 is missing this gem.


Fedora certainly provides net-ftp. It is part of ruby-bundled-gems package. The best way to install it is via `dnf install rubygem(net-ftp)`.


> This may cause some rubygems
> to not run properly under Fedora 36's packaged version of ruby.


Do you have any specific use case? How did you install Ruby?

Just out of curiosity, I have checked net-ftp reverse dependencies [1] and found out that almost nothing depends on net-ftp with honorable exception of Chef.


This might be controversial, but in my opinion, the Ruby in Fedora should be minimal in the first place. Whatever additional modules are needed, Fedora provides dependency mechanism to install them. With other Ruby applications, to my knowledge, they are in majority using Bundler and in this scenario, it is also not clear if the system version of net-ftp is going to be used or library from RubyGems.org will be downloaded.

We could probably add `Recommends: ruby-bundled-gems` or specifically `Recommends: rubygem(net-ftp)` somewhere (and please note that `Requires` would be no-go for me), to make it installed by default, but I have not seen any reason for that so far.

Actually, one place where net-ftp is missing and could probably be added is the @Ruby comps group. But then there is probably missing more then just net-ftp (and this in turn is reason against adding the `Recommends: ruby-bundled-gems` somewhere).

I am going to close this as a NOTABUG for the moment (if I was by a chance persuasive enough 😉), but feel free to reopen for more discussion. This also applies to other Ruby maintainers, should you disagree.


[1]: https://rubygems.org/gems/net-ftp/reverse_dependencies

> Fedora certainly provides net-ftp. It is part of ruby-bundled-gems package. The best way to install it is via `dnf install rubygem(net-ftp)`.

rubygems(net-ftp) should be made a dependency of the ruby package so it is installed by default to match the upstream ruby version.

> Do you have any specific use case? How did you install Ruby?

I installed ruby via `sudo dnf install -y ruby-devel`. The lack of the net-ftp gem on Fedora broke one of my projects, Ronin (https://ronin-rb.dev/), which is undergoing beta testing right now. Testing Ronin against upstream ruby-3.1.3 works fine, but attempting to run it under Fedora's Ruby package caused an error message about net/ftp being missing. Please do not deviate from upstream ruby and break other Ruby software, as this creates more work for me the upstream maintainer. While I do also maintain a bash installation script that can workaround this issue, not every user is going to use the installer script and some have already reportedly installed Ronin via `gem install ronin --pre`. I also cannot simply add net-ftp as a dependency of the gem, as it contains C extensions which are not supported under JRuby (and JRuby provides their own implementation of net/ftp).

Under Fedora's ruby-3.1.3:

$ gem install ronin --pre
$ ruby -r ronin/support -e 'p Net::FTP'
ronin/network/ftp requires the net-ftp gem be listed in the Gemfile.
<internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:85:in `require': cannot load such file -- net/ftp (LoadError)

Under upstream ruby-3.1.3:

$ gem install ronin --pre
$ ruby -r ronin/support -e 'p Net::FTP'
Net::FTP

> This might be controversial, but in my opinion, the Ruby in Fedora should be minimal in the first place.

The default ruby package should not deviate from the upstream ruby, so that it does not break other Ruby software which expects a stock ruby environment. If people want a minimal Ruby package, then perhaps a ruby-minimal or ruby-base package should be created instead.

> they are in majority using Bundler and in this scenario

Bundler is primarily used in development or managing the dependencies of Ruby web-apps, not for CLI apps (which Ronin is a CLI app). Ruby CLI apps are typically installed globally either via rubygems, or as a standalone script or tar-ball.

(In reply to postmodern from comment #2)
I think that if and how much Ruby differs and if net-ftp is installed or not is red herring. What you really want is the best user experience for your users. We want to have best experience for Fedora users but also for upstream. So let me take a look at Ronin and provide you some feedback from Fedora developer POV:

1) I don't think that `sudo dnf install -y gcc make ruby-devel readline-devel sqlite-devel libxml2-devel libxslt-devel` is the right suggestion. What I can tell from this is that Ronin probably depends on Sqlite and Nokogiri. So why not install 'rubygem(nokogir)' and 'rubygem-sqlite' instead of installing GCC?

2) The `sudo` should not be recommended with `gem install`, unless this really should be installed system wide.

3) Ideal status would be if GCC is removed from the question.

4) And in the best case, the `gem install` should be removed from the equation and DNF should be used instead.

That is what Fedora users want to use.

So let me start how I would try your SW. I have no idea what it really does, how it works, etc, but I'll try to record my steps. I'll be using Mock [1] to experiment with Ronin and to keep minimal environment:

~~~
$ mock -r fedora-rawhide-x86_64 --scrub all
~~~

It seems that you certainly want to use `gem command`. So let me install it:

~~~
$ mock -r fedora-rawhide-x86_64 --pm-cmd --setopt=install_weak_deps=True install /usr/bin/gem

... snip ...

================================================================================
 Package                  Architecture Version               Repository    Size
================================================================================
Installing:
 rubygems                 noarch       3.4.1-177.fc38        fedora       287 k
Installing dependencies:
 libyaml                  x86_64       0.2.5-8.fc37          fedora        60 k
 ruby                     x86_64       3.2.0-177.fc38        fedora        42 k
 ruby-default-gems        noarch       3.2.0-177.fc38        fedora        43 k
 ruby-libs                x86_64       3.2.0-177.fc38        fedora       4.0 M
 rubygem-io-console       x86_64       0.6.0-177.fc38        fedora        26 k
 rubygem-json             x86_64       2.6.3-201.fc38        fedora        68 k
 rubygem-psych            x86_64       5.0.1-177.fc38        fedora        60 k
 rubypick                 noarch       1.1.1-17.fc37         fedora       9.9 k
Installing weak dependencies:
 rubygem-bigdecimal       x86_64       3.1.3-177.fc38        fedora        69 k
 rubygem-bundler          noarch       2.4.1-177.fc38        fedora       434 k
 rubygem-rdoc             noarch       6.5.0-177.fc38        fedora       489 k

Transaction Summary
================================================================================
Install  12 Packages

... snip ...
~~~

Please note that I could used just `mock -r fedora-rawhide-x86_64 -i /usr/bin/gem`, but Mock by default does not install soft dependencies. Now lets try the installation (no sudo here):

~~~
$ mock -r fedora-rawhide-x86_64 shell --unpriv --enable-network

<mock-chroot> sh-5.2$ gem install ronin --pre

... snip ...

Building native extensions. This could take a while...
ERROR:  Error installing ronin:
	ERROR: Failed to build gem native extension.

    current directory: /builddir/.local/share/gem/ruby/gems/unf-0.2.0.beta2/ext
/usr/bin/ruby mkrf_conf.rb

current directory: /builddir/.local/share/gem/ruby/gems/unf-0.2.0.beta2/ext
rake RUBYARCHDIR\=/builddir/.local/share/gem/ruby/extensions/x86_64-linux/3.2.0/unf-0.2.0.beta2 RUBYLIBDIR\=/builddir/.local/share/gem/ruby/extensions/x86_64-linux/3.2.0/unf-0.2.0.beta2
rake failedNo such file or directory - rake

Gem files will remain installed in /builddir/.local/share/gem/ruby/gems/unf-0.2.0.beta2 for inspection.
Results logged to /builddir/.local/share/gem/ruby/extensions/x86_64-linux/3.2.0/unf-0.2.0.beta2/gem_make.out
~~~

Hm, `unf` build failure. We have `unf` available in Fedora:

~~~
$ $ mock -r fedora-rawhide-x86_64 --pm-cmd --setopt=install_weak_deps=True install rubygem-unf
~~~

Lets try again:

~~~
	ERROR: Failed to build gem native extension.

    current directory: /builddir/.local/share/gem/ruby/gems/sqlite-1.0.2/ext/sqlite
/usr/bin/ruby -I /usr/share/rubygems extconf.rb
mkmf.rb can't find header files for ruby at /usr/share/include/ruby.h
~~~

Again we have Sqlite3 in Fedora. But after installation and on the second look, it installs sqlite 1.0.2, what is that? It seems to be this [2] library. But this makes me wonder why. Anyway, ok, we need to compile it. Lets install ruby-devel, sqlite-devel and gcc, otherwise we are facing various issues. And now the installation is done:

~~~
... snip ...

*************************************************************************

  Thank you for installing Ronin!

  To list the available commands:

      $ ronin help

  To view help information about a command:

      $ ronin help COMMAND

*************************************************************************
Successfully installed ronin-2.0.0.beta2

... snip ...

17 gems installed

$ ~/bin/ronin help

... snip ...

Additional Ronin Commands:
    $ ronin-repos
    $ ronin-db
    $ ronin-web
    $ ronin-fuzzer
    $ ronin-payloads
    $ ronin-exploits
    $ ronin-vulns
~~~

And I can reproduce the issue:

~~~
$ ruby -r ronin/support -e 'p Net::FTP'
ronin/network/ftp requires the net-ftp gem be listed in the Gemfile.
<internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:85:in `require': cannot load such file -- net/ftp (LoadError)
	from <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from /builddir/.local/share/gem/ruby/gems/ronin-support-1.0.0.beta2/lib/ronin/support/network/ftp/mixin.rb:22:in `<top (required)>'
	from <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from /builddir/.local/share/gem/ruby/gems/ronin-support-1.0.0.beta2/lib/ronin/support/network/mixin.rb:26:in `<top (required)>'
	from <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from /builddir/.local/share/gem/ruby/gems/ronin-support-1.0.0.beta2/lib/ronin/support/mixin.rb:22:in `<top (required)>'
	from <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from /builddir/.local/share/gem/ruby/gems/ronin-support-1.0.0.beta2/lib/ronin/support/core_ext/kernel.rb:19:in `<top (required)>'
	from <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from /builddir/.local/share/gem/ruby/gems/ronin-support-1.0.0.beta2/lib/ronin/support/core_ext.rb:24:in `<top (required)>'
	from <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from /builddir/.local/share/gem/ruby/gems/ronin-support-1.0.0.beta2/lib/ronin/support.rb:21:in `<top (required)>'
	from <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:160:in `require'
	from <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:160:in `rescue in require'
	from <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:149:in `require'
<internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:85:in `require': cannot load such file -- ronin/support (LoadError)
	from <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:85:in `require'
~~~

The fix is easy:

~~~
$ mock -r fedora-rawhide-x86_64 -i 'rubygem(net-ftp)'

... snip ...

<mock-chroot> sh-5.2$ ruby -r ronin/support -e 'p Net::FTP'
Net::FTP
~~~

Now I am a bit smarter. Lets star a fresh:

~~~
$ mock -r fedora-rawhide-x86_64 --init

$ mock -r fedora-rawhide-x86_64 -i rubygem-nokogiri 'rubygem(net-ftp)' rubygem-mechanize gcc ruby-devel sqlite-devel

<mock-chroot> sh-5.2$ gem install ronin --pre

... snip ...

<mock-chroot> sh-5.2$ ruby -r ronin/support -e 'p Net::FTP'
Net::FTP
~~~

And now several takeaways:

1) The installation command suggested for Fedora [3] is not precise. The libxml2-devel and libxslt-devel are not needed, because the precompiled version of Nokogiri is used. If Nokogiri should be build during installation, the installation command would need to include `--platform ruby`. Also `make` is not required to be listed explicitly, because it is pulled in via gcc.

2) If you can suggest various dependencies, I don't see issue specifying that the `dnf install` needs to include `'rubygem(net-ftp)'`. BTW, there is also missing Rake in the dependencies, because the original UNF build issue was `rake failedNo such file or directory - rake`.

3) The rubygem-timers is also available in Fedora, but it would need update (bug 1253999), because more recent version is required for some reason.

4) I still don't understand, why are you reluctant to fully specify your dependencies? Do I understand the you said that `net/ftp` is not properly supported by JRuby? Why is that? Is is reported and is that going to be fixed?

5) Would you be satisfied, if `dnf install @Ruby` installed everything what is part of upstream package?

Anyway, this IMHO just demonstrates that there are more then couple of lines one can draw, where you can use Fedora or go your way. The point is the I totally see your view and I wish I could provide you with more acceptable answer. However I can't do this change without sacrificing something else. I can't even recommend you "Package roning for Fedora into RPM and everything will be fine and dandy", because this solution has its own drawbacks.


P.S. The above was tested in Fedora Rawhide and the results might differ on older Fedoras.


[1] https://github.com/rpm-software-management/mock/
[2] https://github.com/mikeowens/SQLiteRuby
[3] https://ronin-rb.dev/install/fedora/

(In reply to Vít Ondruch from comment #3)
> But after installation and on the second
> look, it installs sqlite 1.0.2, what is that? It seems to be this [2]
> library. But this makes me wonder why.

There is also this bit of information:

~~~
First we will need to install gcc and make. These are required to install gems which contain C extensions (aka C bindings) to other C libraries, such as the sqlite3 gem which compiles against libsqlite3.
~~~

Which is incorrect, because this is not sqlite3 gem.

> 1) I don't think that `sudo dnf install -y gcc make ruby-devel readline-devel sqlite-devel libxml2-devel libxslt-devel` is the right suggestion. What I can tell from this is that Ronin probably depends on Sqlite and Nokogiri. So why not install 'rubygem(nokogir)' and 'rubygem-sqlite' instead of installing GCC?

Nokogiri does contain pre-compiled C extensions and their own copy of libxml2/libxslt, however I do believe it can switch back to compile against the system's copy of libxml2/libxslt. I will have to test this.

> 2) The `sudo` should not be recommended with `gem install`, unless this really should be installed system wide.

Ronin is supposed to be installed system wide and available to all users.

> 3) Ideal status would be if GCC is removed from the question.

There are many other gems which contain C extensions that the user might end up installing, and have not yet been packaged by Fedora (ex: digest-crc). Also, the user may need to use gcc to compile other C code needed for payloads or other exploits, so having a C compiler installed may be useful.

> 4) And in the best case, the `gem install` should be removed from the equation and DNF should be used instead.

Ronin and many of it's dependencies are not available via Fedora's package repository, so installing everything via `dnf` is not viable. Installing rubygems via `gem install` is normal for Ruby users, even Ruby users who use Fedora. Maybe in the future when I can convert all of the gems to RPMs, and ensure they are always up-to-date when new gem versions are released, I might be able to install all of Ronin via COPR, but I simply do not have the time or resources to accomplish that right now. Would you be interested in volunteering for such an effort? ;)

> Which is incorrect, because this is not sqlite3 gem.

Good catch! ronin-db should depend on the sqlite3 gem (which is used by activerecord), not the sqlite gem.

I appreciate the feedback, but I feel like we are getting off topic here. The topic of the bug was that Fedora's ruby package does not provide net/ftp by default, due to the missing dependency on ruby-bundled-gems, but upstream ruby does provide by default, therefor other Ruby software which requires net/ftp will break when ran on Fedora's ruby. While I appreciate the feedback on Ronin, it derails the discussion away from Fedora's ruby package and to debugging Ronin, which is not the topic of this bug report.

The fact that Fedora's ruby or ruby-devel packages do not automatically install the ruby-bundled-gems package as a dependency is still a problem for Ronin. While I have gone to great lengths to both provide an automated installation script as well as explicit manual installation instructions, and even _if_ Ronin could be installed entirely via COPR, not all users will use those methods and some may attempt to install it themselves by running `sudo dnf install -y ruby` and `gem install ronin --pre`; one user in Discord said they did just that. Therefor, I am asking you politely to add ruby-bundled-gems to the ruby package as a dependency. This should not break or diminish the current user experience for Fedora Ruby users, but instead it will prevent other Ruby software which requires net/ftp from breaking. That is, imho, an improvement.

(In reply to postmodern from comment #5)
> I appreciate the feedback, but I feel like we are getting off topic here.

Sorry, I did not want to derail the discussion. I have just tried to illustrate, that we have to do choices for Fedora as well as demonstrate that you (representing upstream) make choices (e.g. we choose to not install net/ftp by default where you choose to not specify the dependency).

The Ruby in Fedora is tailored towards use of other RPM packages. We have to inevitably differ from upstream.

> not all users will
> use those methods and some may attempt to install it themselves by running
> `sudo dnf install -y ruby` and `gem install ronin --pre`; one user in
> Discord said they did just that. Therefor, I am asking you politely to add
> ruby-bundled-gems to the ruby package as a dependency. This should not break
> or diminish the current user experience for Fedora Ruby users, but instead
> it will prevent other Ruby software which requires net/ftp from breaking.

I agree, that this would help, but:

1) It would increase disk usage for majority of Fedora Ruby users without them ever needing net/ftp.
2) I cannot consider net/ftp without considering Rake, RDoc, Minitest, Test-Unit, IRB and possibly others. All these are not installed by default while upstream Ruby does that. I cannot consider this without even thinking about installing ruby-devel by default. This is not acceptable.

I am sorry, but we have to agree to disagree.

How about creating a ruby-full meta package, similar to Ubuntu/Debian's ruby-full meta package, which simply pulls in all of the various ruby dependency packages in order to create a stock/standard ruby environment? This way, if someone using Fedora needs to run an arbitrary Ruby script or project, which was not packaged as an rpm with proper rpm dependencies, they could install ruby-full and ensure everything is installed that a ruby script or project might possibly require from a stock/standard ruby install. This would at least give that group of users an option, which would also prevent them from submitting additional bug reports about how 'net/ftp', 'net/imap', 'net/pop', 'net/smtp', etc, are missing from Fedora's 'ruby' package. If there is not some sort of easily installable stock/standard ruby package (or meta package) available in Fedora, then it might end up encouraging upstream Ruby project maintainers to instead recommend users install a Ruby Version Manager just to install a stock/standard ruby environment (not ideal imho due to lack of automated security updates, which you get for free from a package manager); this certainly occurred in the past due to macOS's outdated ruby installation and due to past issues with Ubuntu's ruby package before they added the ruby-full meta package.

(In reply to postmodern from comment #7)
> How about creating a ruby-full meta package

Meta packages were always discouraged in Fedora AFAIR. We have comps groups [1, 2]. You can install them via `dnf install @Ruby` or `dnf install @ruby`. I have mentioned it at least twice, but maybe I have failed to convey the message clearly or you might not be familiar with the concept.


[1] https://fedoraproject.org/wiki/How_to_use_and_edit_comps.xml_for_package_groups
[2] https://pagure.io/fedora-comps/blob/main/f/comps-f38.xml.in#_4615-4626