Bug 975660 - ruby-devel installs header files in the wrong location
Summary: ruby-devel installs header files in the wrong location
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: ruby
Version: 20
Hardware: All
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: Jeroen van Meeuwen
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-06-19 05:15 UTC by Akinori MUSHA
Modified: 2015-06-30 00:39 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-06-30 00:39:39 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Akinori MUSHA 2013-06-19 05:15:29 UTC
Description of problem:

The ruby-devel package, starting from 1.9.x, installs header files directly under /usr/include, but they are not supposed to be there.  They should instead be placed in a version specific directory, just as with the default configuration.

Each installation of ruby has its own set of header files that are made visible only when building a module for the specific installation of ruby, the one used for running extconf.rb.

The problem is, not a few ruby extension libraries configure themselves using have_header("ruby/***.h") in their extconf.rb to check if the ruby to use supports a specific feature or internal framework, but placing ruby header files in the universal location (/usr/include) breaks those tests, leading to wrong configuration and build errors.

Version-Release number of selected component (if applicable):

ruby-devel 1.9.3-* and 2.0.0-*

How reproducible:


Steps to Reproduce:
1. Install ruby-devel 1.9.3 or later.
2. Install ruby 1.8.7 manually.
3. Try to build and install any of the following libraries with the ruby installed in step 2.

- cairo
- json_pure
- passenger
- rmagic
- unf_ext

Actual results:

extconf.rb picks ruby/st.h, ruby/re.h, ruby/encoding.h or ruby/io.h from /usr/include even if this is ruby 1.8.7, and then compiler errors occur due to macro/type discrepancy.

Expected results:

Ruby 1.8.7 should not locate the header files of the wrong version in the first place.

Additional info:

This is not a problem limited to ruby 1.8.7 vs. 1.9+.
It will recur whenever ruby adds a new header file after ruby 2.0.

Comment 1 Vít Ondruch 2013-06-19 06:50:06 UTC
Could you please provide more details, such as where is your ruby 1.8.7 installed, your Makefiles generated by extconf.rb etc?

Comment 2 Akinori MUSHA 2013-06-19 12:33:50 UTC
It's independent from where ruby 1.8.7 is installed, because
/usr/include is a default header search path.

First of all, I believe installing ruby's header files right under
/usr/include gives you no gain but only harm.  They are only useful
and successfully compilable when appropriate compiler options are
given through a Makefile created by create_makefile() of mkmf.

Anyway, here's how to reproduce the problem, step-by-step.

$ yum install -y ruby-devel openssl-devel compat-readline5-devel patch
(snip)

$ curl ftp://ftp.ruby-lang.org/pub/ruby/ruby-1.8.7-p358.tar.bz2 | tar jxf -
(snip)

$ cd ruby-1.8.7-p358

# This patch is necessary to avoid build failure on Fedora.
# See: http://bugs.ruby-lang.org/issues/5108
$ curl http://bugs.ruby-lang.org/attachments/download/1931/stdout-rouge-fix.patch | patch -p1

$ ./configure --prefix=/tmp/foo && make && make install
(snip)

$ /tmp/foo/bin/ruby -v
ruby 1.8.7 (2012-02-08 patchlevel 358) [x86_64-linux]

# create an example extconf.rb
$ cat > extconf.rb
require 'mkmf'
have_header('ruby/encoding.h')
create_makefile('foo')
^D

$ /tmp/foo/bin/ruby extconf.rb
checking for ruby/encoding.h... yes
creating Makefile

$ grep CPPFLAGS Makefile
CPPFLAGS = -DHAVE_RUBY_ENCODING_H  $(DEFS) $(cppflags)

We are using ruby 1.8.7, but have_header() has located a header
file of ruby 1.9.3 installed under /usr/include.
If this extension had lines that goes like below, build will fail:

#ifdef HAVE_RUBY_IO_H
#include "ruby/io.h"
#endif


In fact, calling have_header("ruby/***.h") is unnecessary because HAVE_RUBY_***_H is predefined by ruby.h starting from 1.9.2.  But
that's another story; there still are extensions that do it anyway.
(I've fixed unf_ext and json_pure seems to have it fixed in the repo)

Comment 3 Vít Ondruch 2013-06-25 15:24:23 UTC
Thanks for detailed steps. Based on them, I did some testing and bit of research and although I acknowledge that you might encounter this issue, I am afraid I have to reject it. There are several reasons:

1) We support on Fedora only single system version of Ruby, therefore the version is pointless.

2) Even if we support multiple versions of Ruby, we would not be able to support multiple versions of 1.8 or 1.9.x branches. The version string in that case would not be helpful anyway, since the had the same ABI, therefor it would conflict.

3) Although Ruby provides backward compatibility, it does not provide forward compatibility. In your extconf.rb, you are asking Ruby 1.8.7 for ruby/encoding.h file, which was not shipped with Ruby 1.8.7, so it finds what it finds. It is just by coincidence, that you have this header provided by other version of Ruby.

4) Program, which is using Ruby, such as:

#include "ruby.h"

int main(int argc, char **argv)
{
  return 0;
}

should be compilable without any special configuration script, such as mkmf (it actually does not work, due to arch specific ruby/config.h, but is should).

5) It seems strange to check some Ruby properties by checking for its headers. The have_header should be used for checking for headers of other libraries, IMO.

Comment 4 Akinori MUSHA 2013-06-26 01:08:53 UTC
1) It's your call.

2) (skip)

3) I'm reporting that the Fedora rpm installs header files in the system directory purposely altering our (ruby's) default configuration.  How should I call this a coincidence?  I don't know of any other platform like that.

4) No.  People should not write such programs, and you shouldn't encourage that.  It's non-portable and far from supported.

5) It is very well to say that, but as I said there is existing breakage out there, and it wouldn't occur unless the rpm did not install header files in a non-default location.

Comment 5 Vít Ondruch 2013-06-26 14:37:37 UTC
I apologize, but could you please clarify your role in this report? Are you speaking as a Fedora user, who needs to solve his issue or as a Ruby commiter, who is not satisfied how Fedora is configured? 

For the former, Ruby on Fedora is configured like this for almost 1.5 years (precisely since 2012-01-18) and I am not aware of any unsolvable issues which would suggest, that it was wrong decision. Nevertheless, this does not mean there are no corner cases, as the one you pointed out. I apologize for any inconvenience it caused to you, but from my POV, it is not reason to change Ruby configuration of Fedora. I just can suggest you to make the configuration scripts of above referenced libraries more robust. If there is anything I can help in this direction I'll gladly do so.

And as of the latter, I'd like to use this opportunity to encourage you to pay better attention to your downstreams, such as Fedora. When we introduced this configuration, I tried repeatedly to get upstream feedback. However, there was almost no response [1] or denial without any explanation [2]. Surprisingly, the first patches [1] were proposed almost 2 years ago, 5 months prior they landed in Fedora. That was the right period to discuss this configuration.



[1] https://bugs.ruby-lang.org/issues/5231
[2] https://bugs.ruby-lang.org/issues/7807

Comment 6 Akinori MUSHA 2013-06-26 16:36:47 UTC
I apologize if I offended you.  I reported this issue as part of
a reaction to a seemingly suspicious bug report with my unf_ext
gem:

    https://github.com/knu/ruby-unf_ext/issues/5

As you can see from the log, I first thought the error was due to
the reporter's personal misconfiguration, but later I found out
that there was an rpm that installs /usr/include/ruby/encoding.h,
which I had no idea why, and thought it should be fixed.

Then, this issue was immediately closed.  I kind of decided to
put my committer hat on (by using the word "our") because I felt
that the reasons given did not explain why this rpm had to change
the default configuration despite my report that there was
existing problems only occurring on Fedora/RHEL due to its
unique configuration, which was against how ruby was supposed to
be installed.  As one of the committers who have been working on
ruby's build/installation process, I thought, "This looks weird
and simply wrong.  Or if Fedora really needs to configure ruby
that way, ruby may need some change to support its local
situation."

I'm sorry I didn't respond to the issues you submitted on
bugs.ruby-lang.org, but even after reading them again, I still
don't get what was the problem you were trying to solve.  Your
reports had only patches without explaining enough as to why.

As I told in the previous comment, any program that includes
ruby's header files and calls the API wouldn't be able to build
without any "special configuration script" because extra compiler
flags and/or extra linker flags (in addition to just -lruby)
might be necessary, which could never be deduced from outside.
That's why ruby has long had mkmf.rb, and added a pkg-config file
in 1.9.3. (1.9.3 was released a couple of months before
2012-01-18)

So, back to my question: Why versioned paths need to be disabled?
I checked openSUSE and Debian, and their packages had versioned
paths for header files.

Comment 7 Vít Ondruch 2013-06-27 06:40:10 UTC
(In reply to Akinori MUSHA from comment #6)
> I apologize if I offended you.

No offense taken ;)

> I reported this issue as part of
> a reaction to a seemingly suspicious bug report with my unf_ext
> gem:
> 
>     https://github.com/knu/ruby-unf_ext/issues/5
> 
> As you can see from the log, I first thought the error was due to
> the reporter's personal misconfiguration, but later I found out
> that there was an rpm that installs /usr/include/ruby/encoding.h,
> which I had no idea why, and thought it should be fixed.

Thanks for explanation. Now we have full story at least.

> I'm sorry I didn't respond to the issues you submitted on
> bugs.ruby-lang.org, but even after reading them again, I still
> don't get what was the problem you were trying to solve.  Your
> reports had only patches without explaining enough as to why.

Although I would always like to explain every reason why I opening bug, if I do so, it may help on one hand to better understand, on the other hand, it tends to distract and get different solution for different problem. So it is always question how many information to disclose. We are both guilty in this point, me in the issues I referenced and you in this one (sorry, no offense ;)

> As I told in the previous comment, any program that includes
> ruby's header files and calls the API wouldn't be able to build
> without any "special configuration script" because extra compiler
> flags and/or extra linker flags (in addition to just -lruby)
> might be necessary, which could never be deduced from outside.
> That's why ruby has long had mkmf.rb, and added a pkg-config file
> in 1.9.3. (1.9.3 was released a couple of months before
> 2012-01-18)

You are right. I would say we both exaggerate to some extent in our claims to support our case, which both are valid, although may be a bit contradicting. So to say, mkmf.rb and pkg-config are both valuable tools and they do a great job, but they have no place in my "dream world", because they just add unneeded complexity (yes, it is unreachable, but I'd like to get as close as possible).

> So, back to my question: Why versioned paths need to be disabled?
> I checked openSUSE and Debian, and their packages had versioned
> paths for header files.

That is true and I'll make several notes on this topic:

* What openSUSE or Debian does is not binding for us. It even does not say nothing about correctness. I would say that they just take what upstream provides and they packaged it as it is - the path of the lowest resistance (don't want to offend anybody, since I don't know any reason for their decisions).

* I don't know what is the reason to have the ABI version number anywhere, but it just complicates everything and confuses everybody and it serves for no purpose what so ever. I really love to see some reasoning why it should be there, although it does not allows you to install Ruby 1.8.5, 1.8.6 and 1.8.7 side by side, the same applies for Ruby 1.9.1, 1.9.2 and 1.9.3 and I think there were times, where a lot of developers would welcome this to be possible.

* I cannot remember, how many times I saw the bugreports about 1.9.1 version used in 1.9.3 paths. That is just confusing.

* There are also other platforms, such as RubyInstaller for Windows (there were times I cared about Ruby on Windows more then I care now ;)) where the version number makes no sense as well, since you install every Ruby version into separate directory.

* For your case, the same job (i.e. make ruby/encodings.h undiscoverable without use of external configuration tool) would do, if we move:

/usr/include/ruby.h -> /usr/include/ruby/ruby.h
/usr/include/ruby/* -> /usr/include/ruby/ruby/*

But ruby/ruby looks ugly. The version number gives it more legitimate feel.

* If we support more versions of Ruby, such as Ruby 1.9.1 and Ruby 1.9.3 (just an example), we would need to version the include directories somehow and therefore diverge from the default upstream configuration yet again, since our versioned paths would need to correspond with Ruby versions, not with Ruby ABI.

So, to sum it up, I am not saying that "versioned paths need to be disabled", I am just saying that they are not designed to suit our needs and rise more question then they solve. I'd love to see what real world problem does the default configuration, with versions, actually solves.

Comment 8 Akinori MUSHA 2013-06-28 09:44:46 UTC
I checked openSUSE and Debian because I thought changing the header
location might be based on some policy or standards (such as FHS) that
are shared among major GNU/Linux based systems, though it turned out
not the case.  I then checked Gentoo Linux and Arch Linux only to find
out they all simply use the default configuration.  So, it seems most
distros don't think there is a problem with it and thus have no need
for making any decision.

Aside from the real problem I faced, putting a header file right under
/usr/include/ usually means it is usable stand-alone, which is, again,
not the case for ruby.h.  Having /usr/include/ruby.h just gives user
the misconception which I think is harmful.

You picked the word "correctness", but what's the basis of correctness
here when nothing is really broken?  I don't mean to play with words,
but your "dream world" thing sounds to me like it's your personal
preference after all...


I might have said too much.  Let's move on to the bright side.

For the ABI thing, the development team decided to take it more
seriously and ABI checks have been integrated into RubyCI [*]
(although still experimental).

* http://rubyci.org

So, I think your concerns will hopefully become a thing of the past in
the long run.

The importance of ABI/API version separated from the ruby release
version was first suggested by Yamada Akira in [ruby-dev:37748] (*)
right before 1.9.2, who is one of the package maintainers of Debian
where relatively strict ABI compatibility policy was adopted in
package management.  As far as I've heard, it has served well for
users.

* Originally in Japanese, so here's a link to Google Translate:
  http://translate.google.com/translate?u=http%3A%2F%2Fblade.nagaokaut.ac.jp%2Fcgi-bin%2Fscat.rb%2Fruby%2Fruby-dev%2F37748&hl=ja&langpair=auto|en&tbb=1&ie=EUC-JP

> * I cannot remember, how many times I saw the bugreports about 1.9.1
> version used in 1.9.3 paths. That is just confusing.

Yes, such reports are seen occasionally on Japanese mailing lists too,
but there's already pointers and it hasn't grown a big problem, while
I fully understand your frustration when you received repeated
questions more than enough, because I used to be (and still am, sort
of) a package maintainer.

I could also point out that, besides this ABI version thing, there are
a lot of things in Ruby that are the way they are for a reason
(technical or historical) but may look weird to those who first see
them.  So, I would rather take them as documentation problem and work
on improving documentation while they can't be easily changed.

That being said, I promise I'll add the ABI version thing to rdoc in
some way.  Actually I'm planning to resurrect the (official) Ruby FAQ.
So stay tuned, and please join and share your FAQ's when it's ready.
Thanks.

Comment 9 Akinori MUSHA 2013-06-28 10:03:27 UTC
To be fair and productive, I can name some OS distributions that
change the header directory.  It's not Linux, but FreeBSD and
OpenBSD.  They specify a configure option as below:

  --with-rubyhdrdir="$PREFIX/include/ruby-$MAJOR.$MINOR/"

So they look like /usr/local/include/ruby-1.9/ for example.  They
have a policy that only one teeny version of ruby can be
installed from one major.minor series at the same time, this was
fine. (ABI compatibility is handled by other means)

FYI, they both allow installing Ruby 2.0.0, 1.9.3 and 1.8.7 under
the same prefix at the same time.  Ruby has many configure
options to avoid conflicts between multiple installations of
ruby, like --program-suffix, --with-soname and
--with-ruby-version.

Maybe this could be acceptable to you, I guess?

Comment 10 KOSAKI Motohiro 2013-07-08 15:56:15 UTC
> 1) We support on Fedora only single system version of Ruby, therefore 
> the version is pointless.

Vit, this is technically incorrect. Fedora and RHEL have Software-collections (http://developerblog.redhat.com/2013/06/05/red-hat-software-collections-1-0-beta-now-available/). So, it is dangerous configuration and could be confusable. 

I wonder why do you want to ignore ruby gem eco-system. In fact, ruby development package is not only for C-ext developers. It is also used from Rails users.

It's ok either /usr/include/ruby/ruby or /usr/include/ruby-1.9/ruby or something else. But /usr/include is too dangerous and bring us wrong breakage.

If you continue lazy, we need to forcibly change upstream. Please help and cooperate with us.

Comment 11 Akinori MUSHA 2013-08-28 03:18:33 UTC
So, what's the next step we should take?  Am I supposed to submit a patch?

Comment 12 Fedora End Of Life 2013-09-16 16:44:01 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 20 development cycle.
Changing version to '20'.

More information and reason for this action is here:
https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Fedora20

Comment 13 Fedora End Of Life 2015-05-29 09:07:51 UTC
This message is a reminder that Fedora 20 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 20. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '20'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 20 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 14 Fedora End Of Life 2015-06-30 00:39:39 UTC
Fedora 20 changed to end-of-life (EOL) status on 2015-06-23. Fedora 20 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.