Bug 1308081 - rubygem-sanitize: FTBFS in rawhide
rubygem-sanitize: FTBFS in rawhide
Product: Fedora
Classification: Fedora
Component: rubygem-sanitize (Show other bugs)
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Achilleas Pipinellis
Fedora Extras Quality Assurance
Depends On:
Blocks: F24FTBFS
  Show dependency treegraph
Reported: 2016-02-13 17:24 EST by Fedora Release Engineering
Modified: 2017-08-08 09:00 EDT (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2017-08-08 09:00:13 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
build.log (10.74 KB, text/plain)
2016-02-13 17:24 EST, Fedora Release Engineering
no flags Details
root.log (82.74 KB, text/plain)
2016-02-13 17:24 EST, Fedora Release Engineering
no flags Details
state.log (657 bytes, text/plain)
2016-02-13 17:24 EST, Fedora Release Engineering
no flags Details
A script to check nokogiri behavior (586 bytes, text/plain)
2016-04-22 13:21 EDT, Jun Aruga
no flags Details

  None (edit)
Description Fedora Release Engineering 2016-02-13 17:24:21 EST
Your package rubygem-sanitize failed to build from source in current rawhide.


For details on mass rebuild see https://fedoraproject.org/wiki/Fedora_24_Mass_Rebuild
Comment 1 Fedora Release Engineering 2016-02-13 17:24:24 EST
Created attachment 1126304 [details]
Comment 2 Fedora Release Engineering 2016-02-13 17:24:25 EST
Created attachment 1126305 [details]
Comment 3 Fedora Release Engineering 2016-02-13 17:24:27 EST
Created attachment 1126306 [details]
Comment 4 Jan Kurik 2016-02-24 10:10:44 EST
This bug appears to have been reported against 'rawhide' during the Fedora 24 development cycle.
Changing version to '24'.

More information and reason for this action is here:
Comment 5 Jun Aruga 2016-04-22 12:55:51 EDT
Let me check this!
Comment 6 Jun Aruga 2016-04-22 13:21 EDT
Created attachment 1149855 [details]
A script to check nokogiri behavior

I am still trying to fix this test suite issue.

But I would like to share a information which I found. I looks useful.

After checking below build.log files at koschei [1], and checking the source code, I found the difference of the behavior for the ruby environments.

At fedora-rawhide-x86_64 mock environment, I did run the test for both system ruby (rubypick-1.1.1-5.fc24.noarch) (= describe ruby [a]), and the ruby that was installed from source code (= describe ruby [b]).

system ruby [a]: failure the test.
source compiled ruby [b]: success the test.

The difference of the result is the difference for result of running the attached script. See following result.

*GCC version*
# gcc --version
gcc (GCC) 6.0.0 20160406 (Red Hat 6.0.0-0.20)

*ruby [a]*
# /usr/bin/ruby -v
ruby 2.3.0p0 (2015-12-25 revision 53290) [x86_64-linux]

# rpm -qf /usr/bin/ruby

# /usr/bin/gem list

*** LOCAL GEMS ***

bigdecimal (1.2.8)
did_you_mean (1.0.0)
io-console (0.4.5)
json (1.8.3)
minitest (4.7.0)
nokogiri (
psych (2.0.17)
rdoc (4.2.1)

# /usr/bin/ruby test_nokogiri.rb
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">

=> false means with test failure.

*ruby [b]*

how to install the ruby [b] -> [2]

# /usr/local/ruby-2.3.0/bin/ruby -v
ruby 2.3.0p0 (2015-12-25 revision 53290) [x86_64-linux]

# /usr/local/ruby-2.3.0/bin/gem list

*** LOCAL GEMS ***

bigdecimal (1.2.8)
did_you_mean (1.0.0)
io-console (0.4.5)
json (1.8.3)
mini_portile2 (2.0.0)
minitest (5.8.3)
net-telnet (0.1.1)
nokogiri (
power_assert (0.2.6)
psych (2.0.17)
rake (10.4.2)
rdoc (4.2.1)
test-unit (3.1.5)

# /usr/local/ruby-2.3.0/bin/ruby test_nokogiri.rb
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">

=> true means with test success.

[1] https://apps.fedoraproject.org/koschei/package/rubygem-sanitize

*Get source to install these things with off-line in mock environment.

- https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.0.tar.gz
- gem fetch mini_portile2 -v 2.0.0
- gem fetch nokogiri -v 1.6.7

*Install package to run the test script.

mock -r fedora-rawhide-x86_64 -i zlib-devel
  => zlib-devel-1.2.8-10.fc24.x86_64
mock -r fedora-rawhide-x86_64 -i openssl-devel
  => openssl-devel-1.0.2g-4.fc25.x86_64

*Install Ruby from source in the mock environment.

tar xzvf ruby-2.3.0.tar.gz
./configure --prefix=/usr/local/ruby-2.3.0
make install

*Install gem packages in the mock environment.
/usr/local/ruby-2.3.0/bin/gem install --local mini_portile2-2.0.0.gem
/usr/local/ruby-2.3.0/bin/gem install --local nokogiri-
Comment 7 Vít Ondruch 2016-04-25 06:34:31 EDT
Well, this is reoccurring issue. I think we changed this escaping back and forward several times already. You should check what libxml2 is in use for each of your scenarios, since nokogiri bundles libxml2 as far as I know, so the "gem install" version is very likely using different version of libxml2 then the rubygem-nokogiri.
Comment 8 Jun Aruga 2016-04-25 09:39:43 EDT
Hi, Vit

Thanks for your message.
After checking my environment, I found that it was the difference of libxml2 version.

*ruby [a] / system ruby
-> rubygem-nokogiri-
-> system libxml2 v2.9.3
-> rubygem-sanitize test failure

*ruby [b] / the ruby installed from source by myself.
-> nokogiri
-> bundled libxml2 v2.9.2
->  rubygem-sanitize test success

About ruby [b], after updating libxml2 to nokogiri-1.6.8.rc3 (using bundled libxml2 v2.9.3), the test was failed. That became same with ruby [a].

So, now everything is clear for me.
I will update rubygem-sanitize test suite to pass for libxml2 v2.9.3.

Comment 9 Jun Aruga 2016-04-25 10:11:30 EDT
Firstly I will check rubygem-nokogiri package.
Comment 10 Jun Aruga 2016-04-26 04:33:34 EDT
(In reply to Jun Aruga from comment #9)
> Firstly I will check rubygem-nokogiri package.

I checked rubygem-sanitize test suite with rubygem-nokogiri 1.6.8.rc3 that I created by myself for test. [1]
But the test suite was failed in the mock environment, though I expected that nokogiri 1.6.8.rc3 fixed the difference of libxml2 v2.9.2 and v2.9.3.

# /usr/bin/gem list

*** LOCAL GEMS ***

bigdecimal (1.2.8)
did_you_mean (1.0.0)
io-console (0.4.5)
json (1.8.3)
minitest (4.7.0)
nokogiri (1.6.8.rc3)
pkg-config (1.1.7)
psych (2.0.17)
rdoc (4.2.1)

# ruby -Ilib -e 'Dir.glob "./test/test_*.rb", &method(:require)'
142 tests, 194 assertions, 5 failures, 0 errors, 0 skips

# ruby test_nokogiri.rb
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">

[1] Install rubygem-nokogiri 1.6.8.rc3 rpm package to mock environment.
After creating rubygem-nokogiri-1.6.8-0.1.rc3.fc25.x86_64.rpm package by myself.

mock -r fedora-rawhide-x86_64 -i rubygem-pkg-config
mock -r fedora-rawhide-x86_64 -i nokogiri_work/rubygem-nokogiri-1.6.8-0.1.rc3.fc25.x86_64.rpm
Comment 11 Pavel Valena 2016-05-04 13:16:14 EDT

I think it is a similar problem to I have had with rails-html-sanitizer[1].  

It did not get resolved though. It is some difference between libxml2 v2.9.2 and v2.9.3, which causes the failures. I think this would be best addressed upstream.

[1] https://github.com/rails/rails-html-sanitizer/issues/49
Comment 12 Jun Aruga 2016-05-16 04:45:27 EDT
Hi, Pavel. Thanks for the information!
Comment 13 Jun Aruga 2016-05-16 04:47:45 EDT
I am also asking to upstream: "nokogiri", that is used by "sanitize"[1].

[1] https://github.com/sparklemotion/nokogiri/issues/1465
Comment 14 Jun Aruga 2016-05-16 12:21:39 EDT
(In reply to Jun Aruga from comment #13)
> I am also asking to upstream: "nokogiri", that is used by "sanitize"[1].
> [1] https://github.com/sparklemotion/nokogiri/issues/1465

Today I got the response from the owner of nokogiri.

> Because Nokogiri on MRI relies on libxml to do markup correction, I absolutely > believe that the version of libxml affects this behavior.
> Further, there is no standard way to correct broken or invalid markup, and so > this cannot be classified as a bug, since the behavior of parsers is undefined.
> Finally, this correction behavior is entirely internal within libxml, and so the Nokogiri code cannot reasonably "fix" the output to be consistent across libxml versions (though we have on occasion tried). 

So, I am planning 2 ways as my next steps.

1. Try to ask the owner of libxml about this situation, what is the reason for the changing.
2. I will try to update the sanitize version to latest one with 2 new rubygem packages: rubygem-crass", "rubygem-nokogumbo" that we need to create, in my local environment.
Comment 15 Fedora End Of Life 2017-07-25 16:07:49 EDT
This message is a reminder that Fedora 24 is nearing its end of life.
Approximately 2 (two) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 24. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '24'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 24 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.
Comment 16 Fedora End Of Life 2017-08-08 09:00:13 EDT
Fedora 24 changed to end-of-life (EOL) status on 2017-08-08. Fedora 24 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this

Thank you for reporting this bug and we are sorry it could not be fixed.

Note You need to log in before you can comment on or make changes to this bug.