Bug 1079042

Summary: MD5 makes ruby interpreter crash in FIPS mode (again)
Product: Red Hat Enterprise Linux 6 Reporter: jared jennings <jjennings>
Component: rubyAssignee: Vít Ondruch <vondruch>
Status: CLOSED WONTFIX QA Contact: BaseOS QE - Apps <qe-baseos-apps>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 6.6CC: janarula, joseph.yaworski, vondruch
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-10-21 08:30:59 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1075802    
Attachments:
Description Flags
Patches Ruby 1.8.7-p352 to check for MD5 digest init failure in ext/digest. none

Description jared jennings 2014-03-20 20:26:54 UTC
When OpenSSL is running in FIPS compliant mode, it refuses to perform an MD5 checksum, because MD5 is not a FIPS Approved algorithm. When any Ruby script tries to do an MD5 checksum, the interpreter crashes. It would be much nicer if it threw an exception instead.

Version-Release number of selected component (if applicable):
ruby-1.8.7.352-13.el6.x86_64
openssl-1.0.1e-16.el6_5.4.x86_64

To reproduce:
OPENSSL_FORCE_FIPS_MODE= ruby -rdigest -e "puts Digest::MD5.hexdigest('hi')"

Actual results:
md5_dgst.c(78): OpenSSL internal error, assertion failed: Digest MD5 forbidden in FIPS mode!
Aborted (core dumped)

Expected results:
An exception is raised which can be caught inside the Ruby program; the Ruby interpreter keeps executing.

This bug has the same root cause as BZ717709, but it lies in the digest extension, not the openssl extension.

To fix, make Ruby's digest extension check the return code of algo->init_func in ext/digest/digest.c.

Comment 2 jared jennings 2014-03-24 23:01:33 UTC
As detailed in the upstream bug, the fix I suggested in the description above will not work.

Comment 3 Vít Ondruch 2014-03-25 11:12:24 UTC
Hi Jared,

Thank you for reporting into BZ as well as opening upstream ticket. I am wondering, if you are testing just with 1.8.7 or with latest upsteram version. I am asking, since I doubt that upstream would be interested in 1.8.7. But if you worked on trunk, it could result in some backportable patch ...

Comment 4 Vít Ondruch 2014-03-25 11:13:00 UTC
BTW you might be interested to test Ruby 2.0.0 in RHEL7 beta as well.

Comment 5 jared jennings 2014-03-28 21:47:25 UTC
Created attachment 880030 [details]
Patches Ruby 1.8.7-p352 to check for MD5 digest init failure in ext/digest.

This one patch is the same as the three patches posted to the upstream bug, but backported to Ruby 1.8.7-p352 as found in RHEL6. It applies cleanly when done immediately after Patch46, the earlier md5 digest init fail patch for ext/openssl.

Comment 6 jared jennings 2014-03-28 21:49:12 UTC
You may want to look at the upstream bug to see all of the things I'm not sure I did in the best way possible in this patch.

Vít, since the Ruby trunk has this problem, I'd imagine Ruby 2.0 as found in RHEL7 beta would too.

Comment 7 Vít Ondruch 2014-06-25 07:53:33 UTC
Hi Jared,

Looking closer at your patch, I don't think it is acceptable for Ruby 1.8.7 in RHEL6. Since you are changing function signatures on several places (although just return value), I am afraid that it breaks API/ABI and hence it could break existing applications/libraries.

Comment 8 jared jennings 2014-07-01 14:05:48 UTC
That's a fair analysis, Vít. Obviously I would have liked to avoid changing the API but I couldn't see any way around it, and I still can't.

Is there any Ruby interpreter available from Red Hat which can have this patch applied to it? (ruby193 software collection? ruby in RHEL7?) Or are they all subject to the same concerns over API/ABI breakage? Put the opposite way: Does this fix have to happen solely in the upstream, and then percolate into Red Hat products over years? If that is so, please say it in the upstream issue, because they may think this problem is sort of Red Hat-specific. I'd hate to arrive at a ... eheh, 'Patch-22.'

Comment 9 Vít Ondruch 2014-07-15 10:12:07 UTC
(In reply to jared jennings from comment #8)
Hi Jared,

Sorry for late reply.

> Is there any Ruby interpreter available from Red Hat which can have this
> patch applied to it? (ruby193 software collection? ruby in RHEL7?) Or are
> they all subject to the same concerns over API/ABI breakage?

Well, as long as it breaks API/ABI, there is no way how to apply the patch into minor release of our product. I.e. the first possibilities to introduce this patch would be RHEL8 or RHSCL2, but not much known yet about them.

> Put the
> opposite way: Does this fix have to happen solely in the upstream, and then
> percolate into Red Hat products over years?

Well, this depends on timing, but upstreaming the patch is of course the best think.

> If that is so, please say it in
> the upstream issue, because they may think this problem is sort of Red
> Hat-specific. I'd hate to arrive at a ... eheh, 'Patch-22.'

Done, not sure if that helps.

Comment 10 Vít Ondruch 2014-10-21 08:30:59 UTC
Hello Jared,

The patches were accepted upstream and will go into Ruby 2.2. However, since they break API/ABI I am afraid I can't do more for you ATM and hence closing the ticket as WONTFIX.