Bug 717709 - MD5 makes ruby interpreter crash in FIPS mode
MD5 makes ruby interpreter crash in FIPS mode
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: ruby (Show other bugs)
6.1
All Linux
high Severity medium
: rc
: ---
Assigned To: Vít Ondruch
Aleš Mareček
:
Depends On:
Blocks: BaseOS-FIPS-Tracker
  Show dependency treegraph
 
Reported: 2011-06-29 11:52 EDT by jared jennings
Modified: 2014-03-20 13:42 EDT (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2011-12-06 07:08:01 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)

  None (edit)
Description jared jennings 2011-06-29 11:52:21 EDT
Description of problem:
When OpenSSL is running in FIPS compliant mode, it refuses to perform an MD5 checksum, because MD5 is not a FIPS Approved algorithm. When any Ruby script tries to do an MD5 checksum, the interpreter crashes. It would be much nicer if it threw an exception instead.

Version-Release number of selected component (if applicable):
ruby-1.8.7.299-7.el6.i686
openssl-1.0.0-10.el6.i686

To reproduce, make sure /proc/sys/crypto/fips_enabled contains '1' and OpenSSL works properly otherwise. Write the following script to a file fips-md5.rb.

 require 'openssl'
 md5 = OpenSSL::Digest::MD5.new
 md5 << 'hi'
 puts md5.hexdigest

Now, run ruby fips-md5.rb.

Actual results:
 fips-md5.rb:3: [BUG] Segmentation fault
 ruby 1.8.7 (2010-06-23 patchlevel 299) [i386-linux]

 Aborted (core dumped)

Expected results: an exception is raised at fips-md5.rb line 2.

To fix, make Ruby's openssl extension check the return code of EVP_DigestInit_ex in ext/openssl/ossl_digest.c.

The upstream issue is at http://redmine.ruby-lang.org/issues/4944. Details of how the interpreter crashes, and a patch against the nightly snapshot, are there.

A patch against ruby-1.8.7.299-5.el6.i686 is at http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/384989. I think it would apply to the current release 7 also.

Discussion is at http://projects.puppetlabs.com/issues/8120#note-2.
Comment 2 Vít Ondruch 2011-07-29 08:23:10 EDT
Hello Jared,

What is the current state in upstream? If the patch is applied upstream and confirmed its functionality, we could apply this patch into RHEL.
Comment 3 jared jennings 2011-08-03 11:04:35 EDT
The upstream developers have improved on my patch; their patch (against the Ruby trunk) is at http://redmine.ruby-lang.org/projects/ruby-19/repository/revisions/32606/diff.diff. They don't have a host configured for FIPS compliance, so I've just confirmed their fix. Details at http://redmine.ruby-lang.org/issues/4944.
Comment 5 Vít Ondruch 2011-08-08 09:06:22 EDT
Short reproducer:

Before:
mock-chroot> ruby -ropenssl -e "puts OpenSSL::Digest::MD5.new('hi').hexdigest"
49f68a5c8493ec2c0bf489821c21fc3b

mock-chroot> OPENSSL_FORCE_FIPS_MODE= ruby -ropenssl -e "puts OpenSSL::Digest::MD5.new('hi').hexdigest"
/usr/lib/ruby/1.8/openssl/digest.rb:40: [BUG] Segmentation fault
ruby 1.8.7 (2010-06-23 patchlevel 299) [x86_64-linux]

Neúspěšně ukončen (SIGABRT) (core dumped [obraz paměti uložen])


After:
mock-chroot> ruby -ropenssl -e "puts OpenSSL::Digest::MD5.new('hi').hexdigest"
49f68a5c8493ec2c0bf489821c21fc3b

mock-chroot> OPENSSL_FORCE_FIPS_MODE= ruby -ropenssl -e "puts OpenSSL::Digest::MD5.new('hi').hexdigest"
/usr/lib/ruby/1.8/openssl/digest.rb:40:in `initialize': Digest initialization failed.: unknown cipher (OpenSSL::Digest::DigestError)
	from /usr/lib/ruby/1.8/openssl/digest.rb:40:in `initialize'
	from -e:1:in `new'
	from -e:1
Comment 8 errata-xmlrpc 2011-12-06 07:08:01 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2011-1581.html
Comment 9 jared jennings 2014-03-20 13:42:44 EDT
It appears that redmine.ruby-lang.org is gone, replaced by bugs.ruby-lang.org; so the links to the upstream issue and change should now be accessed as

https://bugs.ruby-lang.org/issues/4944
https://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/32606/diff/

Note You need to log in before you can comment on or make changes to this bug.