Bug 717709

Summary:	MD5 makes ruby interpreter crash in FIPS mode
Product:	Red Hat Enterprise Linux 6	Reporter:	jared jennings <jjennings>
Component:	ruby	Assignee:	Vít Ondruch <vondruch>
Status:	CLOSED ERRATA	QA Contact:	Aleš Mareček <amarecek>
Severity:	medium	Docs Contact:
Priority:	high
Version:	6.1	CC:	amarecek, eng-i18n-bugs
Target Milestone:	rc
Target Release:	---
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2011-12-06 12:08:01 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	691449

Description jared jennings 2011-06-29 15:52:21 UTC

Description of problem:
When OpenSSL is running in FIPS compliant mode, it refuses to perform an MD5 checksum, because MD5 is not a FIPS Approved algorithm. When any Ruby script tries to do an MD5 checksum, the interpreter crashes. It would be much nicer if it threw an exception instead.

Version-Release number of selected component (if applicable):
ruby-1.8.7.299-7.el6.i686
openssl-1.0.0-10.el6.i686

To reproduce, make sure /proc/sys/crypto/fips_enabled contains '1' and OpenSSL works properly otherwise. Write the following script to a file fips-md5.rb.

 require 'openssl'
 md5 = OpenSSL::Digest::MD5.new
 md5 << 'hi'
 puts md5.hexdigest

Now, run ruby fips-md5.rb.

Actual results:
 fips-md5.rb:3: [BUG] Segmentation fault
 ruby 1.8.7 (2010-06-23 patchlevel 299) [i386-linux]

 Aborted (core dumped)

Expected results: an exception is raised at fips-md5.rb line 2.

To fix, make Ruby's openssl extension check the return code of EVP_DigestInit_ex in ext/openssl/ossl_digest.c.

The upstream issue is at http://redmine.ruby-lang.org/issues/4944. Details of how the interpreter crashes, and a patch against the nightly snapshot, are there.

A patch against ruby-1.8.7.299-5.el6.i686 is at http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/384989. I think it would apply to the current release 7 also.

Discussion is at http://projects.puppetlabs.com/issues/8120#note-2.

Comment 2 Vít Ondruch 2011-07-29 12:23:10 UTC

Hello Jared,

What is the current state in upstream? If the patch is applied upstream and confirmed its functionality, we could apply this patch into RHEL.

Comment 3 jared jennings 2011-08-03 15:04:35 UTC

The upstream developers have improved on my patch; their patch (against the Ruby trunk) is at http://redmine.ruby-lang.org/projects/ruby-19/repository/revisions/32606/diff.diff. They don't have a host configured for FIPS compliance, so I've just confirmed their fix. Details at http://redmine.ruby-lang.org/issues/4944.

Comment 5 Vít Ondruch 2011-08-08 13:06:22 UTC

Short reproducer:

Before:
mock-chroot> ruby -ropenssl -e "puts OpenSSL::Digest::MD5.new('hi').hexdigest"
49f68a5c8493ec2c0bf489821c21fc3b

mock-chroot> OPENSSL_FORCE_FIPS_MODE= ruby -ropenssl -e "puts OpenSSL::Digest::MD5.new('hi').hexdigest"
/usr/lib/ruby/1.8/openssl/digest.rb:40: [BUG] Segmentation fault
ruby 1.8.7 (2010-06-23 patchlevel 299) [x86_64-linux]

Neúspěšně ukončen (SIGABRT) (core dumped [obraz paměti uložen])


After:
mock-chroot> ruby -ropenssl -e "puts OpenSSL::Digest::MD5.new('hi').hexdigest"
49f68a5c8493ec2c0bf489821c21fc3b

mock-chroot> OPENSSL_FORCE_FIPS_MODE= ruby -ropenssl -e "puts OpenSSL::Digest::MD5.new('hi').hexdigest"
/usr/lib/ruby/1.8/openssl/digest.rb:40:in `initialize': Digest initialization failed.: unknown cipher (OpenSSL::Digest::DigestError)
	from /usr/lib/ruby/1.8/openssl/digest.rb:40:in `initialize'
	from -e:1:in `new'
	from -e:1

Comment 8 errata-xmlrpc 2011-12-06 12:08:01 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2011-1581.html

Comment 9 jared jennings 2014-03-20 17:42:44 UTC

It appears that redmine.ruby-lang.org is gone, replaced by bugs.ruby-lang.org; so the links to the upstream issue and change should now be accessed as

https://bugs.ruby-lang.org/issues/4944
https://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/32606/diff/