Bug 802946 - hardcoded MD5 use leads to SSL server failure in FIPS mode
Summary: hardcoded MD5 use leads to SSL server failure in FIPS mode
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: ruby
Version: 6.2
Hardware: Unspecified
OS: Unspecified
Target Milestone: rc
: ---
Assignee: Vít Ondruch
QA Contact: Iveta Wiedermann
Depends On:
Blocks: 947775 1070830
TreeView+ depends on / blocked
Reported: 2012-03-13 18:58 UTC by jared jennings
Modified: 2018-12-09 16:47 UTC (History)
5 users (show)

Fixed In Version: ruby-
Doc Type: Bug Fix
Doc Text:
Clone Of:
Last Closed: 2014-10-14 06:41:32 UTC
Target Upstream Version:

Attachments (Terms of Use)
Use srand and rand in Ruby 1.8.7 to make a session id (553 bytes, patch)
2012-08-23 14:10 UTC, jared jennings
no flags Details | Diff

System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2014:1470 0 normal SHIPPED_LIVE ruby bug fix and enhancement update 2014-10-14 01:10:44 UTC

Description jared jennings 2012-03-13 18:58:57 UTC

Puppet's "master" subcommand, which uses an HTTPS server based on WEBrick, does not work when the system is in FIPS-compliant mode. WEBrick uses the Ruby 'openssl' module to access the OpenSSL library, but the openssl module is hard-coded to use MD5 for a session cache. On a FIPS-compliant system, the OpenSSL library refuses to do MD5 hashing, because MD5 is no longer a FIPS Approved algorithm.

As of this writing, upstream has not yet decided what to do.

Version-Release number of selected component (if applicable):

Comment 2 Vít Ondruch 2012-07-10 07:26:42 UTC
This was fixed in upstream [1]. I did not tried to backport the patch, but it should be doable I guess.

[1] http://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/36005

Comment 3 ruckc@yahoo.com 2012-08-23 03:49:25 UTC
This would be extremely helpful if it was backported to ruby 1.8.7

Comment 4 jared jennings 2012-08-23 14:10:47 UTC
Created attachment 606606 [details]
Use srand and rand in Ruby 1.8.7 to make a session id

The porting concern is that Ruby 1.8.7 doesn't have a Random class, but the patch against the Ruby trunk <http://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/36005/diff/ext/openssl/lib/openssl/ssl.rb?format=diff> uses the Random class.

This equivalent, which should work under Ruby 1.8.7, uses srand and rand instead of the Random class. It is inspired by answers from <http://stackoverflow.com/questions/88311/how-best-to-generate-a-random-string-in-ruby>.

A risk here is that the PRNG state changed by srand is global, unlike the PRNG state encapsulated in the Random object used in the patch against Ruby trunk, and this code messes with that global state every time a new SSL session happens. An attacker who knows enough could maybe find out the seeds being used, and do mischief. But if you need to avoid using MD5 so badly, you're probably in FIPS-compliant mode, and so OpenSSL may not be using the same PRNG for cryptography as Ruby is for rand calls. That could mitigate the risk.

Comment 5 RHEL Program Management 2012-09-07 05:07:16 UTC
This request was evaluated by Red Hat Product Management for
inclusion in the current release of Red Hat Enterprise Linux.
Because the affected component is not scheduled to be updated
in the current release, Red Hat is unable to address this
request at this time.

Red Hat invites you to ask your support representative to
propose this request, if appropriate, in the next release of
Red Hat Enterprise Linux.

Comment 11 Vít Ondruch 2014-07-09 14:13:39 UTC
Hi Jared,

Thinking about this once more, I think the most straight forward way will be to replace the MD5 by SHA256, as you proposed in your upstream ticket. This should work for us, since we don't share upstream concerns about older OpenSSL library.

diff --git a/ext/openssl/lib/openssl/ssl-internal.rb b/ext/openssl/lib/openssl/ssl-internal.rb
index 9ef6f92..01dda99 100644
--- a/ext/openssl/lib/openssl/ssl-internal.rb
+++ b/ext/openssl/lib/openssl/ssl-internal.rb
@@ -148,7 +148,7 @@ module OpenSSL
         @svr = svr
         @ctx = ctx
         unless ctx.session_id_context
-          session_id = OpenSSL::Digest::MD5.hexdigest($0)
+          session_id = OpenSSL::Digest::SHA256.hexdigest($0)[0...32]
           @ctx.session_id_context = session_id
         @start_immediately = true

This would be the patch. Lets see what our FIPS guys think about it.

Comment 12 Tomas Mraz 2014-07-09 14:16:11 UTC
Yes, this patch looks perfectly acceptable to me.

Comment 21 errata-xmlrpc 2014-10-14 06:41:32 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.