Bug 1234801

Summary: Renegotiation seems partially broken in latest openssl
Product: Red Hat Enterprise Linux 7 Reporter: Tomas Mraz <tmraz>
Component: opensslAssignee: Tomas Mraz <tmraz>
Status: CLOSED CURRENTRELEASE QA Contact: BaseOS QE Security Team <qe-baseos-security>
Severity: high Docs Contact:
Priority: urgent    
Version: 7.2CC: bruno, hkario, jeharris, jherrman, jprokes, ksrot, michele, mschuppe, praiskup, psklenar, qe-baseos-security, tgl, tmraz
Target Milestone: rcKeywords: Regression, ZStream
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: openssl-1.0.1e-51.el7 Doc Type: Bug Fix
Doc Text:
Under certain circumstances, the server-side renegotiation support previously did not work as expected. A PostgreSQL failure of database dumps through TLS connection could occur when the size of the dumped data was larger than the value defined in the ssl_renegotiation_limit setting. The regression that caused this bug has been fixed, and the PostgreSQL database dumps through TLS connection no longer fail in the described situation.
Story Points: ---
Clone Of: 1234487
: 1234931 (view as bug list) Environment:
Last Closed: 2015-11-20 10:22:08 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1234931    

Description Tomas Mraz 2015-06-23 09:36:10 UTC
Description of problem:
After updating to last week's openssl security release, Postgres is no longer able to transfer more than about 2GB across an SSL-encrypted connection.

Version-Release number of selected component (if applicable):
openssl-1.0.1e-30.el6_6.11.x86_64
Problem did not occur with openssl-1.0.1e-30.el6_6.9.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Configure a Postgres database with SSL enabled.
2. Create a database having more than 2GB of data in it (doesn't much matter what)
3. pg_dump -h localhost databasename >dumpfile

(must connect to localhost, not via a Unix socket, else SSL isn't used)

Actual results:
After emitting something over 2GB into the dump file, pg_dump fails with
pg_dump: [archiver (db)] query failed: SSL error: unexpected message
pg_dump: [archiver (db)] query was: SET search_path = public, pg_catalog
(the specific query that fails varies depending on the exact test case).

The server-side log looks like
LOG:  SSL error: sslv3 alert unexpected message
LOG:  could not receive data from client: Connection reset by peer
LOG:  unexpected EOF on client connection with an open transaction

Expected results:
Dump should complete successfully.

Additional info:
In my tests, setting ssl_renegotiation_limit = 0 (ie, disabling renegotiation) in postgresql.conf allows the dump to complete.  Curiously, however, changing ssl_renegotiation_limit from its default of 512MB to something else such as 10MB doesn't seem to have any effect on how soon the failure occurs.  In either case several renegotiations should have been completed before failure.  It kind of smells like a 32-bit-counter-overflow problem, but I dunno where such a counter could be.

I've personally reproduced this with current Postgres development tip, and there are reports in the Postgres mailing lists of similar failures with Postgres 8.4.9 and 9.2.13, so it seems pretty much independent of exactly which Postgres version you use.  Also, we are not hearing reports of such failures with any other vendor's OpenSSL.

This is of course not to rule out that the problem could be in Postgres.  I'd be happy to help try to narrow it down, if you can suggest things to test.

Comment 1 Tomas Mraz 2015-06-23 09:38:07 UTC
So our patch for the CVE-2015-1791 is the culprit. In the patch we included changes that depend on additional changes in the s3_clnt.c file and that breaks some cases of renegotiation.
We need to fix that urgently.

Comment 8 Jan Kurik 2015-11-20 10:22:08 UTC
This bug has been closed as CURRENTRELEASE due to delivery of the fix in a z-stream. As the component is not on ACL, the fix is currently included in y-stream as well.

For more information please see the zstream process documentation:
* https://engineering.redhat.com/trac/ZStream/attachment/wiki/WikiStart/Z-Stream_process_update_4.odp .