Bug 524664 - Large messages cause hangs and crashes when using digest-md5 and security layer (ssf 128)
Summary: Large messages cause hangs and crashes when using digest-md5 and security lay...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise MRG
Classification: Red Hat
Component: qpid-cpp
Version: 1.1.1
Hardware: All
OS: Linux
high
high
Target Milestone: 1.2
: ---
Assignee: mick
QA Contact: Jiri Kolar
URL:
Whiteboard:
Depends On: 501792
Blocks: 527551 CVE-2009-5004
TreeView+ depends on / blocked
 
Reported: 2009-09-21 16:45 UTC by Gordon Sim
Modified: 2010-10-12 17:40 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Messaging bug fix C: Large messages being sent while using digest-md5 and security layer C: The broker would either hang or crash. F: added an encode buffer to the CyrusSecurityLayer class and encrypts the minimum of the encode buffer and the max input size. R: crashes no longer happen when using MD5 mechanism on large messages. When very large messages were sent using digest-md5 and a security layer, the broker would either hang or crash. An encode buffer was added to the CyrusSecurityLayer class, and crashes no longer occur.
Clone Of: 501792
Environment:
Last Closed: 2009-12-03 09:15:53 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
backport of fix to 1.1.6 (3.32 KB, patch)
2009-09-28 19:31 UTC, mick
no flags Details | Diff


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2009:1633 0 normal SHIPPED_LIVE Red Hat Enterprise MRG Messaging and Grid Version 1.2 2009-12-03 09:15:33 UTC

Description Gordon Sim 2009-09-21 16:45:17 UTC
+++ This bug was initially created as a clone of Bug #501792 +++

See https://issues.apache.org/jira/browse/QPID-1819

--- Additional comment from gsim on 2009-06-01 13:01:45 EDT ---

Fixed on qpid trunk by r780719. (Workaround until that is available through new MRG packages is to reduce the max-frame-size option in ConnectionSettings).

To test, send large messages (e.g. using perftest) with the cyrus-sasl-md5 package installed and use the DIGEST-MD5 mechanism if available (it is usually preferred over other mechanisms, but can be forced by specifying the mechanism on command line).

Comment 1 mick 2009-09-28 19:31:35 UTC
Created attachment 362937 [details]
backport of fix to 1.1.6

This is the backport of this  fix to 752581-26 ( 1.1.6 ) .

However -- I still see the hang.

hang reproduced this way:

    1. make sure that cyrus-sasl-md5 is installed on your system

    2. get trunk svn version 752581

    3. update with -26 patch

    4. start broker on port P
       make sure auth=yes, 
       confirm from log statements that it is using MD5

    5. ./perftest --username USERNAME -p P --mechanism DIGEST-MD5 --size 100000
       # enter passwd for USERNAME several times when asked.
       
       see it hang here:
           Processing 1 messages from pub_done .

    6. Repeat with --size=100  and see it not hang.

Comment 2 mick 2009-09-29 20:40:16 UTC
Gsim's patch, attached in comment 1 above, *is* effective!

I was not waiting long enough to see the test complete -- I forgot how many messages perftest was trying to send.   To see the patch work, you should reproduce the problem with fewer messages being transmitted, this way:

./perftest --username guest -p 5815 --mechanism DIGEST-MD5 --size 100000 --count 100


With only 100 messages being sent, the patched version of -26 terminates successfully within one second.    Without this new patch, it does not terminate.

Comment 3 mick 2009-09-30 16:42:47 UTC
pushed to 1.1.x branch.

Comment 5 Irina Boverman 2009-10-22 17:45:51 UTC
Release note added. If any revisions are required, please set the 
"requires_release_notes" flag to "?" and edit the "Release Notes" field accordingly.
All revisions will be proofread by the Engineering Content Services team.

New Contents:
Corrected problem with large messages causing hangs and crashes when using digest-md5 and security layer (ssf 128) (524664)

Comment 7 Jiri Kolar 2009-10-23 16:50:35 UTC
Tested:
on -26 bug aapears (but only on x86_64)
on -30 has been fixed

validated on RHEL RHEL5-Server-U4/RHEL4-U8 i386 / x86_64 

packages:

# rpm -qa | grep -E '(qpid|openais|rhm)' | sort -u
openais-0.80.6-8.el5_4.1
python-qpid-0.5.752581-4.el5
qpidc-0.5.752581-30.el5
qpidc-devel-0.5.752581-30.el5
qpidc-perftest-0.5.752581-30.el5
qpidc-rdma-0.5.752581-30.el5
qpidc-ssl-0.5.752581-30.el5
qpidd-0.5.752581-30.el5
qpidd-acl-0.5.752581-30.el5
qpidd-cluster-0.5.752581-30.el5
qpidd-devel-0.5.752581-30.el5
qpid-dotnet-0.4.738274-2.el5
qpidd-rdma-0.5.752581-30.el5
qpidd-ssl-0.5.752581-30.el5
qpidd-xml-0.5.752581-30.el5
qpid-java-client-0.5.751061-9.el5
qpid-java-common-0.5.751061-9.el5
rhm-0.5.3206-17.el5
rhm-docs-0.5.756148-1.el5
rh-tests-distribution-MRG-Messaging-qpid_common-1.5-15

->VERIFIED

Comment 8 Lana Brindley 2009-11-26 21:38:19 UTC
Release note updated. If any revisions are required, please set the 
"requires_release_notes"  flag to "?" and edit the "Release Notes" field accordingly.
All revisions will be proofread by the Engineering Content Services team.

Diffed Contents:
@@ -1 +1,8 @@
-Corrected problem with large messages causing hangs and crashes when using digest-md5 and security layer (ssf 128) (524664)+Messaging bug fix
+
+C: Large messages being sent while using digest-md5 and security layer
+C: The broker would either hang or crash.
+F:
+R:
+
+NEED FURTHER INFO FOR RELNOTE.

Comment 9 Lana Brindley 2009-12-01 23:36:24 UTC
Release note updated. If any revisions are required, please set the 
"requires_release_notes"  flag to "?" and edit the "Release Notes" field accordingly.
All revisions will be proofread by the Engineering Content Services team.

Diffed Contents:
@@ -2,7 +2,7 @@
 
 C: Large messages being sent while using digest-md5 and security layer
 C: The broker would either hang or crash.
-F:
-R:
+F: added an encode buffer to the CyrusSecurityLayer class and encrypts the minimum of the encode buffer and the max input size.
+R: crashes no longer happen when using MD5 mechanism on large messages.
 
-NEED FURTHER INFO FOR RELNOTE.+When very large messages were sent using digest-md5 and a security layer, the broker would either hang or crash. An encode buffer was added to the CyrusSecurityLayer class, and crashes no longer occur.

Comment 10 errata-xmlrpc 2009-12-03 09:15:53 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHEA-2009-1633.html


Note You need to log in before you can comment on or make changes to this bug.