Bug 703891 - Deadlock when multiple threads concurrently operate on the same receiver
Summary: Deadlock when multiple threads concurrently operate on the same receiver
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise MRG
Classification: Red Hat
Component: qpid-cpp
Version: 1.3
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: 3.0
: ---
Assignee: Gordon Sim
QA Contact: mick
URL:
Whiteboard:
Depends On:
Blocks: 698367 803771
TreeView+ depends on / blocked
 
Reported: 2011-05-11 14:52 UTC by Jason Dillaman
Modified: 2014-09-24 15:02 UTC (History)
5 users (show)

Fixed In Version: qpid-cpp-0.22-4.el6, qpid-cpp-0.22-4.el5
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-09-24 15:02:47 UTC
Target Upstream Version:


Attachments (Terms of Use)
Deadlock Backtrace (6.46 KB, text/plain)
2011-05-11 14:52 UTC, Jason Dillaman
no flags Details
3-thread reproducer : threads ack, fetch, and close, with random jitter (7.90 KB, text/x-c++src)
2013-07-18 18:34 UTC, mick
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Apache JIRA QPID-4764 0 None None None Never
Red Hat Product Errata RHEA-2014:1296 0 normal SHIPPED_LIVE Red Hat Enterprise MRG Messaging 3.0 Release 2014-09-24 19:00:06 UTC

Description Jason Dillaman 2011-05-11 14:52:22 UTC
Created attachment 498309 [details]
Deadlock Backtrace

Description of problem:
Multiple threads are performing various operations on a single QPID receiver at the same time.  One thread is acknowledging a previously received message, another thread is getting the next message from the receiver, and the third thread is attempting to close the receiver.

Version-Release number of selected component (if applicable):
qpid-cpp-client-0.7.946106-28_ptc_hotfix_5_v2.el5

How reproducible:
100% when the above conditions are met

Steps to Reproduce:
1. 1st thread acks a message from a given receiver
2. 2nd thread is fetching the next message from the given receiver
3. 3rd thread is attempting to close the receiver
  
Actual results:
The three threads become deadlocks.  Thread 1 holds SessionImpl's lock and is blocked on IncomingMessage's lock.  Thread 2 holds IncomingMessage's lock and is blocked on ReceiverImpl's lock.  Thread 3 holds the ReceiverImpl's lock and is blocked on SessionImpl's lock. 

Expected results:
No deadlock and the receiver is closed.

Additional info:
See attached backtrace

Comment 1 Gordon Sim 2013-04-25 12:30:01 UTC
Fix committed as http://svn.apache.org/viewvc?view=revision&revision=1475723 that should break this deadlock. Note that I was unable to reproduce it however.

Comment 2 mick 2013-07-18 18:34:00 UTC
Created attachment 775458 [details]
3-thread reproducer :  threads ack, fetch, and close, with random jitter

Please see comment at top of code in this attachment.

Comment 3 mick 2013-07-18 18:35:37 UTC
I have been unable to repro this bug after more than 21,000 iterations of my reproducer (see attached.)  so I am calling it verified.

Comment 4 mick 2013-07-19 14:20:07 UTC
Post Scriptum
-------------------------

I finally stopped my reproducer-attempt at 45,000 iterations, with no joy.

Comment 5 errata-xmlrpc 2014-09-24 15:02:47 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2014-1296.html


Note You need to log in before you can comment on or make changes to this bug.