Bug 712188 - corosync-objctl -t run multiple times with heavy load deadlocks corosync
Summary: corosync-objctl -t run multiple times with heavy load deadlocks corosync
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: corosync
Version: 6.1
Hardware: All
OS: All
medium
high
Target Milestone: rc
: ---
Assignee: Jan Friesse
QA Contact: Cluster QE
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-06-09 18:20 UTC by Steven Dake
Modified: 2016-04-26 14:05 UTC (History)
3 users (show)

Fixed In Version: corosync-1.4.0-1.el6
Doc Type: Bug Fix
Doc Text:
Cause: A race condition when using the tracking functionality of the internal object database. Consequence: Corosync would lock-up under heavy load with contrived test cases. Fix: Resolved race condition. Result: Corosync now doesn't lock up with corosync-objctl -t is run multiple times with heavy objdb load.
Clone Of:
Environment:
Last Closed: 2011-12-06 11:51:09 UTC


Attachments (Terms of Use)
Proposed patch (5.95 KB, patch)
2011-06-16 15:21 UTC, Jan Friesse
no flags Details | Diff
Second version of proposed patch (6.04 KB, patch)
2011-06-17 12:35 UTC, Jan Friesse
no flags Details | Diff


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2011:1515 normal SHIPPED_LIVE corosync bug fix and enhancement update 2011-12-06 00:38:47 UTC

Description Steven Dake 2011-06-09 18:20:23 UTC
Description of problem:
During customer investigation of different issue, a scenario was found where it is possible to lock up corosync.  This use model could occur in the typical cluster suite "reload" operatoin.

Version-Release number of selected component (if applicable):
corosync-1.2.3-36.el6

How reproducible:
100%

Steps to Reproduce:
1. run corosync (or cluster suite)
2. run testa one time
3. run corosync-objctl -t test
4. run testall (make sure test is on the system in the cwd)
  
Actual results:
corosync deadlocks and stops processing and participating in cluster membership

Expected results:
corosync shouldn't deadlock

Additional info:

Comment 2 Steven Dake 2011-06-09 18:22:26 UTC
Honza,

Make sure to add a test case to the automated test suite for this scenario.

Comment 3 Steven Dake 2011-06-09 18:23:13 UTC
step 3 should be run atleast 2 times to generate the deadlock.

Comment 5 Jan Friesse 2011-06-16 15:21:04 UTC
Created attachment 505074 [details]
Proposed patch

Patch sent to ML

Comment 6 Jan Friesse 2011-06-17 12:35:26 UTC
Created attachment 505267 [details]
Second version of proposed patch

Patch sent to ML. It should fix all reviewer notes.

Comment 11 Steven Dake 2011-10-27 18:49:25 UTC
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Cause: A race condition when using the tracking functionality of the internal object database.
  Consequence: Corosync would lock-up under heavy load with contrived test cases.
  Fix: Resolved race condition.
  Result: Corosync now doesn't lock up with corosync-objctl -t is run multiple times with heavy objdb load.

Comment 12 errata-xmlrpc 2011-12-06 11:51:09 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2011-1515.html


Note You need to log in before you can comment on or make changes to this bug.