Bug 743815

Summary: Resolve a deadlock between the timer and serialize locks.
Product: Red Hat Enterprise Linux 6 Reporter: Jan Friesse <jfriesse>
Component: corosyncAssignee: Jan Friesse <jfriesse>
Status: CLOSED ERRATA QA Contact: Cluster QE <mspqa-list>
Severity: high Docs Contact:
Priority: high    
Version: 6.2CC: cluster-maint, jkortus, sdake
Target Milestone: rc   
Target Release: 6.3   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: corosync-1.4.1-5.el6 Doc Type: Bug Fix
Doc Text:
Cause Using EVT service from OpenAIS. Consequence Sometimes (it's race so really depend on many things like number of cores, load, scheduler, ...) corosync deadlocks. Fix Change order of locking. Result Corosync doesn't deadlocks any longer.
 Story Points: ---
Clone Of: Environment:
Last Closed: 2012-06-20 12:22:49 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 756082    
Attachments:
Description Flags
Proposed patch none

Description Jan Friesse 2011-10-06 07:34:29 UTC 
Created attachment 526640 [details]
Proposed patch

Description of problem:
This patch resolves a deadlock between the serialize lock (in
    exec/main.c) and the timer lock (in exec/timer.c).  I observed this
    deadlock happening fairly quickly on a cluster using the EVT service
    from OpenAIS.  (OpenAIS 1.1.4, Corosync 1.4.1)

    In prioritized_timer_thread(), it was grabbing:
        1) timer lock
        2) serialize lock

    In another thread, you have:
        1) grab the serialize lock in deliver_fn() of exec/main.c
        2) grab the timer lock in corosync_timer_add_duration().

    The patch just swaps the locking order in the timer thread.


Version-Release number of selected component (if applicable):
1.4.1
Comment 5 Jan Friesse 2012-03-07 07:31:57 UTC 
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Cause
Using EVT service from OpenAIS.

Consequence
Sometimes (it's race so really depend on many things like number of cores, load, scheduler, ...) corosync deadlocks.

Fix
Change order of locking.

Result
Corosync doesn't deadlocks any longer.
Comment 8 errata-xmlrpc 2012-06-20 12:22:49 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2012-0777.html