Bug 743815 - Resolve a deadlock between the timer and serialize locks.
Summary: Resolve a deadlock between the timer and serialize locks.
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: corosync
Version: 6.2
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: rc
: 6.3
Assignee: Jan Friesse
QA Contact: Cluster QE
URL:
Whiteboard:
Depends On:
Blocks: 756082
TreeView+ depends on / blocked
 
Reported: 2011-10-06 07:34 UTC by Jan Friesse
Modified: 2012-06-20 12:22 UTC (History)
3 users (show)

Fixed In Version: corosync-1.4.1-5.el6
Doc Type: Bug Fix
Doc Text:
Cause Using EVT service from OpenAIS. Consequence Sometimes (it's race so really depend on many things like number of cores, load, scheduler, ...) corosync deadlocks. Fix Change order of locking. Result Corosync doesn't deadlocks any longer.
Clone Of:
Environment:
Last Closed: 2012-06-20 12:22:49 UTC
Target Upstream Version:

Attachments (Terms of Use)
Proposed patch (1.39 KB, patch)
2011-10-06 07:34 UTC, Jan Friesse 		no flags Details | Diff
View All Add an attachment (proposed patch, testcase, etc.)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2012:0777 0 normal SHIPPED_LIVE corosync bug fix and enhancement update 2012-06-19 20:35:04 UTC

Description Jan Friesse 2011-10-06 07:34:29 UTC 
Created attachment 526640 [details]
Proposed patch

Description of problem:
This patch resolves a deadlock between the serialize lock (in
    exec/main.c) and the timer lock (in exec/timer.c).  I observed this
    deadlock happening fairly quickly on a cluster using the EVT service
    from OpenAIS.  (OpenAIS 1.1.4, Corosync 1.4.1)

    In prioritized_timer_thread(), it was grabbing:
        1) timer lock
        2) serialize lock

    In another thread, you have:
        1) grab the serialize lock in deliver_fn() of exec/main.c
        2) grab the timer lock in corosync_timer_add_duration().

    The patch just swaps the locking order in the timer thread.


Version-Release number of selected component (if applicable):
1.4.1
Comment 5 Jan Friesse 2012-03-07 07:31:57 UTC 
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Cause
Using EVT service from OpenAIS.

Consequence
Sometimes (it's race so really depend on many things like number of cores, load, scheduler, ...) corosync deadlocks.

Fix
Change order of locking.

Result
Corosync doesn't deadlocks any longer.
Comment 8 errata-xmlrpc 2012-06-20 12:22:49 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2012-0777.html
Note You need to log in before you can comment on or make changes to this bug.