Bug 959189 - Corosync doesn't preempt in confdb properly and can hang on exit
Corosync doesn't preempt in confdb properly and can hang on exit
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: corosync (Show other bugs)
6.4
Unspecified Unspecified
medium Severity medium
: rc
: ---
Assigned To: Jan Friesse
Cluster QE
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-05-03 06:15 EDT by Jan Friesse
Modified: 2013-11-20 23:34 EST (History)
4 users (show)

See Also:
Fixed In Version: corosync-1.4.1-16.el6
Doc Type: Bug Fix
Doc Text:
Cause: Confdb client uses corosync heavily and corosync is exited (sending SIGINT or corosync-cfgtool). Consequence: Corosync will never exit. Fix: Make sure all confdb IPC connections are closed on corosync exit. Result: Corosync will exit properly.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-11-20 23:34:16 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Proposed patch - part 1 - Free confdb message holder list on confdb exit (1.63 KB, patch)
2013-05-03 06:15 EDT, Jan Friesse
no flags Details | Diff
Proposed patch - part 2 - Make objdb_notify_dispatch preemptable (3.87 KB, patch)
2013-05-03 06:16 EDT, Jan Friesse
no flags Details | Diff

  None (edit)
Description Jan Friesse 2013-05-03 06:15:27 EDT
Created attachment 743143 [details]
Proposed patch - part 1 - Free confdb message holder list on confdb exit

Description of problem:
objdb_notify_dispatch traverse thru confdb_ipc_message_holder_list list
until it's empty. If IPC connection is still feeding confdb/objdb with
changes then iteration will never end. To prevent this and let corosync
properly preempt, terminal condition is enhanced with maximum number of
processed items from list. Change makes confdb preemptable so corosync
can accept new connections, receive/send token, ...

Also when confdb service is unloaded. it's needed to free confdb message holder list. This operation will not only properly free memory, but also decreases
connection reference(s). This should solve deadloop in
coroipcs_ipc_service_exit, because confdb_exit_fn removes not only
notify_pipe but also deletes poll_dispatch, so nobody is left to
unreference connections.

Version-Release number of selected component (if applicable):
Flatiron

How reproducible:
98.76

Steps to Reproduce:
1. Start corosync -p
2. Start IPC client which track key changes and also make changes of that key
3. Exit corosync (sending SIGINT)

"Unit" test:
https://github.com/jfriesse/csts/blob/master/tests/stop-with-confdb-load.sh

Actual results:
Corosync will not exit (unit test $? != 0)

Expected results:
Corosync will properly exit (unit test $? == 0)

Additional info:
Comment 1 Jan Friesse 2013-05-03 06:16:10 EDT
Created attachment 743144 [details]
Proposed patch - part 2 - Make objdb_notify_dispatch preemptable
Comment 8 Jaroslav Kortus 2013-09-11 11:33:46 EDT
Verified with stop-with-confdb-load.sh test:
FAIL on corosync-1.4.1-15.el6.x86_64 (RHEL6.4)
PASS on corosync-1.4.1-17.el6.x86_64 (RHEL6.5)
Comment 10 errata-xmlrpc 2013-11-20 23:34:16 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1531.html

Note You need to log in before you can comment on or make changes to this bug.