Feature:
Corosync should be able to detect if it was not scheduled for long time, ether because the process itself was not scheduled, or VM was not scheduled.
Reason:
Corosync displays "Token was lost" if not scheduled for long time. Sadly, same message can be result of real Token lost on network. It is potentially useful to be able to distinguish between these two types of Token lost.
Result (if any):
When corosync was not scheduled for long time, message is logged.
Created attachment 732591[details]
Proposed patch
Description of problem:
When corosync is running in VM, it can happen, that VM (and thus corosync) is not scheduled for long time (longer then token timeout). Because in such environments, it's very hard to find out, if token was lost because of network overload or because of guest OS didn't scheduled corosync process or because host OS didn't scheduled VM, it would be nice improvement to implement pause detector in corosync. Corosync then displays warning message, when it was not scheduled for long enough time (0.8 * token_timeout). This will give us idea, if problem was network or scheduling and together with guest load monitoring should give as idea if problem was in guest or host.
Version-Release number of selected component (if applicable):
All
How reproducible:
100%
Steps to Reproduce:
1. Pause corosync
2. Unpause corosync
Actual results:
Token is lost, but it's not clear why.
Expected results:
Message "Corosync main process was not scheduled for X ms (threshold is Y ms). Consider token timeout increase." should be displayed.
Additional info:
Verified with scheduler-pause-detector.sh
FAIL on corosync-1.4.1-15.el6.x86_64 (RHEL6.4)
PASS on corosync-1.4.1-17.el6.x86_64 (RHEL6.5)
Sep 11 18:15:31 virt-014 corosync[5927]: [MAIN ] Corosync main process was not scheduled for 11066.0088 ms (threshold is 8000.0000 ms). Consider token timeout increase
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
http://rhn.redhat.com/errata/RHBA-2013-1531.html
Created attachment 732591 [details] Proposed patch Description of problem: When corosync is running in VM, it can happen, that VM (and thus corosync) is not scheduled for long time (longer then token timeout). Because in such environments, it's very hard to find out, if token was lost because of network overload or because of guest OS didn't scheduled corosync process or because host OS didn't scheduled VM, it would be nice improvement to implement pause detector in corosync. Corosync then displays warning message, when it was not scheduled for long enough time (0.8 * token_timeout). This will give us idea, if problem was network or scheduling and together with guest load monitoring should give as idea if problem was in guest or host. Version-Release number of selected component (if applicable): All How reproducible: 100% Steps to Reproduce: 1. Pause corosync 2. Unpause corosync Actual results: Token is lost, but it's not clear why. Expected results: Message "Corosync main process was not scheduled for X ms (threshold is Y ms). Consider token timeout increase." should be displayed. Additional info: