Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
When pacemaker executes an action, there is some behind-the-scenes communication:
* The controller on the DC decides the action is required, and tells the controller on the relevant node what to do
* The controller on the relevant node receives the request, and passes it to its local executor
* The executor on the relevant node receives the request, and executes the resource agent
* The result is passed back up the chain (from executor to local controller to DC controller)
Usually the only timeout visible to the user is if the resource agent itself runs longer than its configured timeout. The daemon-to-daemon communication is brief and reliable. However it is possible for that communication to time out, which is happening here. It indicates something really unusual is happening, which I haven't isolated yet.
To answer the side questions: The stonith resource was started successfully, and a recurring monitor was initiated. That monitor had a communication timeout; I don't see any indication that the request was even received by the controller on the relevant node. That timeout necessitated a stop of the resource, which also had a communication timeout -- this time, the stop was received by the relevant node, but we lose the logs after that due to the fencing, so I am not sure exactly which step timed out (it wasn't the stop itself but one of the communication steps). The failed stop then necessitated the node being fenced.
This issue will be fixed only in RHEL 8 (Bug 1856015) because of the late stage of RHEL 7's life cycle, the availability of a workaround (explicitly configuring a higher start timeout), and technical challenges in backporting the fix to the RHEL 7 code base.