Bug 1288929 – service pacemaker_remote stop causes node to be fenced

Bugzilla will be upgraded to version 5.0 on a still to be determined date in the near future. The original upgrade date has been delayed.

Bug 1288929 - service pacemaker_remote stop causes node to be fenced

Summary:	service pacemaker_remote stop causes node to be fenced

Status:	CLOSED ERRATA

Aliases:	None

Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	pacemaker (Show other bugs)
Sub Component:
Version:	7.2
Hardware:	Unspecified Unspecified

Priority	urgent Severity urgent
Target Milestone:	rc
Target Release:	7.3
Assigned To:	Ken Gaillot
QA Contact:	Asaf Hirshberg
Docs Contact:	Steven J. Levine

URL:
Whiteboard:
Keywords:	ZStream

Depends On:	1304771
Blocks:	1185030 1297564 1299348 1323259 1325009
	Show dependency tree / graph

Reported:	2015-12-06 22:37 EST by Andrew Beekhof
Modified:	2016-11-03 14:57 EDT (History)
CC List:	9 users (show)

See Also:
Fixed In Version:	pacemaker-1.1.15-1.2c148ac.git.el7
Doc Type:	Release Note
Doc Text:	Graceful migration of resources when the pacemaker_remote service is stopped on an active Pacemaker Remote node If the pacemaker_remote service is stopped on an active Pacemaker Remote node, the cluster will gracefully migrate resources off the node before stopping the node. Previously, Pacemaker Remote nodes were fenced when the service was stopped (including by commands such as "yum update"), unless the node was first explicitly taken out of the cluster. Software upgrades and other routine maintenance procedures are now much easier to perform on Pacemaker Remote nodes. Note: All nodes in the cluster must be upgraded to a version supporting this feature before it can be used on any node.
Story Points:	---
Clone Of:
Clones:	1297564 1299348 (view as bug list)
Environment:
Last Closed:	2016-11-03 14:57:31 EDT
Type:	Bug
Regression:	---
Mount Type:	---
Documentation:	---
CRM:
Verified Versions:
Category:	---
oVirt Team:	---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---

Attachments	(Terms of Use)
Add an attachment (proposed patch, testcase, etc.)

External Trackers
Tracker	ID	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHSA-2016:2578	normal	SHIPPED_LIVE	Moderate: pacemaker security, bug fix, and enhancement update	2016-11-03 08:07:24 EDT

Groups: None (edit)

Description Andrew Beekhof 2015-12-06 22:37:24 EST

Description of problem:

Graceful shutdown doesn't work

Version-Release number of selected component (if applicable):


How reproducible:

100%

Steps to Reproduce:
1. Run: service pacemaker_remote stop 

Actual results:

Fencing!

Expected results:

1. Contacts/notifies peer cluster
2. Peer cluster stops all services
3. Peer cluster tells pacemaker_remote it can shut down
4. Peer cluster recognises that the remote node was expected to shutdown

Additional info:

Esp. relevant for OSP upgrades

Comment 1 Raoul Scarazzini 2015-12-24 05:28:02 EST

As a side note, and as a workaround, the sequence of the commands we are using to avoid fencing is this one:

1) Reboot the compute node from console
2) Do a nova stop <computenodeid> from the undercloud
3) Do a nova start <computenodeid> from the undercloud
4) Do a cycle like this on one of the controller:
$ while true; do sudo pcs resource cleanup overcloud-novacompute-0; sleep 5; done
from a controller node
5) Once the machine is up, stop the cycle from step 4

Comment 2 Ken Gaillot 2016-01-08 16:29:33 EST

Fixed upstream as of commit da17fd0

Comment 4 Mike McCune 2016-03-28 18:52:26 EDT

This bug was accidentally moved from POST to MODIFIED via an error in automation, please see mmccune@redhat.com with any questions

Comment 6 Asaf Hirshberg 2016-06-06 05:41:59 EDT

Verified on RHEL-OSP director 9.0 puddle - 2016-06-03.1

Using "systemctl stop pacemaker_remote.service" stopping the service and pacemaker change the status to stopped:
overcloud-novacompute-1	(ocf::pacemaker:remote):	Stopped

When using kill command the status is changed to FAILED and the compute is fenced:
overcloud-novacompute-1	(ocf::pacemaker:remote):	FAILED



rpm:
pacemaker-debuginfo-1.1.13-10.el7_2.2.x86_64
pacemaker-1.1.13-10.el7_2.2.x86_64
pacemaker-libs-1.1.13-10.el7_2.2.x86_64
pacemaker-cluster-libs-1.1.13-10.el7_2.2.x86_64
pacemaker-cli-1.1.13-10.el7_2.2.x86_64
pacemaker-nagios-plugins-metadata-1.1.13-10.el7_2.2.x86_64
pacemaker-doc-1.1.13-10.el7_2.2.x86_64
pacemaker-remote-1.1.13-10.el7_2.2.x86_64

Comment 7 Steven J. Levine 2016-08-12 12:57:38 EDT

I'm putting in the same doc text that we had for this feature for RHEL 6.8 for the release notes.

Comment 9 errata-xmlrpc 2016-11-03 14:57:31 EDT

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2016-2578.html

Note You need to log in before you can comment on or make changes to this bug.