Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
Bug 1237329 - Overcloud: HA: Pacemaker and ironic fighting for control causing fencing to fail when rebooting/powering off the node.
Overcloud: HA: Pacemaker and ironic fighting for control causing fencing to f...
Status: CLOSED ERRATA
Product: Red Hat OpenStack
Classification: Red Hat
Component: rhosp-director (Show other bugs)
Director
Unspecified Unspecified
high Severity high
: ga
: Director
Assigned To: Lucas Alvares Gomes
Leonid Natapov
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-06-30 16:05 EDT by Leonid Natapov
Modified: 2015-08-05 09:57 EDT (History)
11 users (show)

See Also:
Fixed In Version: instack-undercloud-2.1.2-14
Doc Type: Bug Fix
Doc Text:
Pacemaker and ironic fought for control over power management, which caused issues with fencing. This fix sets force_power_state_during_sync=False in /etc/ironic/ironic.conf by default. This stops ironic automatically restoring the power state of the node during its synchronization. Pacemaker can now successfully fence the node.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-08-05 09:57:56 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Gerrithub.io 238337 None None None Never
Red Hat Product Errata RHEA-2015:1549 normal SHIPPED_LIVE Red Hat Enterprise Linux OpenStack Platform director Release 2015-08-05 13:49:10 EDT

  None (edit)
Description Leonid Natapov 2015-06-30 16:05:40 EDT
Overcloud: HA: Pacemaker and ironic fighting for control causing fencing to fail when using ironic to power off  the node.

pacemaker is trying to turn the node off (or on) using fencing while ironic is trying to return the node to correct state.

We have to  configure Ironic to *not* try to restore the node to a correct power state and let the pacemaker do the job.
Comment 2 Leonid Natapov 2015-07-01 09:20:27 EDT
The work around:
1.on the instack node edit /etc/ironic/ironic.conf file
2.Uncomment force_power_state_during_sync and set it to false.
3.restart openstack-ironic-conductor service.
Comment 3 chris alfonso 2015-07-01 13:23:47 EDT
Lucas, can you make sure you default to having the force_power_state_during_sync to false and make sure there are docs describing how to turn it on if needed?
Comment 4 Lucas Alvares Gomes 2015-07-02 09:01:30 EDT
@Chris, will do!

@Leonid, that's not a workaround, that's the right way to do. Ironic by default will try to make sure that the machines are in sync with the database state. But we made it configurable for this kinda of situations.
Comment 5 Lucas Alvares Gomes 2015-07-02 10:04:48 EDT
Btw, we don't have any fencing agent for Ironic in Peacemaker right?

If Peacemaker used the Ironic interface to power on and off the nodes it's fencing this problem wouldn't happen.
Comment 6 James Slagle 2015-07-02 12:44:15 EDT
Lucas, a question on the gerrithub review about the use of tabs there. If that needs to be fixed, can you update the patch?

once it's in shape, please submit to code.engineering as well.
Comment 8 James Slagle 2015-07-07 07:08:00 EDT
upstream and downstream patches merged
Comment 10 Leonid Natapov 2015-07-20 10:33:55 EDT
ironic.conf includes force_power_state_during_sync=False

instack-undercloud-2.1.2-21.el7ost.noarch
Comment 12 errata-xmlrpc 2015-08-05 09:57:56 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2015:1549

Note You need to log in before you can comment on or make changes to this bug.