Bug 1127752 - controller_ips changes during deployment lifetime
Summary: controller_ips changes during deployment lifetime
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: ruby193-rubygem-staypuft
Version: Foreman (RHEL 6)
Hardware: Unspecified
OS: Unspecified
Target Milestone: ga
: Installer
Assignee: John Eckersberg
QA Contact: Leonid Natapov
Depends On:
TreeView+ depends on / blocked
Reported: 2014-08-07 13:22 UTC by John Eckersberg
Modified: 2014-08-21 18:08 UTC (History)
5 users (show)

Fixed In Version: ruby193-rubygem-staypuft-0.2.1-1.el6ost
Doc Type: Bug Fix
Doc Text:
Clone Of:
Last Closed: 2014-08-21 18:08:26 UTC

Attachments (Terms of Use)

System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2014:1090 normal SHIPPED_LIVE Red Hat Enterprise Linux OpenStack Platform Enhancement Advisory 2014-08-22 15:28:08 UTC

Description John Eckersberg 2014-08-07 13:22:08 UTC
Staypuft uses the controller_ips method in several places to template
class parameters, one of the most important being
pacemaker_cluster_members.  It is imperative that this method returns
a consistent list of IP addresses, because it directly drives the
pacemaker configuration file, and any change will break the cluster.

Presently, staypuft generates this list using the Host.ip value for
each host in the deployment.  This is from the Foreman Host model.
Foreman maintains the value of ip based on the value provided in the
ipaddress fact from facter.  In turn, facter takes the value of
ipaddress to be the first address encountered in the output of
ifconfig that does not match /^127./.

Here's what we've observed happening during an HA deployment:

- All three controllers provision on the management network, and run
  puppet during kickstart %post.  The ipaddress fact for each is the
  management network address, so this is correct so far.

- Hosts reboot, run puppet.  At this point it gets a little ambiguous.
  If the management interface is the first alphabetically, it will
  still be chosen as the ipaddress fact and the foreman host will
  still be correct.  However it's possible that the host provisioned
  off of eth1, and now the host has brought up eth0 as a DHCP
  interface.  In that case, ipaddress will be the eth0 address, and
  foreman will update the host ip accordingly.  This will cause
  controller_ips to change on the next puppet run.

- If we manage to get through the first puppet runs correctly across
  all controllers (e.g. eth0 was in fact the management interface),
  then the second time puppet runs the external bridge address will
  almost certainly take over the host ip, since br-ex comes before eth
  alphabetically.  The next puppet run will cause the controller_ips
  value to change to reflect this, and the pacemaker cluster will

Comment 3 John Eckersberg 2014-08-07 20:56:49 UTC

Comment 5 Scott Seago 2014-08-11 14:31:24 UTC
Moving back to ON_DEV pending a new fix. We're going to revert foreman PR 258 and push a fix in the installer instead. Installer fix is merged already:


Comment 7 Leonid Natapov 2014-08-21 05:17:14 UTC
Controller IPs stay the same.

Comment 8 errata-xmlrpc 2014-08-21 18:08:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.