Staypuft uses the controller_ips method in several places to template
class parameters, one of the most important being
pacemaker_cluster_members. It is imperative that this method returns
a consistent list of IP addresses, because it directly drives the
pacemaker configuration file, and any change will break the cluster.
Presently, staypuft generates this list using the Host.ip value for
each host in the deployment. This is from the Foreman Host model.
Foreman maintains the value of ip based on the value provided in the
ipaddress fact from facter. In turn, facter takes the value of
ipaddress to be the first address encountered in the output of
ifconfig that does not match /^127./.
Here's what we've observed happening during an HA deployment:
- All three controllers provision on the management network, and run
puppet during kickstart %post. The ipaddress fact for each is the
management network address, so this is correct so far.
- Hosts reboot, run puppet. At this point it gets a little ambiguous.
If the management interface is the first alphabetically, it will
still be chosen as the ipaddress fact and the foreman host will
still be correct. However it's possible that the host provisioned
off of eth1, and now the host has brought up eth0 as a DHCP
interface. In that case, ipaddress will be the eth0 address, and
foreman will update the host ip accordingly. This will cause
controller_ips to change on the next puppet run.
- If we manage to get through the first puppet runs correctly across
all controllers (e.g. eth0 was in fact the management interface),
then the second time puppet runs the external bridge address will
almost certainly take over the host ip, since br-ex comes before eth
alphabetically. The next puppet run will cause the controller_ips
value to change to reflect this, and the pacemaker cluster will
Moving back to ON_DEV pending a new fix. We're going to revert foreman PR 258 and push a fix in the installer instead. Installer fix is merged already:
Controller IPs stay the same.
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.