dhcp-libs-4.2.5-36.el7.x86_64
dhcp-common-4.2.5-36.el7.x86_64
dhcp-4.2.5-36.el7.x86_64


Description of problem:

I am doing a leap second vulnerability tests on HA deployment. RHOS6 A2.
I am running a script that inserts a leap second and checking if everything still works OK.  

I have HA deployment with 3 controllers and 2 compute nodes.
After some time running the leap second script on one of the controllers
I have noticed that it reboots. After investigating the issue I saw that it reboots because heart beat interface goes away and other cluster node fences the node where heart beat NIC is gone.

It happens only when I am running a leap second script and using DHCP.

It looks like a bug in DHCP client that can't deal with the leap second.
Dhcpclient takes the leap second as a "best to reconfigure the interface" in order to deal with dhcp timers.
In our case ,cluster's heart beat interface relays on DHCP. A fault in the dhcp server would take the cluster down.

dhcp server dies -> dhcp goes for a renewal -> can't renew -> IP goes away -> Cluster looses connectivity ti it's heart beat NIC--->Reboot.

In order to make sure that this is the problem I have re-configured cluster's heart beat interface not to use DHCP. I configured static IP and re-run the test.
No reboot occurs. Repeated test several times - no reboot. Everything work fine.

Here is /var/log/messages output before host reboots:
As you could see the failure related to corosync component.
------------------------------------------------------------------------------
    Apr  9 03:00:00 macf04da2732fb1 systemd: Time has been changed
        Apr  9 03:00:01 macf04da2732fb1 dhclient[1619]: DHCPDISCOVER on eno1 to 255.255.255.255 port 67 interval 8 (xid=0x8d3ac2)
        Apr  9 03:00:01 macf04da2732fb1 dhclient[1619]: DHCPREQUEST on eno1 to 255.255.255.255 port 67 (xid=0x8d3ac2)
        Apr  9 03:00:01 macf04da2732fb1 dhclient[1619]: DHCPOFFER from 192.168.0.1
        Apr  9 03:00:01 macf04da2732fb1 dhclient[1619]: DHCPACK from 192.168.0.1 (xid=0x8d3ac2)
        Apr 10 02:59:50 macf04da2732fb1 systemd: Time has been changed
        Apr 10 02:59:50 macf04da2732fb1 corosync[2777]: [TOTEM ] A processor failed, forming new configuration.
        Apr 10 02:59:50 macf04da2732fb1 corosync[2777]: [TOTEM ] The network interface is down.
        Apr 10 02:59:50 macf04da2732fb1 corosync[2777]: [TOTEM ] adding new UDPU member {192.168.0.3}
        Apr 10 02:59:50 macf04da2732fb1 corosync[2777]: [TOTEM ] adding new UDPU member {192.168.0.5}
        Apr 10 02:59:50 macf04da2732fb1 corosync[2777]: [TOTEM ] adding new UDPU member {192.168.0.6}
        Apr 10 02:59:50 macf04da2732fb1 Delay(ceilometer-delay)[105652]: INFO: Delay is running OK
        Apr 10 02:59:50 macf04da2732fb1 NET[106166]: /usr/sbin/dhclient-script : updated /etc/resolv.conf
        Apr 10 02:59:50 macf04da2732fb1 dhclient[1619]: bound to 192.168.0.5 -- renewal in -86093 seconds.
        Apr 10 02:59:50 macf04da2732fb1 dhclient[1619]: DHCPDISCOVER on eno1 to 255.255.255.255 port 67 interval 4 (xid=0x78f489f2)
        Apr 10 02:59:50 macf04da2732fb1 dhclient[1619]: DHCPREQUEST on eno1 to 255.255.255.255 port 67 (xid=0x78f489f2)
        Apr 10 02:59:50 macf04da2732fb1 dhclient[1619]: DHCPOFFER from 192.168.0.1
        Apr 10 02:59:50 macf04da2732fb1 dhclient[1619]: DHCPACK from 192.168.0.1 (xid=0x78f489f2)
        Apr 10 02:59:52 macf04da2732fb1 systemd-logind: Power key pressed.
        Apr 10 02:59:52 macf04da2732fb1 systemd-logind: Powering Off...
        Apr 10 02:59:52 macf04da2732fb1 systemd-logind: System is powering down.
        Apr 10 02:59:52 macf04da2732fb1 systemd: Stopping Session 86 of user root.
        Connection to macf04da2732fb1 closed by remote host.