Bug 1975577

Description of problem:
This issue was found while investigating bug 1973735. On a dual-stack setup, each interface gets an IPv6 link-local address plus only one routable address, either IPv4 or IPv6 but never both.

The Node IP is either IPv4 or IPv6, and if the address chosen for the Node (i.e. that the kubelet is listening on) does not appear in the HardwareDetails, then the Node cannot be linked to a Machine.

The issue is reminiscent of the one in bug 1946079 - in that case after deployment CoreOS could sometimes not fetch the Ignition file because of a change in the initrd in 4.6, where the network-manager dracut module replaced the network-legacy one, with a resulting change in default that it only waits for one IP address instead of always waiting for an IPv6 address: https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/issues/729

In this case the affected OS is RHEL, not RHCOS. However, RHEL made the same change in 8.3. I think the cluster in question was created after the RHEL 8.3 release, but it was a 4.4 cluster so I'm not sure what that means for the version used in the IPA image. Also in this case, IPA runs once the host is fully up, not as part of boot, so it's not clear that this should have the same effect. Perhaps this is always NetworkManager's default behaviour?

A possibly-relevant factor is that the interfaces in question are meant to be bonded. Since the IPA image has no custom network configuration (and thus sends no LACPDUs), the ports going up and down during boot can provoke long timeouts before LACP on the switches allow the links to come up, and this can delay obtaining an IP address over DHCP(v6).

How reproducible:
10-15% of Hosts see only IPv6 addresses

Steps to Reproduce:
1. Add BareMetalHost on IPv4/v6 network with NIC bonding
2. Wait for inspection
3. Check HardwareDetails for both IPv4 & IPv6 addresses on interface

Actual results:
Each interface gets an IPv6 link-local address plus *either* a routable IPv6 address or an IPv4 address from DHCP, but never both.
When the Node comes up, it cannot be linked to a Machine. As a result, CertificateSigningRequests for the Node are not signed.

Expected results:
Interfaces on networks with both DHCP and DHCPv6 should report both IP addresses via inspection. Nodes are linked to Machines, and CSRs get approved in fairly short order.

Additional info:
See bug 1973735 and the linked case.