Bug 1801662 - On a DHCP6 lease renew, the node gets in NotReady state
Summary: On a DHCP6 lease renew, the node gets in NotReady state
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 4.4
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: 4.4.0
Assignee: Antoni Segura Puimedon
QA Contact: Victor Voronkov
Depends On:
Blocks: 1801638
TreeView+ depends on / blocked
Reported: 2020-02-11 13:08 UTC by Juan Manuel Parrilla Madrid
Modified: 2020-05-04 11:36 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1801638
Last Closed: 2020-05-04 11:35:29 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Github openshift machine-config-operator pull 1456 0 None closed Bug 1801662: baremetal: all resolvconf editing to NM dispatcher 2021-02-04 19:26:04 UTC
Red Hat Product Errata RHBA-2020:0581 0 None None None 2020-05-04 11:36:01 UTC

Description Juan Manuel Parrilla Madrid 2020-02-11 13:08:03 UTC
+++ This bug was initially created as a clone of Bug #1801638 +++

Description of problem:

When a master wants to renew the DHCP6 lease, it loses sone entries from resolv.conf which are the NS entry and the search XXXX statement. This causes that kubelet sets the node as NotReady with all the consecuences...

Version-Release number of selected component (if applicable):

How reproducible:

Deploy a IPv6 Disconnected Baremental cluster using this build: 4.3.0-0.nightly-2020-02-06-120247-ipv6.6, then wait, it will happen eventually.

Actual results:

NetworkManager triggers the prepender script (/etc/NetworkManager/dispatcher.d/30-resolv-prepender) and loses the main entries from resolv.conf

Expected results:

Have the resolv.conf like this:

# Generated by KNI resolv prepender NM dispatcher script
search xxx.xxx.xxx.xxx.xxx.redhat.com
nameserver fd35:919d:4042:2:c7ed:9a9f:a9ec:7  # <== VIP
nameserver fd35:919d:4042:2::1000             # <== dnsmasq

Additional info:

Build info: 4.3.0-0.nightly-2020-02-06-120247-ipv6.6

- 30-resolv-prepender:
# If $DHCP6_FQDN_FQDN contains a "-"
[[ "$DHCP6_FQDN_FQDN" =~ - ]] && hostname $DHCP6_FQDN_FQDN
case "$STATUS" in
    logger -s "NM resolv-prepender triggered by ${1} ${2}."
    set +e
    if [[ -n "$NAMESERVER_IP" ]]; then
        logger -s "NM resolv-prepender: Prepending 'nameserver $NAMESERVER_IP' to /etc/resolv.conf (other nameservers from /var/run/NetworkManager/resolv.conf)"
        sed "/^search .*$/a nameserver $NAMESERVER_IP" /var/run/NetworkManager/resolv.conf > /etc/resolv.conf
        logger -s "NM resolv-prepender: Couldn't find a Virtual IP, just updating resolv.conf"
        cp /var/run/NetworkManager/resolv.conf /etc/resolv.conf

- resolv.conf:
# Generated by NetworkManager
nameserver fd35:919d:4042:2::1000

- dhclient.conf:
supersede domain-search "kni7.cloud.lab.eng.bos.redhat.com";

Comment 1 Juan Manuel Parrilla Madrid 2020-02-12 16:29:57 UTC
Working fine on 4.3.0-0.nightly-2020-02-10-055634-ipv6.3 (validated by my side)

Comment 3 Victor Voronkov 2020-03-12 09:32:09 UTC
Verified on 4.4.0-0.ci-2020-03-11-095511 with IPv6

[core@master-0 ~]$ sudo cat /var/lib/NetworkManager/dhclient6-10388e58-59dd-48e2-abf4-d42411e9d79b-enp5s0.lease | grep enp5s0
  interface "enp5s0";
  interface "enp5s0";
  interface "enp5s0";
  interface "enp5s0";
  interface "enp5s0";
  interface "enp5s0";
  interface "enp5s0";

[core@master-0 ~]$ cat /etc/resolv.conf 
# Generated by KNI resolv prepender NM dispatcher script
search ocp-edge-cluster.qe.lab.redhat.com
nameserver fd2e:6f44:5dd8:c956:0:0:0:2
nameserver fe80::5054:ff:fec0:8d50%enp5s0
nameserver fd2e:6f44:5dd8:c956::1

[kni@provisionhost-0 ~]$ oc get nodes
NAME                                          STATUS   ROLES    AGE    VERSION
master-0.ocp-edge-cluster.qe.lab.redhat.com   Ready    master   141m   v1.17.1.lease" [readonly] 157L, 4788C
master-1.ocp-edge-cluster.qe.lab.redhat.com   Ready    master   141m   v1.17.1
master-2.ocp-edge-cluster.qe.lab.redhat.com   Ready    master   141m   v1.17.1
worker-0.ocp-edge-cluster.qe.lab.redhat.com   Ready    worker   122m   v1.17.1
worker-1.ocp-edge-cluster.qe.lab.redhat.com   Ready    worker   121m   v1.17.1

Comment 5 errata-xmlrpc 2020-05-04 11:35:29 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.