Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 2102739

Summary: Baremetal IPI Installation with ipv6 fails on because of missing resolv.conf on masters
Product: OpenShift Container Platform Reporter: Karim Boumedhel <kboumedh>
Component: InstallerAssignee: Ben Nemec <bnemec>
Installer sub component: OpenShift on Bare Metal IPI QA Contact: Amit Ugol <augol>
Status: CLOSED DUPLICATE Docs Contact:
Severity: high    
Priority: urgent CC: bnemec, dmellado, mavazque, shardy
Version: 4.11   
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-07-11 21:34:00 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Karim Boumedhel 2022-06-30 13:53:15 UTC
Version:

$ openshift-install version

openshift-baremetal-install 4.11.0-0.nightly-2022-06-23-044003
built from commit 8c3216b957108286f52e21210cad180ba405bb6f
release image quay.io/openshift-release-dev/ocp-release-nightly@sha256:5cc4c2912c03b49bf9b8b2b4e36319c467c1483f30689291a87e05097e6844bb
release architecture amd64

Platform:

#Please specify the platform type: baremetal

Please specify:
* IPI

What happened?

an install using disconnected with ipv6 fails because resolv.conf is not present on masters.

Comment 1 Karim Boumedhel 2022-07-03 19:06:41 UTC
i feel it has a relation with the file /etc/NetworkManager/conf.d/99-kni.conf, with the following content

[main]
rc-manager=unmanaged
[connection]
ipv6.dhcp-duid=ll
ipv6.dhcp-iaid=mac

Comment 2 Dmitry Tantsur 2022-07-05 14:52:24 UTC
Could you please provide some logs? I assume must-gather is not possible, maybe journalctl, container logs, anything? How did you come to the conclusion about 99-kni.conf?

Comment 3 Karim Boumedhel 2022-07-05 21:24:01 UTC
the same file seems to work fine with previous releases, so I rather think it's a change of behaviour in NetworkManager (1.36 in rhcos4.11 vs 1.30 in rhcos4.10)

Comment 4 Karim Boumedhel 2022-07-05 21:24:26 UTC
i can provide a live environment with the issue when needed, just ping me

Comment 5 Steven Hardy 2022-07-06 09:26:41 UTC
> i feel it has a relation with the file /etc/NetworkManager/conf.d/99-kni.conf, with the following content

You may be right, but as already mentioned we'll need logs or some live-debug to figure out what's wrong.

The `rc-manager=unmanaged` is because we have a NM dispatcher script that discovers the DNS servers via DHCP, then prepends localhost for the local coredns service (which resolves cluster-internal names)

https://github.com/openshift/machine-config-operator/blob/master/templates/common/on-prem/files/NetworkManager-resolv-prepender.yaml

To see the logs for that script check the NetworkManager-dispatcher.service journal on the masters, if you can attach the logs here we may be able to spot some clues, thanks!

Comment 8 Karim Boumedhel 2022-07-07 22:12:32 UTC
using the following machineconfig (to set rc-manager=file) allows the install to work just fine 

```
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
  name: empty-99-kni
  labels:
    machineconfiguration.openshift.io/role: master
spec:
  config:
    ignition:
      version: 2.2.0
    storage:
      files:
        - contents:
            source: data:text/plain;charset=utf-8;base64,W21haW5dCnJjLW1hbmFnZXI9ZmlsZQpbY29ubmVjdGlvbl0KaXB2Ni5kaGNwLWR1aWQ9bGwKaXB2Ni5kaGNwLWlhaWQ9bWFjCg==
            verification: {}
          filesystem: root
          mode: 420
          path: /etc/NetworkManager/conf.d/99-kni.conf
```

Comment 9 Tomas Sedovic 2022-07-11 16:24:35 UTC
This seems to be a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=2105069. Ben, can you please confirm and close it as such if so?

I've assigned it to you as it seems to be within the metal networking wheelhouse. Please flip it back if incorrect.

Comment 10 Ben Nemec 2022-07-11 21:34:00 UTC
Yeah, sorry, I meant to mark this as duplicate when I first got pinged.

*** This bug has been marked as a duplicate of bug 2105069 ***