Hide Forgot
Created attachment 1851812 [details] install-config.yaml Description of problem: Baremetal IPI deployment with IPv6 control plane and disabled provisioning network fails as the nodes do not pass introspection. Version-Release number of selected component (if applicable): quay.io/openshift-release-dev/ocp-release:4.10.0-fc.1-x86_64 How reproducible: 100% Steps to Reproduce: 1. Run baremetal IPI process with the attached install-config.yaml Actual results: time="2022-01-18T17:28:53-05:00" level=error time="2022-01-18T17:28:53-05:00" level=error msg="Error: could not inspect: could not inspect node, node is currently 'inspect failed' , last error was 'timeout reached while inspecting the node'" time="2022-01-18T17:28:53-05:00" level=error time="2022-01-18T17:28:53-05:00" level=error msg=" on ../../tmp/openshift-install-masters-1588391933/main.tf line 13, in resource \"ironic_node_v1\" \"openshift-master-host\":" time="2022-01-18T17:28:53-05:00" level=error msg=" 13: resource \"ironic_node_v1\" \"openshift-master-host\" {" time="2022-01-18T17:28:53-05:00" level=error time="2022-01-18T17:28:53-05:00" level=error time="2022-01-18T17:28:53-05:00" level=error time="2022-01-18T17:28:53-05:00" level=error msg="Error: could not inspect: could not inspect node, node is currently 'inspect failed' , last error was 'timeout reached while inspecting the node'" time="2022-01-18T17:28:53-05:00" level=error time="2022-01-18T17:28:53-05:00" level=error msg=" on ../../tmp/openshift-install-masters-1588391933/main.tf line 13, in resource \"ironic_node_v1\" \"openshift-master-host\":" time="2022-01-18T17:28:53-05:00" level=error msg=" 13: resource \"ironic_node_v1\" \"openshift-master-host\" {" time="2022-01-18T17:28:53-05:00" level=error time="2022-01-18T17:28:53-05:00" level=error time="2022-01-18T17:28:53-05:00" level=error time="2022-01-18T17:28:53-05:00" level=error msg="Error: could not inspect: could not inspect node, node is currently 'inspect failed' , last error was 'timeout reached while inspecting the node'" time="2022-01-18T17:28:53-05:00" level=error time="2022-01-18T17:28:53-05:00" level=error msg=" on ../../tmp/openshift-install-masters-1588391933/main.tf line 13, in resource \"ironic_node_v1\" \"openshift-master-host\":" time="2022-01-18T17:28:53-05:00" level=error msg=" 13: resource \"ironic_node_v1\" \"openshift-master-host\" {" time="2022-01-18T17:28:53-05:00" level=error time="2022-01-18T17:28:53-05:00" level=error Expected results: Deployment succeeds. Additional info: Nodes are unreachable via their reserved DHCP addresses and the console reports localhost in the prompt which indicates that the nodes didn't get and address via DHCP, attaching screenshot..
Created attachment 1851813 [details] introspection_console.png
Looks like the kernel commandline in grub has ip=dhcp, attached screenshot. When changing it to ip=dhcp6 the node obtains DHCP address correctly.
Created attachment 1851817 [details] grub kernel cmdline
The issue reproduced on 4.10.0-0.nightly-2022-01-18-044014 as well.
Derek has found the issue. https://github.com/openshift/installer/blob/08d043c78dc3feb74b3593645550b3a55aa35bff/data/data/bootstrap/baremetal/files/usr/local/bin/startironic.sh.template#L107-L111 miscalculates the IP options when provisioning network is disabled. The the wrong value is used for coreos-downloader: https://github.com/openshift/installer/blob/08d043c78dc3feb74b3593645550b3a55aa35bff/data/data/bootstrap/baremetal/files/usr/local/bin/startironic.sh.template#L116.
I assume it's a blocker since an important scenario is broken. The fix is on the way.
verified on 4.10.0-0.nightly-2022-01-22-102609
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:0056