Bug 2048600 - Networking Day 1 - Bootstrap Doesn't Get External IP when no DHCP Server
Summary: Networking Day 1 - Bootstrap Doesn't Get External IP when no DHCP Server
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 4.10
Hardware: Unspecified
OS: Linux
urgent
urgent
Target Milestone: ---
: 4.11.0
Assignee: Honza Pokorny
QA Contact: Jad Haj Yahya
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-01-31 15:20 UTC by Adina Wolff
Modified: 2023-09-18 04:31 UTC (History)
16 users (show)

Fixed In Version:
Doc Type: Known Issue
Doc Text:
Cause: During installation of a baremetal IPI cluster, you can use the networkConfig field in the install-config.yaml to configure the control-plane network interfaces for the cluster hosts, but currently you cannot configure the bootstrap VM networking via the install-config. Consequence: In an environment where no DHCP server is running, the bootstrap VM will not get an IP on the controlplane NIC, causing cluster installation to fail. Workaround (if any): The bootstrap VM controlplane NIC can be configured via the installer create ignition-configs target, which allows the bootstrap ignition file to be modified, adding a static network configuration, for example: Create a NetworkManager keyfile for the external NIC of the bootstrap VM: BOOTSTRAP_CONFIG="[connection] type=ethernet interface-name=ens3 [ethernet] [ipv4] method=manual addresses=${BOOTSTRAP_IP}/${BOOTSTRAP_PREFIX} gateway=${PROVISIONING_HOST_EXTERNAL_IP} dns=${PROVISIONING_HOST_EXTERNAL_IP}" And merge additional ignition config into the generated bootstrap.ign before deployment e.g: cat > bootstrap_network_config.ign << EOF { "path": "/etc/NetworkManager/system-connections/ens3.nmconnection", "mode": 384, "contents": { "source": "data:text/plain;charset=utf-8;base64,$(echo "${BOOTSTRAP_CONFIG}" | base64 -w 0)" } } EOF openshift-baremetal-install --dir foo create ignition-configs mv foo/bootstrap.ign foo/bootstrap.ign.orig jq '.storage.files += $input' foo/bootstrap.ign.orig --slurpfile input bootstrap_network_config.ign > foo/bootstrap.ign openshift-baremetal-install --dir foo create cluster Result: With the workaround applied, it is possible to deploy a baremetal IPI cluster in an environment where no DHCP server is running.
Clone Of:
Environment:
Last Closed: 2023-02-15 07:43:17 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift installer pull 5787 0 None Merged baremetal: static IP for bootstrap node 2023-01-31 13:26:30 UTC
Red Hat Product Errata RHSA-2023:0651 0 None None None 2023-02-15 07:43:19 UTC

Description Adina Wolff 2022-01-31 15:20:27 UTC
Version:

4.10

Platform:

baremetal

Please specify:
* IPI 

What happened?

Networking Day1 (metal-1) Epic is meant to allow ipi deployment to run on an environment without a DHCP server.
Bootstrap vm will not get an external (baremetal-0) ip address if there is no DHCP server.

Comment 1 Steven Hardy 2022-02-01 11:16:42 UTC
Currently this configuration isn't automated via the installer, so it will be necessary to either set the bootstrap VM external IP via DHCP, or configure it via ignition e.g:

https://github.com/openshift/installer/blob/master/docs/user/customization.md#os-customization-unvalidated

This means creating a NetworkManager keyfile for the external NIC of the boostrap VM:

  BOOTSTRAP_CONFIG="[connection]
  type=ethernet
  interface-name=ens3
  [ethernet]
  [ipv4]
  method=manual
  addresses=${BOOTSTRAP_IP}/${BOOTSTRAP_PREFIX}
  gateway=${PROVISIONING_HOST_EXTERNAL_IP}
  dns=${PROVISIONING_HOST_EXTERNAL_IP}"

And merge additional config into the generated bootstrap.ign before deployment e.g:

  cat > bootstrap_network_config.ign << EOF
      {
        "path": "/etc/NetworkManager/system-connections/ens3.nmconnection",
        "mode": 384,
        "contents": {
        "source": "data:text/plain;charset=utf-8;base64,$(echo "${BOOTSTRAP_CONFIG}" | base64 -w 0)"
        }
      }
  EOF


  openshift-baremetal-install --dir foo create ignition-configs
  mv foo/bootstrap.ign foo/bootstrap.ign.orig
  jq '.storage.files += $input' foo/bootstrap.ign.orig --slurpfile input bootstrap_network_config.ign > foo/bootstrap.ign
  openshift-baremetal-install --dir foo create cluster


In future it may make sense to add an install-config option similar to `bootstrapProvisioningIP` e.g `bootstrapExternalIP` which templates this into the ignition file automatically - we also need to decide if we want to document this workaround in the meantime.

Comment 2 Bob Fournier 2022-02-01 17:13:11 UTC
Removed blocker flag. Adina - please try the workaround that Steve has described.

Comment 4 Tomas Sedovic 2022-02-07 11:17:49 UTC
Adina, are you able to try the workaround described in Comment 1? https://bugzilla.redhat.com/show_bug.cgi?id=2048600#c1

Or are you blocked by something else to try that?

Comment 9 Adina Wolff 2022-02-14 15:45:08 UTC
Ok Thanks for the input @djuran . We will test this and update how it goes.

Comment 10 Adina Wolff 2022-02-24 10:06:43 UTC
@djuran @dmoessne 
I just tested a deployment on VMs with no dhcp server and no dhcp server. 
The deployment passed successfully and the networking configuration looks correct.

Comment 11 daniel 2022-02-24 15:32:59 UTC
yes, I have tested this as well and as long as there is no dhcp server running on the network it works.
However if there is a dhcp server running dynamic ip is preferred and hence the cluster install fails. When I tested masters were fine, but worker got dynamic IP.

I think that is a bug, but I think a different one than reported here

Comment 12 Adina Wolff 2022-02-24 17:08:42 UTC
Thanks @dmoessne. 
I will try to test without DHCP, as well and open a seperate bug for that.

Comment 13 Adina Wolff 2022-02-24 18:47:15 UTC
@dmoessne Did you test this on real BM or VM?

Comment 14 Adina Wolff 2022-02-28 05:50:27 UTC
(In reply to Adina Wolff from comment #10)
> @djuran @dmoessne 
> I just tested a deployment on VMs with no dhcp server and no dhcp server. 
> The deployment passed successfully and the networking configuration looks
> correct.

This was meant to say no dhcp server and no provisioning network.

Comment 18 Adina Wolff 2022-03-03 10:48:03 UTC
I think also a decision needs to be made as to how long this workaround is meant for and what the long term plan is. 
Steve had written this in commet 1:
In future it may make sense to add an install-config option similar to `bootstrapProvisioningIP` e.g `bootstrapExternalIP` which templates this into the ignition file automatically - we also need to decide if we want to document this workaround in the meantime.

Comment 20 Adina Wolff 2022-03-03 14:35:39 UTC
Yoav from my team opened a BZ for all issues that need documentation: bz2060306

Comment 21 Ian Main 2022-03-03 17:17:25 UTC
(In reply to daniel from comment #11)
> yes, I have tested this as well and as long as there is no dhcp server
> running on the network it works.
> However if there is a dhcp server running dynamic ip is preferred and hence
> the cluster install fails. When I tested masters were fine, but worker got
> dynamic IP.
> 
> I think that is a bug, but I think a different one than reported here

Yes, this is a bug.  I don't understand how it's happening though.. anyway,
is there a BZ open for this already?  If not we'll probably need one.

Thanks!

Comment 25 Tomas Sedovic 2022-05-17 09:14:17 UTC
This PR should be enough to remove the workaround: https://github.com/openshift/installer/pull/5787

Comment 30 Jad Haj Yahya 2023-02-12 06:07:54 UTC
Deployed 4.11 using versioned-installer-packet_libvirt-bootstrap_static-ci template 

Checked that BS vm interfaces are assigned static IPs

[core@localhost ~]$ nmcli connection show nmconnection
connection.id:                          nmconnection
connection.uuid:                        84c43c16-9fd5-3ed0-8e99-6f52fd2322ff
connection.stable-id:                   --
connection.type:                        802-3-ethernet
connection.interface-name:              --
connection.autoconnect:                 yes
802-3-ethernet.accept-all-mac-addresses:-1 (default)
ipv4.method:                            manual
ipv4.dns:                               192.168.111.1

Comment 32 errata-xmlrpc 2023-02-15 07:43:17 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: OpenShift Container Platform 4.11.27 security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:0651

Comment 33 Red Hat Bugzilla 2023-09-18 04:31:03 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days


Note You need to log in before you can comment on or make changes to this bug.