Bug 2048600

Currently this configuration isn't automated via the installer, so it will be necessary to either set the bootstrap VM external IP via DHCP, or configure it via ignition e.g:

https://github.com/openshift/installer/blob/master/docs/user/customization.md#os-customization-unvalidated

This means creating a NetworkManager keyfile for the external NIC of the boostrap VM:

  BOOTSTRAP_CONFIG="[connection]
  type=ethernet
  interface-name=ens3
  [ethernet]
  [ipv4]
  method=manual
  addresses=${BOOTSTRAP_IP}/${BOOTSTRAP_PREFIX}
  gateway=${PROVISIONING_HOST_EXTERNAL_IP}
  dns=${PROVISIONING_HOST_EXTERNAL_IP}"

And merge additional config into the generated bootstrap.ign before deployment e.g:

  cat > bootstrap_network_config.ign << EOF
      {
        "path": "/etc/NetworkManager/system-connections/ens3.nmconnection",
        "mode": 384,
        "contents": {
        "source": "data:text/plain;charset=utf-8;base64,$(echo "${BOOTSTRAP_CONFIG}" | base64 -w 0)"
        }
      }
  EOF


  openshift-baremetal-install --dir foo create ignition-configs
  mv foo/bootstrap.ign foo/bootstrap.ign.orig
  jq '.storage.files += $input' foo/bootstrap.ign.orig --slurpfile input bootstrap_network_config.ign > foo/bootstrap.ign
  openshift-baremetal-install --dir foo create cluster


In future it may make sense to add an install-config option similar to `bootstrapProvisioningIP` e.g `bootstrapExternalIP` which templates this into the ignition file automatically - we also need to decide if we want to document this workaround in the meantime.

Removed blocker flag. Adina - please try the workaround that Steve has described.

Adina, are you able to try the workaround described in Comment 1? https://bugzilla.redhat.com/show_bug.cgi?id=2048600#c1

Or are you blocked by something else to try that?

Ok Thanks for the input @djuran . We will test this and update how it goes.

@djuran @dmoessne 
I just tested a deployment on VMs with no dhcp server and no dhcp server. 
The deployment passed successfully and the networking configuration looks correct.

yes, I have tested this as well and as long as there is no dhcp server running on the network it works.
However if there is a dhcp server running dynamic ip is preferred and hence the cluster install fails. When I tested masters were fine, but worker got dynamic IP.

I think that is a bug, but I think a different one than reported here

Thanks @dmoessne. 
I will try to test without DHCP, as well and open a seperate bug for that.

@dmoessne Did you test this on real BM or VM?

(In reply to Adina Wolff from comment #10)
> @djuran @dmoessne 
> I just tested a deployment on VMs with no dhcp server and no dhcp server. 
> The deployment passed successfully and the networking configuration looks
> correct.

This was meant to say no dhcp server and no provisioning network.

I think also a decision needs to be made as to how long this workaround is meant for and what the long term plan is. 
Steve had written this in commet 1:
In future it may make sense to add an install-config option similar to `bootstrapProvisioningIP` e.g `bootstrapExternalIP` which templates this into the ignition file automatically - we also need to decide if we want to document this workaround in the meantime.

Yoav from my team opened a BZ for all issues that need documentation: bz2060306

(In reply to daniel from comment #11)
> yes, I have tested this as well and as long as there is no dhcp server
> running on the network it works.
> However if there is a dhcp server running dynamic ip is preferred and hence
> the cluster install fails. When I tested masters were fine, but worker got
> dynamic IP.
> 
> I think that is a bug, but I think a different one than reported here

Yes, this is a bug.  I don't understand how it's happening though.. anyway,
is there a BZ open for this already?  If not we'll probably need one.

Thanks!

This PR should be enough to remove the workaround: https://github.com/openshift/installer/pull/5787

Deployed 4.11 using versioned-installer-packet_libvirt-bootstrap_static-ci template 

Checked that BS vm interfaces are assigned static IPs

[core@localhost ~]$ nmcli connection show nmconnection
connection.id:                          nmconnection
connection.uuid:                        84c43c16-9fd5-3ed0-8e99-6f52fd2322ff
connection.stable-id:                   --
connection.type:                        802-3-ethernet
connection.interface-name:              --
connection.autoconnect:                 yes
802-3-ethernet.accept-all-mac-addresses:-1 (default)
ipv4.method:                            manual
ipv4.dns:                               192.168.111.1

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: OpenShift Container Platform 4.11.27 security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:0651

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days