Bug 1785122 - [vSphere] [UPI] - Cannot provide Static IP addresses when using OVA
Summary: [vSphere] [UPI] - Cannot provide Static IP addresses when using OVA
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: RHCOS
Version: 4.2.z
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.6.0
Assignee: Luca BRUNO
QA Contact: Michael Nguyen
URL:
Whiteboard:
Depends On:
Blocks: 1186913 1819833
TreeView+ depends on / blocked
 
Reported: 2019-12-19 08:31 UTC by Anoel Yakoubov
Modified: 2023-12-15 17:06 UTC (History)
18 users (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
Feature: It is now possible to override the default DHCP networking on first boot, by setting the relevant guestinfo property before booting a VM. Reason: This lowers the barrier for automatic RHCOS deployment in environments where DHCP is not provided (and Ignition config requires networking). Result: Higher-level automation can provision RHCOS OVA in environments with static networking.
Clone Of:
Environment:
Last Closed: 2020-10-27 15:54:52 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
VMX file of the relevant VM (3.64 KB, text/plain)
2019-12-19 08:31 UTC, Anoel Yakoubov
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2020:4196 0 None None None 2020-10-27 15:55:19 UTC

Description Anoel Yakoubov 2019-12-19 08:31:40 UTC
Created attachment 1646329 [details]
VMX file of the relevant VM

Description of problem:


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Anoel Yakoubov 2019-12-19 08:57:22 UTC
Description of problem:
I am working in Very Big and Classified customer and this customer must to work with Static IP addresses and this solution should be supported with CoreOS4.2 per our documentation
in https://bugzilla.redhat.com/show_bug.cgi?id=1755047, we are trying to provide Static IPs by using VM options--> advanced--> edit configuration and adding the relevant info (PSA)
We can see all info in right format in VM's vmx file, but still in boot VM can't receive IP and then can't load ign file accordingly. Customer must use OVA for VMs provisioning since we need to 

Version-Release number of selected component (if applicable):
RHCOS 4.2, OCP 4.2.9

How reproducible:
Can be reproduced easy

Steps to Reproduce:
1. Create VM from OVA
2. Add all relevant Guestinfo properties by using the example

guestinfo.hostname = "coreos"
guestinfo.interface.0.role = "private"
guestinfo.dns.server.0 = "8.8.8.8"
guestinfo.interface.0.route.0.gateway = "192.168.178.1"
guestinfo.interface.0.route.0.destination = "0.0.0.0/0"
guestinfo.interface.0.mac = "00:0c:29:63:92:5c"
guestinfo.interface.0.name = "eno1*"
guestinfo.interface.0.dhcp = "no"
guestinfo.interface.0.ip.0.address = "192.168.178.97/24"

3. Power On VM

Actual results: VM cannot load ign file and complete boot since it not receiving IP and have error "Network unreachable"


Expected results:


VM receive Static IP, load ign file, complete boot

Additional info:

It is very crucial for customer to be able to provide static IPs without managing DHCP, without managing mac addresses, with usage of OVA.

Comment 2 Anoel Yakoubov 2019-12-19 08:57:53 UTC
Description of problem:
I am working in Very Big and Classified customer and this customer must to work with Static IP addresses and this solution should be supported with CoreOS4.2 per our documentation
in https://bugzilla.redhat.com/show_bug.cgi?id=1755047, we are trying to provide Static IPs by using VM options--> advanced--> edit configuration and adding the relevant info (PSA)
We can see all info in right format in VM's vmx file, but still in boot VM can't receive IP and then can't load ign file accordingly. Customer must use OVA for VMs provisioning since we need to 

Version-Release number of selected component (if applicable):
RHCOS 4.2, OCP 4.2.9

How reproducible:
Can be reproduced easy

Steps to Reproduce:
1. Create VM from OVA
2. Add all relevant Guestinfo properties by using the example

guestinfo.hostname = "coreos"
guestinfo.interface.0.role = "private"
guestinfo.dns.server.0 = "8.8.8.8"
guestinfo.interface.0.route.0.gateway = "192.168.178.1"
guestinfo.interface.0.route.0.destination = "0.0.0.0/0"
guestinfo.interface.0.mac = "00:0c:29:63:92:5c"
guestinfo.interface.0.name = "eno1*"
guestinfo.interface.0.dhcp = "no"
guestinfo.interface.0.ip.0.address = "192.168.178.97/24"

3. Power On VM

Actual results: VM cannot load ign file and complete boot since it not receiving IP and have error "Network unreachable"


Expected results:


VM receive Static IP, load ign file, complete boot

Additional info:

It is very crucial for customer to be able to provide static IPs without managing DHCP, without managing mac addresses, with usage of OVA.

Comment 3 Micah Abbott 2019-12-19 15:57:43 UTC
Static IP assignment on RHCOS nodes should be done via the kernel command line when doing the initial install of the RHCOS nodes, before the fetching of Ignition is done.

See the following blog post for more details - https://blog.openshift.com/openshift-4-2-vsphere-install-with-static-ips/

Alternately see the following KBase article - https://access.redhat.com/solutions/4531011

Comment 4 Anoel Yakoubov 2019-12-21 07:52:45 UTC
Hi, I am familiar with both solutions, but in our case, as I mentioned before we must to use OVA Template and can't use ISO file, since we are doing automation of this deployment to edge locations, so solution should support usage of OVA provided in our cloud.redhat.com site

Comment 6 linux.squad.redhat.mail 2020-02-24 14:16:20 UTC
Hi
My name is Stav and I'm from a big company in Israel , running in a restricted network.
I have been trying to install OCP4.3 as well from OVA template, using static ip.

I've managed to do that only by using OVA and then editing grub configuration manually with the following parameter:
ip=<host_ip>::<default GW>:<netmask>:<fqdn>:ens192:none nameserver=<dns1>

We would like a more automated way to use static ip, since our network team doesn't work with DHCP for servers.

we deploy the VMs with the terraform module from the openshift/installer github repository but it doesn't really matter.

During initramfs the RHCOS uses DHCP by default.

Comment 7 Micah Abbott 2020-02-26 19:11:38 UTC
The RHCOS team recognizes that the UX for configuring static IPs in a vSphere environment is difficult and we are actively working on improving this experience for the 4.5 release.

Comment 8 Rodrigo Alvares 2020-04-15 17:31:29 UTC
Hello All,

I know it is not related to the OVA.

few months ago I managed to get a automated installation on Vmware using static IPs by building a custom ipxe.iso image which is capable of reading vmware guestinfo.

Please find the link below:

https://github.com/ralvares/ipxe

Best Regards

Comment 9 Dusty Mabe 2020-05-18 15:34:18 UTC
This should be fixed with some feature work Luca is doing that will land in 4.6.

Comment 11 Micah Abbott 2020-06-20 13:32:02 UTC
The fix landed in afterburn-4.4.0-1.rhaos4.6.el8 and is now available in 4.6 RHCOS builds

Setting to MODIFIED

Comment 15 linux.squad.redhat.mail 2020-06-22 19:21:50 UTC
Thanks for the update Micah.
Will it be documented in the official documentaion of future OCP 4.6 version?

Comment 17 Michael Nguyen 2020-07-08 17:00:52 UTC
Verified on 4.6.0-0.nightly-2020-07-07-233934.  Using guestinfo.afterburn.initrd.network-kargs, I was able to set static IPs.

Comment 22 errata-xmlrpc 2020-10-27 15:54:52 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4196

Comment 26 Luca BRUNO 2020-10-29 08:16:07 UTC
> Should customer use the procedure in comment 18 ?

If they are already using `govc` for provisioning OVAs that's a feasible path, yes. The 4.6 release notes also have an entry showing just the relevant network kargs settings.

There are some other ways to perform the same task, depending on customer provisioning flow. Here is an example from a Fedora user via Terraform: https://github.com/coreos/fedora-coreos-tracker/issues/518#issuecomment-648162167.

> The errata is for OCP 4.6, is the procedure also working for OCP 4.5?

No. This is a new feature in 4.6, and it required some invasive underlying groundwork which is not compatible with earlier versions.

> Could it be also configured setting the variable [...] in the VMware GUI [...]?

No. Contrarily to the Ignition ones, this is an optional guestinfo which should only be set where required. The default is to have it unset, thus using DHCP(v4/v6).


Note You need to log in before you can comment on or make changes to this bug.