Bug 1785122

Summary: [vSphere] [UPI] - Cannot provide Static IP addresses when using OVA
Product: OpenShift Container Platform Reporter: Anoel Yakoubov <ayakoubo>
Component: RHCOSAssignee: Luca BRUNO <lucab>
Status: CLOSED ERRATA QA Contact: Michael Nguyen <mnguyen>
Severity: high Docs Contact:
Priority: high    
Version: 4.2.zCC: bbreard, bsmitley, dornelas, dustymabe, imcleod, jligon, linux.squad.redhat.mail, lucab, mfuruta, miabbott, nstielau, oarribas, obockows, ralvares, rbohne, smilner, susuresh, syangsao
Target Milestone: ---   
Target Release: 4.6.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Feature: It is now possible to override the default DHCP networking on first boot, by setting the relevant guestinfo property before booting a VM. Reason: This lowers the barrier for automatic RHCOS deployment in environments where DHCP is not provided (and Ignition config requires networking). Result: Higher-level automation can provision RHCOS OVA in environments with static networking.
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-10-27 15:54:52 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1186913, 1819833    
Attachments:
Description Flags
VMX file of the relevant VM none

Description Anoel Yakoubov 2019-12-19 08:31:40 UTC
Created attachment 1646329 [details]
VMX file of the relevant VM

Description of problem:


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Anoel Yakoubov 2019-12-19 08:57:22 UTC
Description of problem:
I am working in Very Big and Classified customer and this customer must to work with Static IP addresses and this solution should be supported with CoreOS4.2 per our documentation
in https://bugzilla.redhat.com/show_bug.cgi?id=1755047, we are trying to provide Static IPs by using VM options--> advanced--> edit configuration and adding the relevant info (PSA)
We can see all info in right format in VM's vmx file, but still in boot VM can't receive IP and then can't load ign file accordingly. Customer must use OVA for VMs provisioning since we need to 

Version-Release number of selected component (if applicable):
RHCOS 4.2, OCP 4.2.9

How reproducible:
Can be reproduced easy

Steps to Reproduce:
1. Create VM from OVA
2. Add all relevant Guestinfo properties by using the example

guestinfo.hostname = "coreos"
guestinfo.interface.0.role = "private"
guestinfo.dns.server.0 = "8.8.8.8"
guestinfo.interface.0.route.0.gateway = "192.168.178.1"
guestinfo.interface.0.route.0.destination = "0.0.0.0/0"
guestinfo.interface.0.mac = "00:0c:29:63:92:5c"
guestinfo.interface.0.name = "eno1*"
guestinfo.interface.0.dhcp = "no"
guestinfo.interface.0.ip.0.address = "192.168.178.97/24"

3. Power On VM

Actual results: VM cannot load ign file and complete boot since it not receiving IP and have error "Network unreachable"


Expected results:


VM receive Static IP, load ign file, complete boot

Additional info:

It is very crucial for customer to be able to provide static IPs without managing DHCP, without managing mac addresses, with usage of OVA.

Comment 2 Anoel Yakoubov 2019-12-19 08:57:53 UTC
Description of problem:
I am working in Very Big and Classified customer and this customer must to work with Static IP addresses and this solution should be supported with CoreOS4.2 per our documentation
in https://bugzilla.redhat.com/show_bug.cgi?id=1755047, we are trying to provide Static IPs by using VM options--> advanced--> edit configuration and adding the relevant info (PSA)
We can see all info in right format in VM's vmx file, but still in boot VM can't receive IP and then can't load ign file accordingly. Customer must use OVA for VMs provisioning since we need to 

Version-Release number of selected component (if applicable):
RHCOS 4.2, OCP 4.2.9

How reproducible:
Can be reproduced easy

Steps to Reproduce:
1. Create VM from OVA
2. Add all relevant Guestinfo properties by using the example

guestinfo.hostname = "coreos"
guestinfo.interface.0.role = "private"
guestinfo.dns.server.0 = "8.8.8.8"
guestinfo.interface.0.route.0.gateway = "192.168.178.1"
guestinfo.interface.0.route.0.destination = "0.0.0.0/0"
guestinfo.interface.0.mac = "00:0c:29:63:92:5c"
guestinfo.interface.0.name = "eno1*"
guestinfo.interface.0.dhcp = "no"
guestinfo.interface.0.ip.0.address = "192.168.178.97/24"

3. Power On VM

Actual results: VM cannot load ign file and complete boot since it not receiving IP and have error "Network unreachable"


Expected results:


VM receive Static IP, load ign file, complete boot

Additional info:

It is very crucial for customer to be able to provide static IPs without managing DHCP, without managing mac addresses, with usage of OVA.

Comment 3 Micah Abbott 2019-12-19 15:57:43 UTC
Static IP assignment on RHCOS nodes should be done via the kernel command line when doing the initial install of the RHCOS nodes, before the fetching of Ignition is done.

See the following blog post for more details - https://blog.openshift.com/openshift-4-2-vsphere-install-with-static-ips/

Alternately see the following KBase article - https://access.redhat.com/solutions/4531011

Comment 4 Anoel Yakoubov 2019-12-21 07:52:45 UTC
Hi, I am familiar with both solutions, but in our case, as I mentioned before we must to use OVA Template and can't use ISO file, since we are doing automation of this deployment to edge locations, so solution should support usage of OVA provided in our cloud.redhat.com site

Comment 6 linux.squad.redhat.mail 2020-02-24 14:16:20 UTC
Hi
My name is Stav and I'm from a big company in Israel , running in a restricted network.
I have been trying to install OCP4.3 as well from OVA template, using static ip.

I've managed to do that only by using OVA and then editing grub configuration manually with the following parameter:
ip=<host_ip>::<default GW>:<netmask>:<fqdn>:ens192:none nameserver=<dns1>

We would like a more automated way to use static ip, since our network team doesn't work with DHCP for servers.

we deploy the VMs with the terraform module from the openshift/installer github repository but it doesn't really matter.

During initramfs the RHCOS uses DHCP by default.

Comment 7 Micah Abbott 2020-02-26 19:11:38 UTC
The RHCOS team recognizes that the UX for configuring static IPs in a vSphere environment is difficult and we are actively working on improving this experience for the 4.5 release.

Comment 8 Rodrigo Alvares 2020-04-15 17:31:29 UTC
Hello All,

I know it is not related to the OVA.

few months ago I managed to get a automated installation on Vmware using static IPs by building a custom ipxe.iso image which is capable of reading vmware guestinfo.

Please find the link below:

https://github.com/ralvares/ipxe

Best Regards

Comment 9 Dusty Mabe 2020-05-18 15:34:18 UTC
This should be fixed with some feature work Luca is doing that will land in 4.6.

Comment 11 Micah Abbott 2020-06-20 13:32:02 UTC
The fix landed in afterburn-4.4.0-1.rhaos4.6.el8 and is now available in 4.6 RHCOS builds

Setting to MODIFIED

Comment 15 linux.squad.redhat.mail 2020-06-22 19:21:50 UTC
Thanks for the update Micah.
Will it be documented in the official documentaion of future OCP 4.6 version?

Comment 17 Michael Nguyen 2020-07-08 17:00:52 UTC
Verified on 4.6.0-0.nightly-2020-07-07-233934.  Using guestinfo.afterburn.initrd.network-kargs, I was able to set static IPs.

Comment 22 errata-xmlrpc 2020-10-27 15:54:52 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4196

Comment 26 Luca BRUNO 2020-10-29 08:16:07 UTC
> Should customer use the procedure in comment 18 ?

If they are already using `govc` for provisioning OVAs that's a feasible path, yes. The 4.6 release notes also have an entry showing just the relevant network kargs settings.

There are some other ways to perform the same task, depending on customer provisioning flow. Here is an example from a Fedora user via Terraform: https://github.com/coreos/fedora-coreos-tracker/issues/518#issuecomment-648162167.

> The errata is for OCP 4.6, is the procedure also working for OCP 4.5?

No. This is a new feature in 4.6, and it required some invasive underlying groundwork which is not compatible with earlier versions.

> Could it be also configured setting the variable [...] in the VMware GUI [...]?

No. Contrarily to the Ignition ones, this is an optional guestinfo which should only be set where required. The default is to have it unset, thus using DHCP(v4/v6).