Bug 1873125

Summary:	UPI missing instructions to update apiVIP and ingressVIP addresses
Product:	OpenShift Container Platform	Reporter:	Martin André <m.andre>
Component:	Installer	Assignee:	Adolfo Duarte <adduarte>
Installer sub component:	OpenShift on OpenStack	QA Contact:	weiwei jiang <wjiang>
Status:	CLOSED CURRENTRELEASE	Docs Contact:
Severity:	medium
Priority:	medium	CC:	pprinett
Version:	4.6	Keywords:	UpcomingSprint
Target Milestone:	---
Target Release:	4.6.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-10-01 14:20:25 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Martin André 2020-08-27 12:50:58 UTC

Description of problem: The OpenStack UPI explains how to update the node subnet, however these days we also persist the apiVIP and ingressVIP in the install-config.yaml file and these two values need to be updated as well to match the new machine CIDR.

As a consequence, strictly following the UPI documentation, we fail to generate the manifests with a validation error:

❯ openshift-install create manifests     
FATAL failed to fetch Master Machines: failed to load asset "Install Config": invalid "install-config.yaml" file: [platform.openstack.apiVIP: Invalid value: "10.0.0.5": IP is not in the machineNetwork, platform.openstack.ingressVIP: Invalid value: "10.0.0.7": IP is not in the machineNetwork] 

We need to either not write the apiVIP and ingressVIP to the install-config.yaml, or update the instruction in the UPI documentation to update these values as well when changing the machine CIDR.


[1] https://github.com/openshift/installer/blob/f7028b2/docs/user/openstack/install_upi.md#fix-the-node-subnet

Comment 2 Adolfo Duarte 2020-09-03 01:45:54 UTC

https://github.com/openshift/installer/pull/4133

Comment 5 weiwei jiang 2020-09-10 02:13:58 UTC

Assign back this due to following issue, delete should be replaced with del

python2 -c 'import yaml;
import sys;
path = "install-config.yaml";
data = yaml.safe_load(open(path));
if "apiVIP" in data["platform"]["openstack"]:
   delete data["platform"]["openstack"]["apiVIP"] ;
if "ingressVIP" in data["platform"]["openstack"]:
   delete data["platform"]["openstack"]["ingressVIP"];
open(path, "w").write(yaml.dump(data, default_flow_style=False))'
  File "<string>", line 6
    delete data["platform"]["openstack"]["apiVIP"] ;
              ^
SyntaxError: invalid syntax

Comment 7 weiwei jiang 2020-09-11 02:02:30 UTC

Verified.

1. change the machineNetwork
...
networking:
  clusterNetwork:
  - cidr: 10.128.0.0/14
    hostPrefix: 23
  machineNetwork:
  - cidr: 192.0.0.0/24
  networkType: OpenShiftSDN
  serviceNetwork:
  - 172.30.0.0/16

$ python -c 'import yaml
path = "install-config.yaml"
data = yaml.safe_load(open(path))
data["networking"]["machineNetwork"][0]["cidr"] = "192.0.2.0/24"
open(path, "w").write(yaml.dump(data, default_flow_style=False))'

...
networking:
  clusterNetwork:
  - cidr: 10.128.0.0/14
    hostPrefix: 23
  machineNetwork:
  - cidr: 192.0.2.0/24
  networkType: OpenShiftSDN
  serviceNetwork:
  - 172.30.0.0/16

2. delete the VIPs
...
platform:
  openstack:
    apiVIP: 192.0.2.8
    cloud: openstack
    computeFlavor: m1.xlarge
    externalDNS:
    - 10.0.77.163
    externalNetwork: provider_net_cci_8
    ingressFloatingIP: 10.0.102.230
    ingressVIP: 192.0.2.9
    lbFloatingIP: 10.0.103.156
    octaviaSupport: '0'
    region: regionOne
    trunkSupport: '1'

...
platform:
  openstack:
    cloud: openstack
    computeFlavor: m1.xlarge
    externalDNS:
    - 10.0.77.163
    externalNetwork: provider_net_cci_8
    ingressFloatingIP: 10.0.102.230
    lbFloatingIP: 10.0.103.156
    octaviaSupport: '0'
    region: regionOne
    trunkSupport: '1'

3. update the VIPs
...
platform:
  openstack:
    cloud: openstack
    computeFlavor: m1.xlarge
    externalDNS:
    - 10.0.77.163
    externalNetwork: provider_net_cci_8
    ingressFloatingIP: 10.0.102.230
    lbFloatingIP: 10.0.103.156
    octaviaSupport: '0'
    region: regionOne
    trunkSupport: '1'
    apiVIP: 192.168.2.123
    ingress: 192.168.3.234

$ python -c 'import yaml
import sys                         
path = "install-config.yaml"                               
data = yaml.safe_load(open(path))                          
if "apiVIP" in data["platform"]["openstack"]:                    
   data["platform"]["openstack"]["apiVIP"] = "192.0.2.8"          
if "ingressVIP" in data["platform"]["openstack"]:          
   data["platform"]["openstack"]["ingressVIP"] = "192.0.2.9"                                                                                                                                                                                  
open(path, "w").write(yaml.dump(data, default_flow_style=False))'

platform:
  openstack:
    apiVIP: 192.0.2.8
    cloud: openstack
    computeFlavor: m1.xlarge
    externalDNS:
    - 10.0.77.163
    externalNetwork: provider_net_cci_8
    ingress: 192.168.3.234
    ingressFloatingIP: 10.0.102.230
    lbFloatingIP: 10.0.103.156
    octaviaSupport: '0'
    region: regionOne
    trunkSupport: '1'