Hide Forgot
By default, the API VIP takes x.x.x.5 and the Ingress VIP takes x.x.x.7 from the network’s CIDR block. To override these default values, we can set values for platform.openstack.apiVIP and platform.openstack.ingressVIP that are outside of the DHCP allocation pool. There is no validation in the installer at the moment that'll make sure the end-user set this right. If it's not right, it'll cause connectivity problems with the API when deploying the cluster.
currently there is some sort of check using: networking: clusterNetwork: - cidr: 10.128.0.0/14 hostPrefix: 23 machineNetwork: - cidr: 10.0.128.0/17 networkType: OpenShiftSDN serviceNetwork: - 172.30.0.0/16 platform: openstack: apiFloatingIP: 192.168.25.59 apiVIP: 10.0.18.5 cloud: openshift defaultMachinePlatform: type: m1.xxlarge externalDNS: null externalNetwork: hostonly ingressVIP: 10.0.128.7 then running ./openshift-install create ... I get [stack@standalone fake]$ ./openshift-install create install-config FATAL failed to fetch Install Config: failed to load asset "Install Config": invalid "install-config.yaml" file: platform.openstack.apiVIP: Invalid value: "10.0.18.5": IP is not in the machineNetwork doing the same for ingressVIP I get: [stack@standalone fake]$ ./openshift-install create install-config FATAL failed to fetch Install Config: failed to load asset "Install Config": invalid "install-config.yaml" file: platform.openstack.ingressVIP: Invalid value: "10.0.18.7": IP is not in the machineNetwork @emacchi, do you have an install-config.yaml where the check might be failing?
My bug description was really incomplete, I'm sorry for that. I reproduced the issue in the context of a Bring Your Own Network, and if your subnet has an allocation pool that includes .5 and .7, it caused the error that I had. I'll try to reproduce it quickly. Note that your error is because you have a type, your machineNetwork is 10.0.128.0/17, not 10.0.18.0/17.
Related PR validating the API and Ingress VIPs are different: https://github.com/openshift/installer/pull/4946 We can build upon the above PR to add a check for API and Ingress VIPs being outside of the DHCP allocation pool, however I wouldn't be comfortable with the new check to be fatal as there are probably a few corner cases. For example it's possible for the VIPs to be in the allocation pool with the DHCP server is configured in such a way to not allocate the IPs other than to the API and Ingress ports.
Removing the Triaged keyword because: * the target release value is missing * the QE automation assessment (flag qe_test_coverage) is missing
Note for the verifier: this is how I tested my PR --> https://github.com/openshift/installer/pull/6069#issuecomment-1174872791
Verified with openshift-install 4.12.0-0.nightly-2022-08-01-151317 on top of RHOS-16.1-RHEL-8-20220329.n.1: 1. Configured an OpenStack subnet with the following allocation pool: ``` $ openstack subnet show restricted_subnet -c allocation_pools +------------------+-------------------------+ | Field | Value | +------------------+-------------------------+ | allocation_pools | 172.16.0.4-172.16.0.254 | +------------------+-------------------------+ ``` 1.1. Configured an ingressVIP outside the machineNetwork: ``` $ grep -A 1 "machineNetwork" install-config.yaml && grep "VIP" install-config.yaml machineNetwork: - cidr: "172.16.0.0/25" apiVIP: "172.16.0.4" ingressVIP: "172.16.0.254" $ openshift-install create manifests ERROR failed to fetch Master Machines: failed to load asset "Install Config": failed to create install config: invalid "install-config.yaml" file: platform.openstack.ingressVIP: Invalid value: "172.16.0.254": IP is not in the machineNetwork ``` 1.2. Configured the VIPs overlap with the allocation pool: ``` $ grep -A 1 "machineNetwork" install-config.yaml && grep "VIP" install-config.yaml machineNetwork: - cidr: "172.16.0.0/24" apiVIP: "172.16.0.4" ingressVIP: "172.16.0.254" $ openshift-install create manifests ERROR failed to fetch Master Machines: failed to load asset "Install Config": failed to create install config: [platform.openstack.apiVIP: Invalid value: "172.16.0.4": apiVIP can not fall in a MachineNetwork allocation pool, platform.openstack.ingressVIP: Invalid value: "172.16.0.254": ingressVIP can not fall in a MachineNetwork allocation pool] ``` 1.3. Configured the ingressVIP overlaps with the allocation pool: ``` $ grep -A 1 "machineNetwork" install-config.yaml && grep "VIP" install-config.yaml machineNetwork: - cidr: "172.16.0.0/24" apiVIP: "172.16.0.3" ingressVIP: "172.16.0.254" $ openshift-install create manifests ERROR failed to fetch Master Machines: failed to load asset "Install Config": failed to create install config: platform.openstack.ingressVIP: Invalid value: "172.16.0.254": ingressVIP can not fall in a MachineNetwork allocation pool ``` 1.4. Configured the apiVIP overlaps with the allocation pool: ``` $ grep -A 1 "machineNetwork" install-config.yaml && grep "VIP" install-config.yaml machineNetwork: - cidr: "172.16.0.0/24" apiVIP: "172.16.0.4" ingressVIP: "172.16.0.3" $ openshift-install create manifests ERROR failed to fetch Master Machines: failed to load asset "Install Config": failed to create install config: platform.openstack.apiVIP: Invalid value: "172.16.0.4": apiVIP can not fall in a MachineNetwork allocation pool ``` 1.5. Using the default VIPs which overlap with the allocation pool: ``` $ grep -A 1 "machineNetwork" install-config.yaml && grep "VIP" install-config.yaml machineNetwork: - cidr: "172.16.0.0/24" #apiVIP: "172.16.0.4" #ingressVIP: "172.16.0.3" $ openshift-install create manifests ERROR failed to fetch Master Machines: failed to load asset "Install Config": failed to create install config: [platform.openstack.apiVIP: Invalid value: "172.16.0.5": apiVIP can not fall in a MachineNetwork allocation pool, platform.openstack.ingressVIP: Invalid value: "172.16.0.7": ingressVIP can not fall in a MachineNetwork allocation pool] ``` 2. Configured the VIPs outside of the allocation pool, and the Manifests created: ``` $ openstack subnet unset restricted_subnet --allocation-pool start=172.16.0.4,end=172.16.0.254 $ openstack subnet set restricted_subnet --allocation-pool start=172.16.0.8,end=172.16.0.254 $ openstack subnet show restricted_subnet -c allocation_pools +------------------+-------------------------+ | Field | Value | +------------------+-------------------------+ | allocation_pools | 172.16.0.8-172.16.0.254 | +------------------+-------------------------+ $ grep -A 1 "machineNetwork" install-config.yaml && grep "VIP" install-config.yaml machineNetwork: - cidr: "172.16.0.0/24" apiVIP: "172.16.0.5" ingressVIP: "172.16.0.7" $ openshift-install create manifests INFO Credentials loaded from file "/home/cloud-user/clouds.yaml" INFO Consuming Install Config from target directory INFO Manifests created in: manifests and openshift ``` 3. Configured multiple allocation pool: ``` $ openstack subnet set restricted_subnet --allocation-pool start=172.16.0.8,end=172.16.0.100 $ openstack subnet set restricted_subnet --allocation-pool start=172.16.0.110,end=172.16.0.254 $ openstack subnet show restricted_subnet -c allocation_pools +------------------+---------------------------------------------------+ | Field | Value | +------------------+---------------------------------------------------+ | allocation_pools | 172.16.0.8-172.16.0.100,172.16.0.110-172.16.0.254 | +------------------+---------------------------------------------------+ ``` 3.1. Configured the VIPs overlap with the allocation pools: ``` $ grep -A 1 "machineNetwork" install-config.yaml && grep "VIP" install-config.yaml machineNetwork: - cidr: "172.16.0.0/24" apiVIP: "172.16.0.10" ingressVIP: "172.16.0.110" $ openshift-install create manifests ERROR failed to fetch Master Machines: failed to load asset "Install Config": failed to create install config: [platform.openstack.ingressVIP: Invalid value: "172.16.0.110": ingressVIP can not fall in a MachineNetwork allocation pool, platform.openstack.apiVIP: Invalid value: "172.16.0.10": apiVIP can not fall in a MachineNetwork allocation pool] ``` 3.2. Configured the VIPs outside of the allocation pools, and the Manifests created: ``` $ grep -A 1 "machineNetwork" install-config.yaml && grep "VIP" install-config.yaml machineNetwork: - cidr: "172.16.0.0/24" apiVIP: "172.16.0.101" ingressVIP: "172.16.0.102" $ openshift-install create manifests WARNING Discarding the Master Machines that was provided in the target directory because its dependencies are dirty and it needs to be regenerated INFO Credentials loaded from file "/home/cloud-user/clouds.yaml" INFO Consuming Install Config from target directory WARNING Discarding the Worker Machines that was provided in the target directory because its dependencies are dirty and it needs to be regenerated WARNING Discarding the Common Manifests that was provided in the target directory because its dependencies are dirty and it needs to be regenerated WARNING Discarding the Openshift Manifests that was provided in the target directory because its dependencies are dirty and it needs to be regenerated INFO Consuming OpenShift Install (Manifests) from target directory INFO Manifests created in: manifests and openshift ```
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.12.0 bug fix and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:7399