Description of problem: UPI/vsphere installation failed due to wrong config generated in master/worker ignition files without ignitionHost. Installation failed at wait for bootstrap stage since master can not boot up successfully. level=info msg="Waiting up to 30m0s for the Kubernetes API at https://api.qeci-2265.qe.devcluster.openshift.com:6443..." level=error msg="Attempted to gather ClusterOperator status after wait failure: listing ClusterOperator objects: the server could not find the requested resource (get clusteroperators.config.openshift.io)" level=info msg="Use the following commands to gather logs from the cluster" level=info msg="openshift-install gather bootstrap --help" level=fatal msg="waiting for Kubernetes API: context deadline exceeded" Dig more to find that wrong source link generated when "./openshift-install create ignition-configs". // Ignition config for the control plane machines. You should copy the contents of the master.ign generated by the installer. control_plane_ignition = <<END_OF_MASTER_IGNITION {"ignition":{"config":{"append":[{"source":"https://:22623/config/master","verification":{}}]},"security":{"tls":{"certificateAuthorities":[{"source":"data:text/plain;charset=utf-8;base64,XXXXXXXX","verification":{}}]}},"timeouts":{},"version":"2.2.0"},"networkd":{},"passwd":{},"storage":{},"systemd":{}} END_OF_MASTER_IGNITION // Ignition config for the compute machines. You should copy the contents of the worker.ign generated by the installer. compute_ignition = <<END_OF_WORKER_IGNITION {"ignition":{"config":{"append":[{"source":"https://:22623/config/worker","verification":{}}]},"security":{"tls":{"certificateAuthorities":[{"source":"data:text/plain;charset=utf-8;base64,XXXXXXXXXXXXXX","verification":{}}]}},"timeouts":{},"version":"2.2.0"},"networkd":{},"passwd":{},"storage":{},"systemd":{}} END_OF_WORKER_IGNITION =================================================== For an upi/vsphere, should use clusterdomain instead of apivip. https://github.com/openshift/installer/blob/master/pkg/asset/ignition/machine/node.go#L36 Version-Release number of the following components: 4.4.0-0.nightly-2020-01-17-192119 How reproducible: always Steps to Reproduce: 1. Trigger upi/vsphere installation on v4.4 with install-config.yaml such as: --- apiVersion: v1 controlPlane: hyperthreading: Enabled name: master replicas: 1 compute: - hyperthreading: Enabled name: worker replicas: 0 metadata: name: qeci-2265 platform: vsphere: vCenter: vcsa-qe.vmware.devcluster.openshift.com username: xxx password: xxx datacenter: dc1 defaultDatastore: xxx pullSecret: 'xxx' networking: clusterNetwork: - cidr: 10.128.0.0/14 hostPrefix: 23 serviceNetwork: - 172.30.0.0/16 networkType: OpenShiftSDN fips: true baseDomain: qe.devcluster.openshift.com sshKey: xxx 2. 3. Actual results: upi/vsphere installation failed due to wrong source link generated in master/worker ign files. Expected results: upi/vsphere installation succeed. Additional info: Please attach logs from ansible-playbook with the -vvv flag
We can workaround the issue by editing the ignition files manually and then continue left steps for installation. But it's not convenient to workaround it for QE's ci test and automation test. So add testblocker to get the issue fixed asap.
This should have been resolved with: https://github.com/openshift/installer/pull/2956 Please confirm, thanks!
Verified on 4.4.0-0.nightly-2020-02-02-225006 Correct source link generated in ignition files when create ignition-configs files. // Ignition config for the control plane machines. You should copy the contents of the master.ign generated by the installer. control_plane_ignition = <<END_OF_MASTER_IGNITION {"ignition":{"config":{"append":[{"source":"https://api-int.jliu-test.qe.devcluster.openshift.com:22623/config/master","verification":{}}]},"security":{"tls":{"certificateAuthorities":[{"source":"xxx","verification":{}}]}},"timeouts":{},"version":"2.2.0"},"networkd":{},"passwd":{},"storage":{},"systemd":{}} END_OF_MASTER_IGNITION // Ignition config for the compute machines. You should copy the contents of the worker.ign generated by the installer. compute_ignition = <<END_OF_WORKER_IGNITION {"ignition":{"config":{"append":[{"source":"https://api-int.jliu-test.qe.devcluster.openshift.com:22623/config/worker","verification":{}}]},"security":{"tls":{"certificateAuthorities":[{"source":"xxx","verification":{}}]}},"timeouts":{},"version":"2.2.0"},"networkd":{},"passwd":{},"storage":{},"systemd":{}} END_OF_WORKER_IGNITION
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:0581