Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1792705

Summary: installer generate wrong source link in master/worker ign when create ignition files
Product: OpenShift Container Platform Reporter: liujia <jiajliu>
Component: InstallerAssignee: Abhinav Dahiya <adahiya>
Installer sub component: openshift-installer QA Contact: liujia <jiajliu>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: high CC: jcallen
Version: 4.4Keywords: Regression, TestBlocker
Target Milestone: ---   
Target Release: 4.4.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-05-04 11:25:32 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description liujia 2020-01-19 09:14:01 UTC
Description of problem:
UPI/vsphere installation failed due to wrong config generated in master/worker ignition files without ignitionHost.

Installation failed at wait for bootstrap stage since master can not boot up successfully.
level=info msg="Waiting up to 30m0s for the Kubernetes API at https://api.qeci-2265.qe.devcluster.openshift.com:6443..."
level=error msg="Attempted to gather ClusterOperator status after wait failure: listing ClusterOperator objects: the server could not find the requested resource (get clusteroperators.config.openshift.io)"
level=info msg="Use the following commands to gather logs from the cluster"
level=info msg="openshift-install gather bootstrap --help"
level=fatal msg="waiting for Kubernetes API: context deadline exceeded"

Dig more to find that wrong source link generated when "./openshift-install create ignition-configs".

// Ignition config for the control plane machines. You should copy the contents of the master.ign generated by the installer.
control_plane_ignition = <<END_OF_MASTER_IGNITION
{"ignition":{"config":{"append":[{"source":"https://:22623/config/master","verification":{}}]},"security":{"tls":{"certificateAuthorities":[{"source":"data:text/plain;charset=utf-8;base64,XXXXXXXX","verification":{}}]}},"timeouts":{},"version":"2.2.0"},"networkd":{},"passwd":{},"storage":{},"systemd":{}}
END_OF_MASTER_IGNITION

// Ignition config for the compute machines. You should copy the contents of the worker.ign generated by the installer.
compute_ignition = <<END_OF_WORKER_IGNITION
{"ignition":{"config":{"append":[{"source":"https://:22623/config/worker","verification":{}}]},"security":{"tls":{"certificateAuthorities":[{"source":"data:text/plain;charset=utf-8;base64,XXXXXXXXXXXXXX","verification":{}}]}},"timeouts":{},"version":"2.2.0"},"networkd":{},"passwd":{},"storage":{},"systemd":{}}
END_OF_WORKER_IGNITION

===================================================
For an upi/vsphere, should use clusterdomain instead of apivip.

https://github.com/openshift/installer/blob/master/pkg/asset/ignition/machine/node.go#L36

Version-Release number of the following components:
4.4.0-0.nightly-2020-01-17-192119

How reproducible:
always

Steps to Reproduce:
1. Trigger upi/vsphere installation on v4.4 with install-config.yaml such as:
---
apiVersion: v1
controlPlane:
  hyperthreading: Enabled
  name: master
  replicas: 1
compute:
- hyperthreading: Enabled
  name: worker
  replicas: 0
metadata:
  name: qeci-2265
platform:
  vsphere:
    vCenter: vcsa-qe.vmware.devcluster.openshift.com
    username: xxx
    password: xxx
    datacenter: dc1
    defaultDatastore: xxx
pullSecret: 'xxx'
networking:
  clusterNetwork:
  - cidr: 10.128.0.0/14
    hostPrefix: 23
  serviceNetwork:
  - 172.30.0.0/16
  networkType: OpenShiftSDN
fips: true
baseDomain: qe.devcluster.openshift.com
sshKey: xxx
2.
3.

Actual results:
upi/vsphere installation failed due to wrong source link generated in master/worker ign files.

Expected results:
upi/vsphere installation succeed.

Additional info:
Please attach logs from ansible-playbook with the -vvv flag

Comment 1 liujia 2020-01-19 09:36:34 UTC
We can workaround the issue by editing the ignition files manually and then continue left steps for installation. But it's not convenient to workaround it for QE's ci test and automation test. So add testblocker to get the issue fixed asap.

Comment 2 Joseph Callen 2020-01-29 21:04:36 UTC
This should have been resolved with:
https://github.com/openshift/installer/pull/2956

Please confirm, thanks!

Comment 3 liujia 2020-02-03 07:51:38 UTC
Verified on 4.4.0-0.nightly-2020-02-02-225006

Correct source link generated in ignition files when create ignition-configs files.
// Ignition config for the control plane machines. You should copy the contents of the master.ign generated by the installer.
control_plane_ignition = <<END_OF_MASTER_IGNITION
{"ignition":{"config":{"append":[{"source":"https://api-int.jliu-test.qe.devcluster.openshift.com:22623/config/master","verification":{}}]},"security":{"tls":{"certificateAuthorities":[{"source":"xxx","verification":{}}]}},"timeouts":{},"version":"2.2.0"},"networkd":{},"passwd":{},"storage":{},"systemd":{}}
END_OF_MASTER_IGNITION

// Ignition config for the compute machines. You should copy the contents of the worker.ign generated by the installer.
compute_ignition = <<END_OF_WORKER_IGNITION
{"ignition":{"config":{"append":[{"source":"https://api-int.jliu-test.qe.devcluster.openshift.com:22623/config/worker","verification":{}}]},"security":{"tls":{"certificateAuthorities":[{"source":"xxx","verification":{}}]}},"timeouts":{},"version":"2.2.0"},"networkd":{},"passwd":{},"storage":{},"systemd":{}}
END_OF_WORKER_IGNITION

Comment 5 errata-xmlrpc 2020-05-04 11:25:32 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0581