Bug 1792705 - installer generate wrong source link in master/worker ign when create ignition files
Summary: installer generate wrong source link in master/worker ign when create ignitio...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 4.4
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.4.0
Assignee: Abhinav Dahiya
QA Contact: liujia
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-01-19 09:14 UTC by liujia
Modified: 2020-05-04 11:25 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-05-04 11:25:32 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift installer pull 2956 0 None closed Correct ignitionHost for vSphere UPI 2021-01-26 01:11:12 UTC
Red Hat Product Errata RHBA-2020:0581 0 None None None 2020-05-04 11:25:55 UTC

Description liujia 2020-01-19 09:14:01 UTC
Description of problem:
UPI/vsphere installation failed due to wrong config generated in master/worker ignition files without ignitionHost.

Installation failed at wait for bootstrap stage since master can not boot up successfully.
level=info msg="Waiting up to 30m0s for the Kubernetes API at https://api.qeci-2265.qe.devcluster.openshift.com:6443..."
level=error msg="Attempted to gather ClusterOperator status after wait failure: listing ClusterOperator objects: the server could not find the requested resource (get clusteroperators.config.openshift.io)"
level=info msg="Use the following commands to gather logs from the cluster"
level=info msg="openshift-install gather bootstrap --help"
level=fatal msg="waiting for Kubernetes API: context deadline exceeded"

Dig more to find that wrong source link generated when "./openshift-install create ignition-configs".

// Ignition config for the control plane machines. You should copy the contents of the master.ign generated by the installer.
control_plane_ignition = <<END_OF_MASTER_IGNITION
{"ignition":{"config":{"append":[{"source":"https://:22623/config/master","verification":{}}]},"security":{"tls":{"certificateAuthorities":[{"source":"data:text/plain;charset=utf-8;base64,XXXXXXXX","verification":{}}]}},"timeouts":{},"version":"2.2.0"},"networkd":{},"passwd":{},"storage":{},"systemd":{}}
END_OF_MASTER_IGNITION

// Ignition config for the compute machines. You should copy the contents of the worker.ign generated by the installer.
compute_ignition = <<END_OF_WORKER_IGNITION
{"ignition":{"config":{"append":[{"source":"https://:22623/config/worker","verification":{}}]},"security":{"tls":{"certificateAuthorities":[{"source":"data:text/plain;charset=utf-8;base64,XXXXXXXXXXXXXX","verification":{}}]}},"timeouts":{},"version":"2.2.0"},"networkd":{},"passwd":{},"storage":{},"systemd":{}}
END_OF_WORKER_IGNITION

===================================================
For an upi/vsphere, should use clusterdomain instead of apivip.

https://github.com/openshift/installer/blob/master/pkg/asset/ignition/machine/node.go#L36

Version-Release number of the following components:
4.4.0-0.nightly-2020-01-17-192119

How reproducible:
always

Steps to Reproduce:
1. Trigger upi/vsphere installation on v4.4 with install-config.yaml such as:
---
apiVersion: v1
controlPlane:
  hyperthreading: Enabled
  name: master
  replicas: 1
compute:
- hyperthreading: Enabled
  name: worker
  replicas: 0
metadata:
  name: qeci-2265
platform:
  vsphere:
    vCenter: vcsa-qe.vmware.devcluster.openshift.com
    username: xxx
    password: xxx
    datacenter: dc1
    defaultDatastore: xxx
pullSecret: 'xxx'
networking:
  clusterNetwork:
  - cidr: 10.128.0.0/14
    hostPrefix: 23
  serviceNetwork:
  - 172.30.0.0/16
  networkType: OpenShiftSDN
fips: true
baseDomain: qe.devcluster.openshift.com
sshKey: xxx
2.
3.

Actual results:
upi/vsphere installation failed due to wrong source link generated in master/worker ign files.

Expected results:
upi/vsphere installation succeed.

Additional info:
Please attach logs from ansible-playbook with the -vvv flag

Comment 1 liujia 2020-01-19 09:36:34 UTC
We can workaround the issue by editing the ignition files manually and then continue left steps for installation. But it's not convenient to workaround it for QE's ci test and automation test. So add testblocker to get the issue fixed asap.

Comment 2 Joseph Callen 2020-01-29 21:04:36 UTC
This should have been resolved with:
https://github.com/openshift/installer/pull/2956

Please confirm, thanks!

Comment 3 liujia 2020-02-03 07:51:38 UTC
Verified on 4.4.0-0.nightly-2020-02-02-225006

Correct source link generated in ignition files when create ignition-configs files.
// Ignition config for the control plane machines. You should copy the contents of the master.ign generated by the installer.
control_plane_ignition = <<END_OF_MASTER_IGNITION
{"ignition":{"config":{"append":[{"source":"https://api-int.jliu-test.qe.devcluster.openshift.com:22623/config/master","verification":{}}]},"security":{"tls":{"certificateAuthorities":[{"source":"xxx","verification":{}}]}},"timeouts":{},"version":"2.2.0"},"networkd":{},"passwd":{},"storage":{},"systemd":{}}
END_OF_MASTER_IGNITION

// Ignition config for the compute machines. You should copy the contents of the worker.ign generated by the installer.
compute_ignition = <<END_OF_WORKER_IGNITION
{"ignition":{"config":{"append":[{"source":"https://api-int.jliu-test.qe.devcluster.openshift.com:22623/config/worker","verification":{}}]},"security":{"tls":{"certificateAuthorities":[{"source":"xxx","verification":{}}]}},"timeouts":{},"version":"2.2.0"},"networkd":{},"passwd":{},"storage":{},"systemd":{}}
END_OF_WORKER_IGNITION

Comment 5 errata-xmlrpc 2020-05-04 11:25:32 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0581


Note You need to log in before you can comment on or make changes to this bug.