Bug 1891702 - installer get pending when additionalTrustBundle is added into install-config.yaml
Summary: installer get pending when additionalTrustBundle is added into install-confi...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 4.7
Hardware: Unspecified
OS: Unspecified
urgent
high
Target Milestone: ---
: 4.7.0
Assignee: John Hixson
QA Contact: Johnny Liu
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-10-27 06:16 UTC by Johnny Liu
Modified: 2021-02-24 15:29 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-02-24 15:28:28 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
installer log (85.31 KB, text/plain)
2020-10-27 06:17 UTC, Johnny Liu
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Github openshift installer pull 4317 0 None closed bug 1891702: pkg/asset/ignition/bootstrap: exit loop if pem.Decode() fails 2020-11-26 01:59:42 UTC
Red Hat Product Errata RHSA-2020:5633 0 None None None 2021-02-24 15:29:04 UTC

Description Johnny Liu 2020-10-27 06:16:29 UTC
Version:

$ openshift-install version
openshift-install 4.7.0-0.nightly-2020-10-26-152308
built from commit de62b01d451857d942ac41dcfdad9385856fde9e
release image registry.svc.ci.openshift.org/ocp/release@sha256:dc591b4a50e8c709f6b65a118473c7241bda6837a184520f0ef94826752791b2

Platform:
azure/openstack/aws/vsphere

Please specify:
* IPI

What happened?
1. here is my install-config.yaml with additionalTrustBundle is injected.
apiVersion: v1
controlPlane:
  architecture: amd64
  hyperthreading: Enabled
  name: master
  platform:
    azure: {}
  replicas: 3
compute:
- architecture: amd64
  hyperthreading: Enabled
  name: worker
  platform:
    azure:
      type: Standard_D4s_v3
  replicas: 3
metadata:
  name: qeci-10098
platform:
  azure:
    region: centralus
    baseDomainResourceGroupName: os4-common
pullSecret: '{"auths":{"my-registry:5000":{"auth":"xxxxxx="}}}'
networking:
  clusterNetwork:
  - cidr: 10.128.0.0/14
    hostPrefix: 23
  serviceNetwork:
  - 172.30.0.0/16
  machineNetwork:
  - cidr: 10.0.0.0/16
  networkType: OpenShiftSDN
publish: Internal
additionalTrustBundle: |
  -----BEGIN CERTIFICATE-----
  MIIFqTCCA5GgAwIBAgICEAAwDQYJKoZIhvcNAQELBQAwaTELMAkGA1UEBhMCQ04x
  EDAOBgNVBAgMB0JlaWppbmcxEDAOBgNVBAcMB0JlaWppbmcxDDAKBgNVBAoMA09D
  UDEPMA0GA1UECwwGT0NQLVFFMRcwFQYDVQQDDA5PQ1AtUUUtUk9PVC1DQTAeFw0x
  OTA4MTgwNjA4MzRaFw0yOTA4MTUwNjA4MzRaMF4xCzAJBgNVBAYTAkNOMRAwDgYD
  VQQIDAdCZWlqaW5nMQwwCgYDVQQKDANPQ1AxFTATBgNVBAsMDEluc3RhbGxlci1R
  RTEYMBYGA1UEAwwPSW5zdGFsbGVyLVFFLUNBMIICIjANBgkqhkiG9w0BAQEFAAOC
  Ag8AMIICCgKCAgEAwt0MujtrS6uPOx9pV71W5o0Nk9a6Fe4bSojyyOJw1SmDihaC
  AvxrWK3NHaqYV8cqQWLB1ZXtw8LF74BK98/b94PvauqgTn3Kg+Vcqnq3JlpyrgKN
  n5g4ORYScQXlyN/Kzn98cv07qHn1MhwZt8W8lYI9m6z2un0VyPkr8UgSmvDo0cx0
  zwjB5Q7zCvXcoc1IQFa3JkYH4Z6Ccz9FNYnDRtoqu8K3SiWid50WEXcpycMLCSwb
  SVSDAsUR5wwA4aTgW7s32Fdd4fAtNcnfZ2AnLTwyJBZoPeoa5npvmpCr8khLyDdW
  Y9rWDfaKXhB++Ou27FDE6NLWQK/FPMVNPIr+P3xPbHIDlwzWq0eSK8SMsiOZrI9N
  dzMNGtcxv3sfxMYqKhnl3HrZbXbM1ouD9lsv5zGCAIdrnmZoMRI9NTjBatOevZXQ
  ojby2XQzNDX1ouQK4gSTi9q3aa1e8WQfiLbaNPxAU9FlLqS7J16nFsTsWQ6Qt6iN
  yEFaw3pYWeZk6sacGQECvmfrbaHxlI63rQUI3mRxs8mZqb3zJapcbNtUlimEAsqE
  1oj/Tv3oVQKei2MpQHctenJqOZGC0Q/iWeRALD9E656MqbIt5dudEnx56Nq8av4r
  sad+OquDKFB/EnQ69VViYs9s6Ck426bqX5dx6T0Y0Tgk0WcnR5aPO+YrEtUCAwEA
  AaNmMGQwHQYDVR0OBBYEFJhUiRBfCjzfjHxoPLwYEwz5jHuuMB8GA1UdIwQYMBaA
  FG5nokgYqmIIwaW7blM6wHVIQwBIMBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0P
  AQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4ICAQBhcO+rA1blMP7SKt3/qzqsX5di
  BRxqOWqlmpKDgmC9rJts76t/PEodI2XNUVnKtybQD7Fh768b4fo0WO/evWUxs2LM
  4d7jQp5KTqEPhv6oKlrTp9fzw3BGwdnzZSPk6L8ahZvyr0i7Hls9oe5Pvhy5F87e
  qWt/SuDMCztYR3gs78IxBYMv4BPEuCeLsvLlPFW4vl+4lpGjOGcS8GbwwZIwq5X4
  LIdkk00NAMQ6Nmztoc+k/EVnj7O/bj66FY4WZFYUgnKUMlJ33UZy+Uao2GKUAM8j
  znFOl8fHgLYlcHsRYyLWeMGmOk0ukN06AvygnWh0UVBQCRrmTPNsShK+PlRyHmFW
  Zw4TDuPOqEwLx1VcmlEbLbpgc4f4GUWKGegaLHUltfwTwlb/6m1J4HomiYrBhdLJ
  LDReBo7dNYr7mpGPfZIMRdmywz6w10F1zTKe2F1KHb7mR7tyORaZ7NcAtmQmuxDF
  T8sUTrIop4GaQMZnNTPImtPGt23zsNTXUY93IeISJ6eUDKlnDgzYJDQ3pnKWbWHz
  wdWcyjh0Ojh/snItIm6/h1+CQ/FRlnt3+LRP9GxvWHbn1+sS51Kb979m/R0W7Djt
  y4p+AwCHpLwi9sU17Lg1JafgJVFB9Tu2wz/DIocfzdpP+7MUrqTkeDmN0p+Ia1Y9
  bTSegOgySxp2uzPJqg==
  -----END CERTIFICATE-----
  -----BEGIN CERTIFICATE-----
  MIIFuDCCA6CgAwIBAgIJAJk39xzKHHf9MA0GCSqGSIb3DQEBCwUAMGkxCzAJBgNV
  BAYTAkNOMRAwDgYDVQQIDAdCZWlqaW5nMRAwDgYDVQQHDAdCZWlqaW5nMQwwCgYD
  VQQKDANPQ1AxDzANBgNVBAsMBk9DUC1RRTEXMBUGA1UEAwwOT0NQLVFFLVJPT1Qt
  Q0EwHhcNMTkwODE4MDYwNzU4WhcNMzkwODEzMDYwNzU4WjBpMQswCQYDVQQGEwJD
  TjEQMA4GA1UECAwHQmVpamluZzEQMA4GA1UEBwwHQmVpamluZzEMMAoGA1UECgwD
  T0NQMQ8wDQYDVQQLDAZPQ1AtUUUxFzAVBgNVBAMMDk9DUC1RRS1ST09ULUNBMIIC
  IjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA338oV6IIvllZpr/lWOjMMVZq
  4Smm0qA6BOe3ezZlr2LU5TLsgZeY+Oa1YtwXSAka8rRnuuqNa6gZEHGdL1SHTynB
  rEyq05KErChLabRVYb9aotQHt1+G1GG2Mi11QZ4Pdgsfmrs8NC05703C5V4kEL+q
  NXG88O3J54ySsKp+aD4xvOtZ0uXcVdjAo347/CJEm/2HF9C/uIR8ktJ43ZQPq55c
  tgsJjjY/UBSmOOhDsTfRzv9DVrcWuZYW0ZztG7gfC3d2i2l7dLhaAr76kzZ68aH2
  402ghE1Xh9zDlmWugfqOyT/v6RsE7gL/Dkkuk27Eau3jyRdWVIJroqK2Sd/yJcrQ
  DiG1wAzwb7JVlPi5lkQBrWXti+qgm415+Xfcc9KRZP3hv3tbGVuKmNxONpGjbrMw
  GKV2EMWGnpdKepQ0STWb9SC916iNXO9ffCsPlqgEoV1ONiNfvU9G3cCcRcc1yjtF
  8zbMcqmtsvl+AC1RfmM4n8TesSx56vk/obNsUljtU1/FGQIKRlamey4r/dKDR8kJ
  oyDibv7dUGm5pX5/L7bahRb7LoVg0MbV9bGlqL+hpCbjIO1rouMyy3qu3z+NMGh7
  nzVYULulOjdbVw5u14O4VeonavWByyCFUMK4JKqfUOPNjjS7OEXue1HoCy9LBjIv
  qfPUdeulyX0OtbZ8EhECAwEAAaNjMGEwHQYDVR0OBBYEFG5nokgYqmIIwaW7blM6
  wHVIQwBIMB8GA1UdIwQYMBaAFG5nokgYqmIIwaW7blM6wHVIQwBIMA8GA1UdEwEB
  /wQFMAMBAf8wDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4ICAQAcnBrb
  Cde2jE+iumzlN3TNm6nOvMnomIrMupBInuWI0GvA9rGjv8SC8ZAjfx/fZOY28uLx
  ACiZqKWQT0YARjKCgOSe0RxTG+vpNH6E8FpTEiVIq/N+rgdHCZUJiWoY7BA1FNNq
  3UTlqV6RM+RqsVIptu8lk7fVDehng+zQzYYs4ZV6bSLjBQG3yBUBN1lYnFWe3pnS
  WmLuw22Riuunc5MVdH97modji1UDzQHDbYy0FXt8gLM8DRPIrOe039XO1lO+eWWM
  /NI7sZBU6bSotDh3aTLnHIyJdJ0dnh+/wMIK6h5au/7BMV1oK4JsSmpNCmzP+s3O
  cpNINYhkBRqFViA72D/Vim/meP2Q4J/dKsT2JbprY7X/XIYd1+aS48QAyusat2Gn
  KJ1JQNOoYHGijz8bYHm5JVytMIKU5LJ/Rp9SgK3d0ByqmJR76alzyRdUKa3Pmsw3
  Beq8GQSAdjlyIB6C1FpG7XD4ySz1EjGEcOXiGiEi8l9wjDgLtA20U9ALaMcEdODY
  K8zhyirrdXdV8XHBAE7QBkzcuQAVc9iyTNoqCfJBtvl2HYpH2XoRhxP0rX9NtAYE
  Gc+Yc4Tgf2HAERrwj0B6AfWQaDfcjAJtQ0xorONJJpEZpItV8Cl5dSeOtX7howTB
  BvBHcmyVbaW7PGNBmIM1FBKwi/fBJoawSJlslA==
  -----END CERTIFICATE-----
imageContentSources:
- mirrors:
  - my-registry:5000/ocp/release
  source: quay.io/openshift-release-dev/ocp-v4.0-art-dev
- mirrors:
  - my-registry:5000/ocp/release
  source: registry.svc.ci.openshift.org/ocp/release
baseDomain: qe.azure.devcluster.openshift.com
sshKey: ssh-rsa AAAAABBBBBBCCCCC openshift-qe
2. Create ignition files
$ openshift-install create ignition-configs --dir demo6
3. Create clusters
$ openshift-install create cluster --dir demo6 --log-level  debug
INFO Credentials loaded from file "/root/.azure/osServicePrincipal.json" 
INFO Consuming Install Config from target directory 
INFO Ignition-Configs created in: demo6 and demo6/auth 
[root@preserve-jialiu-ansible ~]# openshift-install create cluster --dir demo6 --log-level  debug
DEBUG OpenShift Installer 4.7.0-0.nightly-2020-10-26-152308 
DEBUG Built from commit de62b01d451857d942ac41dcfdad9385856fde9e 
DEBUG Fetching Metadata...                         
DEBUG Loading Metadata...                          
DEBUG   Loading Cluster ID...                      
DEBUG     Loading Install Config...                
DEBUG       Loading SSH Key...                     
DEBUG       Loading Base Domain...                 
DEBUG         Loading Platform...                  
DEBUG       Loading Cluster Name...                
DEBUG         Loading Base Domain...               
DEBUG         Loading Platform...                  
DEBUG       Loading Pull Secret...                 
DEBUG       Loading Platform...                    
DEBUG     Using Install Config loaded from state file 
DEBUG   Using Cluster ID loaded from state file    
DEBUG   Loading Install Config...                  
DEBUG   Loading Bootstrap Ignition Config...       
DEBUG     Loading Install Config...                
DEBUG     Loading Kubeconfig Admin Internal Client... 
DEBUG       Loading Certificate (admin-kubeconfig-client)... 
DEBUG         Loading Certificate (admin-kubeconfig-signer)... 
DEBUG         Using Certificate (admin-kubeconfig-signer) loaded from state file 
DEBUG       Using Certificate (admin-kubeconfig-client) loaded from state file 
DEBUG       Loading Certificate (kube-apiserver-complete-server-ca-bundle)... 
DEBUG         Loading Certificate (kube-apiserver-localhost-ca-bundle)... 
DEBUG           Loading Certificate (kube-apiserver-localhost-signer)... 
DEBUG           Using Certificate (kube-apiserver-localhost-signer) loaded from state file 
DEBUG         Using Certificate (kube-apiserver-localhost-ca-bundle) loaded from state file 
DEBUG         Loading Certificate (kube-apiserver-service-network-ca-bundle)... 
DEBUG           Loading Certificate (kube-apiserver-service-network-signer)... 
DEBUG           Using Certificate (kube-apiserver-service-network-signer) loaded from state file 
DEBUG         Using Certificate (kube-apiserver-service-network-ca-bundle) loaded from state file 
DEBUG         Loading Certificate (kube-apiserver-lb-ca-bundle)... 
DEBUG           Loading Certificate (kube-apiserver-lb-signer)... 
DEBUG           Using Certificate (kube-apiserver-lb-signer) loaded from state file 
DEBUG         Using Certificate (kube-apiserver-lb-ca-bundle) loaded from state file 
DEBUG       Using Certificate (kube-apiserver-complete-server-ca-bundle) loaded from state file 
DEBUG       Loading Install Config...              
DEBUG     Using Kubeconfig Admin Internal Client loaded from state file 
DEBUG     Loading Kubeconfig Kubelet...            
DEBUG       Loading Certificate (kube-apiserver-complete-server-ca-bundle)... 
DEBUG       Loading Certificate (kubelet-client)... 
DEBUG         Loading Certificate (kubelet-bootstrap-kubeconfig-signer)... 
DEBUG         Using Certificate (kubelet-bootstrap-kubeconfig-signer) loaded from state file 
DEBUG       Using Certificate (kubelet-client) loaded from state file 
DEBUG       Loading Install Config...              
DEBUG     Using Kubeconfig Kubelet loaded from state file 
DEBUG     Loading Kubeconfig Admin Client (Loopback)... 
DEBUG       Loading Certificate (admin-kubeconfig-client)... 
DEBUG       Loading Certificate (kube-apiserver-localhost-ca-bundle)... 
DEBUG       Loading Install Config...              
DEBUG     Using Kubeconfig Admin Client (Loopback) loaded from state file 
DEBUG     Loading Master Machines...               
DEBUG       Loading Cluster ID...                  
DEBUG       Loading Platform Credentials Check...  
DEBUG         Loading Install Config...            
DEBUG       Using Platform Credentials Check loaded from state file 
DEBUG       Loading Install Config...              
DEBUG       Loading Image...                       
DEBUG         Loading Install Config...            
DEBUG       Using Image loaded from state file     
DEBUG       Loading Master Ignition Config...      
DEBUG         Loading Install Config...            
DEBUG         Loading Root CA...                   
DEBUG         Using Root CA loaded from state file 
DEBUG       Loading Master Ignition Config from both state file and target directory 
DEBUG       On-disk Master Ignition Config matches asset in state file 
DEBUG       Using Master Ignition Config loaded from state file 
DEBUG     Using Master Machines loaded from state file 
DEBUG     Loading Worker Machines...               
DEBUG       Loading Cluster ID...                  
DEBUG       Loading Platform Credentials Check...  
DEBUG       Loading Install Config...              
DEBUG       Loading Image...                       
DEBUG       Loading Worker Ignition Config...      
DEBUG         Loading Install Config...            
DEBUG         Loading Root CA...                   
DEBUG       Loading Worker Ignition Config from both state file and target directory 
DEBUG       On-disk Worker Ignition Config matches asset in state file 
DEBUG       Using Worker Ignition Config loaded from state file 
DEBUG     Using Worker Machines loaded from state file 
DEBUG     Loading Common Manifests...              
DEBUG       Loading Cluster ID...                  
DEBUG       Loading Install Config...              
DEBUG       Loading Ingress Config...              
DEBUG         Loading Install Config...            
DEBUG       Using Ingress Config loaded from state file 
DEBUG       Loading DNS Config...                  
DEBUG         Loading Install Config...            
DEBUG         Loading Cluster ID...                
DEBUG         Loading Platform Credentials Check... 
DEBUG       Using DNS Config loaded from state file 
DEBUG       Loading Infrastructure Config...       
DEBUG         Loading Cluster ID...                
DEBUG         Loading Install Config...            
DEBUG         Loading Cloud Provider Config...     
DEBUG           Loading Install Config...          
DEBUG           Loading Cluster ID...              
DEBUG           Loading Platform Credentials Check... 
DEBUG         Using Cloud Provider Config loaded from state file 
DEBUG         Loading Additional Trust Bundle Config... 
DEBUG           Loading Install Config...          
DEBUG         Using Additional Trust Bundle Config loaded from state file 
DEBUG       Using Infrastructure Config loaded from state file 
DEBUG       Loading Network Config...              
DEBUG         Loading Install Config...            
DEBUG         Loading Network CRDs...              
DEBUG         Using Network CRDs loaded from state file 
DEBUG       Using Network Config loaded from state file 
DEBUG       Loading Proxy Config...                
DEBUG         Loading Install Config...            
DEBUG         Loading Network Config...            
DEBUG       Using Proxy Config loaded from state file 
DEBUG       Loading Scheduler Config...            
DEBUG         Loading Install Config...            
DEBUG       Using Scheduler Config loaded from state file 
DEBUG       Loading Image Content Source Policy... 
DEBUG         Loading Install Config...            
DEBUG       Using Image Content Source Policy loaded from state file 
DEBUG       Loading Root CA...                     
DEBUG       Loading Certificate (etcd-signer)...   
DEBUG       Using Certificate (etcd-signer) loaded from state file 
DEBUG       Loading Certificate (etcd-ca-bundle)... 
DEBUG         Loading Certificate (etcd-signer)... 
DEBUG       Using Certificate (etcd-ca-bundle) loaded from state file 
DEBUG       Loading Certificate (etcd-client)...   
DEBUG         Loading Certificate (etcd-signer)... 
DEBUG       Using Certificate (etcd-client) loaded from state file 
DEBUG       Loading Certificate (etcd-metric-ca-bundle)... 
DEBUG         Loading Certificate (etcd-metric-signer)... 
DEBUG         Using Certificate (etcd-metric-signer) loaded from state file 
DEBUG       Using Certificate (etcd-metric-ca-bundle) loaded from state file 
DEBUG       Loading Certificate (etcd-metric-signer)... 
DEBUG       Loading Certificate (etcd-metric-signer-client)... 
DEBUG         Loading Certificate (etcd-metric-signer)... 
DEBUG       Using Certificate (etcd-metric-signer-client) loaded from state file 
DEBUG       Loading Certificate (mcs)...           
DEBUG         Loading Root CA...                   
DEBUG         Loading Install Config...            
DEBUG       Using Certificate (mcs) loaded from state file 
DEBUG       Loading CVOOverrides...                
DEBUG       Using CVOOverrides loaded from state file 
DEBUG       Loading EtcdCAConfigMap...             
DEBUG       Using EtcdCAConfigMap loaded from state file 
DEBUG       Loading EtcdClientSecret...            
DEBUG       Using EtcdClientSecret loaded from state file 
DEBUG       Loading EtcdMetricClientSecret...      
DEBUG       Using EtcdMetricClientSecret loaded from state file 
DEBUG       Loading EtcdMetricServingCAConfigMap... 
DEBUG       Using EtcdMetricServingCAConfigMap loaded from state file 
DEBUG       Loading EtcdMetricSignerSecret...      
DEBUG       Using EtcdMetricSignerSecret loaded from state file 
DEBUG       Loading EtcdNamespace...               
DEBUG       Using EtcdNamespace loaded from state file 
DEBUG       Loading EtcdService...                 
DEBUG       Using EtcdService loaded from state file 
DEBUG       Loading EtcdSignerSecret...            
DEBUG       Using EtcdSignerSecret loaded from state file 
DEBUG       Loading KubeCloudConfig...             
DEBUG       Using KubeCloudConfig loaded from state file 
DEBUG       Loading EtcdServingCAConfigMap...      
DEBUG       Using EtcdServingCAConfigMap loaded from state file 
DEBUG       Loading KubeSystemConfigmapRootCA...   
DEBUG       Using KubeSystemConfigmapRootCA loaded from state file 
DEBUG       Loading MachineConfigServerTLSSecret... 
DEBUG       Using MachineConfigServerTLSSecret loaded from state file 
DEBUG       Loading OpenshiftConfigSecretPullSecret... 
DEBUG       Using OpenshiftConfigSecretPullSecret loaded from state file 
DEBUG       Loading OpenshiftMachineConfigOperator... 
DEBUG       Using OpenshiftMachineConfigOperator loaded from state file 
DEBUG     Using Common Manifests loaded from state file 
DEBUG     Loading Openshift Manifests...           
DEBUG       Loading Install Config...              
DEBUG       Loading Cluster ID...                  
DEBUG       Loading Kubeadmin Password...          
DEBUG       Using Kubeadmin Password loaded from state file 
DEBUG       Loading OpenShift Install (Manifests)... 
DEBUG       Using OpenShift Install (Manifests) loaded from state file 
DEBUG       Loading CloudCredsSecret...            
DEBUG       Using CloudCredsSecret loaded from state file 
DEBUG       Loading KubeadminPasswordSecret...     
DEBUG       Using KubeadminPasswordSecret loaded from state file 
DEBUG       Loading RoleCloudCredsSecretReader...  
DEBUG       Using RoleCloudCredsSecretReader loaded from state file 
DEBUG       Loading Private Cluster Outbound Service... 
DEBUG       Using Private Cluster Outbound Service loaded from state file 
DEBUG       Loading Baremetal Config CR...         
DEBUG       Using Baremetal Config CR loaded from state file 
DEBUG       Loading Image...                       
DEBUG     Using Openshift Manifests loaded from state file 
DEBUG     Loading Proxy Config...                  
DEBUG     Loading Certificate (admin-kubeconfig-ca-bundle)... 
DEBUG       Loading Certificate (admin-kubeconfig-signer)... 
DEBUG     Using Certificate (admin-kubeconfig-ca-bundle) loaded from state file 
DEBUG     Loading Certificate (aggregator)...      
DEBUG     Using Certificate (aggregator) loaded from state file 
DEBUG     Loading Certificate (aggregator-ca-bundle)... 
DEBUG       Loading Certificate (aggregator-signer)... 
DEBUG       Using Certificate (aggregator-signer) loaded from state file 
DEBUG     Using Certificate (aggregator-ca-bundle) loaded from state file 
DEBUG     Loading Certificate (system:kube-apiserver-proxy)... 
DEBUG       Loading Certificate (aggregator-signer)... 
DEBUG     Using Certificate (system:kube-apiserver-proxy) loaded from state file 
DEBUG     Loading Certificate (aggregator-signer)... 
DEBUG     Loading Certificate (system:kube-apiserver-proxy)... 
DEBUG       Loading Certificate (aggregator)...    
DEBUG     Using Certificate (system:kube-apiserver-proxy) loaded from state file 
DEBUG     Loading Bootstrap SSH Key Pair...        
DEBUG     Using Bootstrap SSH Key Pair loaded from state file 
DEBUG     Loading Certificate (etcd-ca-bundle)...  
DEBUG     Loading Certificate (etcd-metric-ca-bundle)... 
DEBUG     Loading Certificate (etcd-metric-signer)... 
DEBUG     Loading Certificate (etcd-metric-signer-client)... 
DEBUG     Loading Certificate (etcd-signer)...     
DEBUG     Loading Certificate (etcd-client)...     
DEBUG     Loading Certificate (journal-gatewayd)... 
DEBUG       Loading Root CA...                     
DEBUG     Using Certificate (journal-gatewayd) loaded from state file 
DEBUG     Loading Certificate (kube-apiserver-lb-ca-bundle)... 
DEBUG     Loading Certificate (kube-apiserver-external-lb-server)... 
DEBUG       Loading Certificate (kube-apiserver-lb-signer)... 
DEBUG       Loading Install Config...              
DEBUG     Using Certificate (kube-apiserver-external-lb-server) loaded from state file 
DEBUG     Loading Certificate (kube-apiserver-internal-lb-server)... 
DEBUG       Loading Certificate (kube-apiserver-lb-signer)... 
DEBUG       Loading Install Config...              
DEBUG     Using Certificate (kube-apiserver-internal-lb-server) loaded from state file 
DEBUG     Loading Certificate (kube-apiserver-lb-signer)... 
DEBUG     Loading Certificate (kube-apiserver-localhost-ca-bundle)... 
DEBUG     Loading Certificate (kube-apiserver-localhost-server)... 
DEBUG       Loading Certificate (kube-apiserver-localhost-signer)... 
DEBUG     Using Certificate (kube-apiserver-localhost-server) loaded from state file 
DEBUG     Loading Certificate (kube-apiserver-localhost-signer)... 
DEBUG     Loading Certificate (kube-apiserver-service-network-ca-bundle)... 
DEBUG     Loading Certificate (kube-apiserver-service-network-server)... 
DEBUG       Loading Certificate (kube-apiserver-service-network-signer)... 
DEBUG       Loading Install Config...              
DEBUG     Using Certificate (kube-apiserver-service-network-server) loaded from state file 
DEBUG     Loading Certificate (kube-apiserver-service-network-signer)... 
DEBUG     Loading Certificate (kube-apiserver-complete-server-ca-bundle)... 
DEBUG     Loading Certificate (kube-apiserver-complete-client-ca-bundle)... 
DEBUG       Loading Certificate (admin-kubeconfig-ca-bundle)... 
DEBUG       Loading Certificate (kubelet-client-ca-bundle)... 
DEBUG         Loading Certificate (kubelet-signer)... 
DEBUG         Using Certificate (kubelet-signer) loaded from state file 
DEBUG       Using Certificate (kubelet-client-ca-bundle) loaded from state file 
DEBUG       Loading Certificate (kube-control-plane-ca-bundle)... 
DEBUG         Loading Certificate (kube-control-plane-signer)... 
DEBUG         Using Certificate (kube-control-plane-signer) loaded from state file 
DEBUG         Loading Certificate (kube-apiserver-lb-signer)... 
DEBUG         Loading Certificate (kube-apiserver-localhost-signer)... 
DEBUG         Loading Certificate (kube-apiserver-service-network-signer)... 
DEBUG       Using Certificate (kube-control-plane-ca-bundle) loaded from state file 
DEBUG       Loading Certificate (kube-apiserver-to-kubelet-ca-bundle)... 
DEBUG         Loading Certificate (kube-apiserver-to-kubelet-signer)... 
DEBUG         Using Certificate (kube-apiserver-to-kubelet-signer) loaded from state file 
DEBUG       Using Certificate (kube-apiserver-to-kubelet-ca-bundle) loaded from state file 
DEBUG       Loading Certificate (kubelet-bootstrap-kubeconfig-ca-bundle)... 
DEBUG         Loading Certificate (kubelet-bootstrap-kubeconfig-signer)... 
DEBUG       Using Certificate (kubelet-bootstrap-kubeconfig-ca-bundle) loaded from state file 
DEBUG     Using Certificate (kube-apiserver-complete-client-ca-bundle) loaded from state file 
DEBUG     Loading Certificate (kube-apiserver-to-kubelet-ca-bundle)... 
DEBUG     Loading Certificate (kube-apiserver-to-kubelet-client)... 
DEBUG       Loading Certificate (kube-apiserver-to-kubelet-signer)... 
DEBUG     Using Certificate (kube-apiserver-to-kubelet-client) loaded from state file 
DEBUG     Loading Certificate (kube-apiserver-to-kubelet-signer)... 
DEBUG     Loading Certificate (kube-control-plane-ca-bundle)... 
DEBUG     Loading Certificate (kube-control-plane-kube-controller-manager-client)... 
DEBUG       Loading Certificate (kube-control-plane-signer)... 
DEBUG     Using Certificate (kube-control-plane-kube-controller-manager-client) loaded from state file 
DEBUG     Loading Certificate (kube-control-plane-kube-scheduler-client)... 
DEBUG       Loading Certificate (kube-control-plane-signer)... 
DEBUG     Using Certificate (kube-control-plane-kube-scheduler-client) loaded from state file 
DEBUG     Loading Certificate (kube-control-plane-signer)... 
DEBUG     Loading Certificate (kubelet-bootstrap-kubeconfig-ca-bundle)... 
DEBUG     Loading Certificate (kubelet-client-ca-bundle)... 
DEBUG     Loading Certificate (kubelet-client)...  
DEBUG     Loading Certificate (kubelet-signer)...  
DEBUG     Loading Certificate (kubelet-serving-ca-bundle)... 
DEBUG       Loading Certificate (kubelet-signer)... 
DEBUG     Using Certificate (kubelet-serving-ca-bundle) loaded from state file 
DEBUG     Loading Certificate (mcs)...             
DEBUG     Loading Root CA...                       
DEBUG     Loading Key Pair (service-account.pub)... 
DEBUG     Using Key Pair (service-account.pub) loaded from state file 
DEBUG     Loading Release Image Pull Spec...       
DEBUG     Using Release Image Pull Spec loaded from state file 
DEBUG     Loading Image...                         
<pending here for ever>

What did you expect to happen?
Installation get passed.

How to reproduce it (as minimally and precisely as possible)?
Always

Anything else we need to know?
1. a common IPI install without additionalTrustBundle in install-config.yaml get passed.
2. This issue does not happen with 
4.6.0-0.nightly-2020-10-26-151252 and 4.7.0-0.nightly-2020-10-22-175439

Comment 1 Johnny Liu 2020-10-27 06:17:04 UTC
Created attachment 1724487 [details]
installer log

Comment 2 Matthew Staebler 2020-10-27 17:25:46 UTC
This looks like a regression from https://github.com/openshift/installer/pull/4287. If the additional trust bundle has parts that are not PEM blocks, then the code to determine if the certs in the ignition configs have expired will loop forever.

Comment 3 John Hixson 2020-10-27 19:20:22 UTC
PR: https://github.com/openshift/installer/pull/4317

Comment 6 Johnny Liu 2020-11-10 08:40:07 UTC
Verified this bug with 4.7.0-0.nightly-2020-11-09-235738, and PASS.

<--snip-->
level=debug msg=    Using Release Image Pull Spec loaded from state file
level=debug msg=    Loading Image...
level=debug msg=  Loading Bootstrap Ignition Config from both state file and target directory
level=debug msg=  On-disk Bootstrap Ignition Config matches asset in state file
level=debug msg=  Using Bootstrap Ignition Config loaded from state file
level=debug msg=Using Metadata loaded from state file
<--snip-->

Comment 9 errata-xmlrpc 2021-02-24 15:28:28 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:5633


Note You need to log in before you can comment on or make changes to this bug.