Bug 1893117 - Maintenance mode on vSphere blocks installation.
Summary: Maintenance mode on vSphere blocks installation.
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 4.6
Hardware: All
OS: Unspecified
medium
medium
Target Milestone: ---
: 4.7.0
Assignee: Jeremiah Stuever
QA Contact: jima
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-10-30 08:50 UTC by David Hernández Fernández
Modified: 2023-12-15 19:58 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-02-24 15:29:18 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift installer pull 4566 0 None closed Bug 1893117: terraform/vsphereprivate: ignore hosts in maintenance mode 2021-02-19 01:18:27 UTC
Red Hat Knowledge Base (Solution) 5693101 0 None None None 2021-01-08 15:34:09 UTC
Red Hat Product Errata RHSA-2020:5633 0 None None None 2021-02-24 15:29:47 UTC

Description David Hernández Fernández 2020-10-30 08:50:03 UTC
Version: Openshift 4.5.14 and 4.6.1

Platform: vSphere IPI

What happened?

Installation will fail when the vSphere Cluster has an ESXi host with active maintenance mode. 
The ova cannot be imported because it seems to be selecting the host with maintenance mode and gives error.

When host is not in maintenance mode, it is working. In the cluster is another host not in maintenance mode and this host will never be selected for OVA deployment of RHCOS somehow. We have DRS on the cluster enabled. Seems to be a problem with terraform here.

I was testing the following procedure for setting up a maintenance node and feels like it is reproducible.
~~~
Procedure
Browse to the host in the vSphere Client.
Right-click the host and select Maintenance Mode > Enter Maintenance Mode.
If the host is part of a partially automated or manual DRS cluster, browse to Cluster > Monitor > DRS > Recommendations and click Apply Recommendations.
If the host is part of an automated DRS cluster, virtual machines are migrated to different hosts when the host enters maintenance mode.
If applicable, click Yes.
Results
The host is in maintenance mode until you select Maintenance Mode > Exit Maintenance Mode.
~~~

Full logs:

penshift-install create cluster --dir=/opt/openshift/install --log-level=debug
DEBUG OpenShift Installer 4.5.14                   
DEBUG Built from commit 9893a482f310ee72089872f1a4caea3dbec34f28 
DEBUG Fetching Metadata...                         
DEBUG Loading Metadata...                          
DEBUG   Loading Cluster ID...                      
DEBUG     Loading Install Config...                
DEBUG       Loading SSH Key...                     
DEBUG       Loading Base Domain...                 
DEBUG         Loading Platform...                  
DEBUG       Loading Cluster Name...                
DEBUG         Loading Base Domain...               
DEBUG         Loading Platform...                  
DEBUG       Loading Pull Secret...                 
DEBUG       Loading Platform...                    
DEBUG     Using Install Config loaded from target directory 
DEBUG   Loading Install Config...                  
DEBUG   Fetching Cluster ID...                     
DEBUG     Fetching Install Config...               
DEBUG     Reusing previously-fetched Install Config 
DEBUG   Generating Cluster ID...                   
DEBUG   Fetching Install Config...                 
DEBUG   Reusing previously-fetched Install Config  
DEBUG Generating Metadata...                       
INFO Consuming Install Config from target directory 
DEBUG Purging asset "Install Config" from disk     
DEBUG Fetching Terraform Variables...              
DEBUG Loading Terraform Variables...               
DEBUG   Loading Cluster ID...                      
DEBUG   Loading Install Config...                  
DEBUG   Loading Image...                           
DEBUG     Loading Install Config...                
DEBUG   Loading BootstrapImage...                  
DEBUG     Loading Install Config...                
DEBUG   Loading Bootstrap Ignition Config...       
DEBUG     Loading Install Config...                
DEBUG     Loading Kubeconfig Admin Internal Client... 
DEBUG       Loading Certificate (admin-kubeconfig-client)... 
DEBUG         Loading Certificate (admin-kubeconfig-signer)... 
DEBUG       Loading Certificate (kube-apiserver-complete-server-ca-bundle)... 
DEBUG         Loading Certificate (kube-apiserver-localhost-ca-bundle)... 
DEBUG           Loading Certificate (kube-apiserver-localhost-signer)... 
DEBUG         Loading Certificate (kube-apiserver-service-network-ca-bundle)... 
DEBUG           Loading Certificate (kube-apiserver-service-network-signer)... 
DEBUG         Loading Certificate (kube-apiserver-lb-ca-bundle)... 
DEBUG           Loading Certificate (kube-apiserver-lb-signer)... 
DEBUG       Loading Install Config...              
DEBUG     Loading Kubeconfig Kubelet...            
DEBUG       Loading Certificate (kube-apiserver-complete-server-ca-bundle)... 
DEBUG       Loading Certificate (kubelet-client)... 
DEBUG         Loading Certificate (kubelet-bootstrap-kubeconfig-signer)... 
DEBUG       Loading Install Config...              
DEBUG     Loading Kubeconfig Admin Client (Loopback)... 
DEBUG       Loading Certificate (admin-kubeconfig-client)... 
DEBUG       Loading Certificate (kube-apiserver-localhost-ca-bundle)... 
DEBUG       Loading Install Config...              
DEBUG     Loading Master Machines...               
DEBUG       Loading Cluster ID...                  
DEBUG       Loading Platform Credentials Check...  
DEBUG         Loading Install Config...            
DEBUG       Loading Install Config...              
DEBUG       Loading Image...                       
DEBUG       Loading Master Ignition Config...      
DEBUG         Loading Install Config...            
DEBUG         Loading Root CA...                   
DEBUG     Loading Worker Machines...               
DEBUG       Loading Cluster ID...                  
DEBUG       Loading Platform Credentials Check...  
DEBUG       Loading Install Config...              
DEBUG       Loading Image...                       
DEBUG       Loading Worker Ignition Config...      
DEBUG         Loading Install Config...            
DEBUG         Loading Root CA...                   
DEBUG     Loading Common Manifests...              
DEBUG       Loading Cluster ID...                  
DEBUG       Loading Install Config...              
DEBUG       Loading Ingress Config...              
DEBUG         Loading Install Config...            
DEBUG       Loading DNS Config...                  
DEBUG         Loading Install Config...            
DEBUG         Loading Cluster ID...                
DEBUG         Loading Platform Credentials Check... 
DEBUG       Loading Infrastructure Config...       
DEBUG         Loading Cluster ID...                
DEBUG         Loading Install Config...            
DEBUG         Loading Cloud Provider Config...     
DEBUG           Loading Install Config...          
DEBUG           Loading Cluster ID...              
DEBUG           Loading Platform Credentials Check... 
DEBUG         Loading Additional Trust Bundle Config... 
DEBUG           Loading Install Config...          
DEBUG       Loading Network Config...              
DEBUG         Loading Install Config...            
DEBUG         Loading Network CRDs...              
DEBUG       Loading Proxy Config...                
DEBUG         Loading Install Config...            
DEBUG         Loading Network Config...            
DEBUG       Loading Scheduler Config...            
DEBUG         Loading Install Config...            
DEBUG       Loading Image Content Source Policy... 
DEBUG         Loading Install Config...            
DEBUG       Loading Root CA...                     
DEBUG       Loading Certificate (etcd-signer)...   
DEBUG       Loading Certificate (etcd-ca-bundle)... 
DEBUG         Loading Certificate (etcd-signer)... 
DEBUG       Loading Certificate (etcd-client)...   
DEBUG         Loading Certificate (etcd-signer)... 
DEBUG       Loading Certificate (etcd-metric-ca-bundle)... 
DEBUG         Loading Certificate (etcd-metric-signer)... 
DEBUG       Loading Certificate (etcd-metric-signer)... 
DEBUG       Loading Certificate (etcd-metric-signer-client)... 
DEBUG         Loading Certificate (etcd-metric-signer)... 
DEBUG       Loading Certificate (mcs)...           
DEBUG         Loading Root CA...                   
DEBUG         Loading Install Config...            
DEBUG       Loading CVOOverrides...                
DEBUG       Loading EtcdCAConfigMap...             
DEBUG       Loading EtcdClientSecret...            
DEBUG       Loading EtcdHostServiceEndpoints...    
DEBUG       Loading EtcdHostService...             
DEBUG       Loading EtcdMetricClientSecret...      
DEBUG       Loading EtcdMetricServingCAConfigMap... 
DEBUG       Loading EtcdMetricSignerSecret...      
DEBUG       Loading EtcdNamespace...               
DEBUG       Loading EtcdService...                 
DEBUG       Loading EtcdSignerSecret...            
DEBUG       Loading KubeCloudConfig...             
DEBUG       Loading EtcdServingCAConfigMap...      
DEBUG       Loading KubeSystemConfigmapRootCA...   
DEBUG       Loading MachineConfigServerTLSSecret... 
DEBUG       Loading OpenshiftConfigSecretPullSecret... 
DEBUG       Loading OpenshiftMachineConfigOperator... 
DEBUG     Loading Openshift Manifests...           
DEBUG       Loading Install Config...              
DEBUG       Loading Cluster ID...                  
DEBUG       Loading Kubeadmin Password...          
DEBUG       Loading OpenShift Install (Manifests)... 
DEBUG       Loading CloudCredsSecret...            
DEBUG       Loading KubeadminPasswordSecret...     
DEBUG       Loading RoleCloudCredsSecretReader...  
DEBUG       Loading Private Cluster Outbound Service... 
DEBUG       Loading Baremetal Config CR...         
DEBUG       Loading Image...                       
DEBUG     Loading Proxy Config...                  
DEBUG     Loading Certificate (admin-kubeconfig-ca-bundle)... 
DEBUG       Loading Certificate (admin-kubeconfig-signer)... 
DEBUG     Loading Certificate (aggregator)...      
DEBUG     Loading Certificate (aggregator-ca-bundle)... 
DEBUG       Loading Certificate (aggregator-signer)... 
DEBUG     Loading Certificate (system:kube-apiserver-proxy)... 
DEBUG       Loading Certificate (aggregator-signer)... 
DEBUG     Loading Certificate (aggregator-signer)... 
DEBUG     Loading Certificate (system:kube-apiserver-proxy)... 
DEBUG       Loading Certificate (aggregator)...    
DEBUG     Loading Bootstrap SSH Key Pair...        
DEBUG     Loading Certificate (etcd-ca-bundle)...  
DEBUG     Loading Certificate (etcd-metric-ca-bundle)... 
DEBUG     Loading Certificate (etcd-metric-signer)... 
DEBUG     Loading Certificate (etcd-metric-signer-client)... 
DEBUG     Loading Certificate (etcd-signer)...     
DEBUG     Loading Certificate (etcd-client)...     
DEBUG     Loading Certificate (journal-gatewayd)... 
DEBUG       Loading Root CA...                     
DEBUG     Loading Certificate (kube-apiserver-lb-ca-bundle)... 
DEBUG     Loading Certificate (kube-apiserver-external-lb-server)... 
DEBUG       Loading Certificate (kube-apiserver-lb-signer)... 
DEBUG       Loading Install Config...              
DEBUG     Loading Certificate (kube-apiserver-internal-lb-server)... 
DEBUG       Loading Certificate (kube-apiserver-lb-signer)... 
DEBUG       Loading Install Config...              
DEBUG     Loading Certificate (kube-apiserver-lb-signer)... 
DEBUG     Loading Certificate (kube-apiserver-localhost-ca-bundle)... 
DEBUG     Loading Certificate (kube-apiserver-localhost-server)... 
DEBUG       Loading Certificate (kube-apiserver-localhost-signer)... 
DEBUG     Loading Certificate (kube-apiserver-localhost-signer)... 
DEBUG     Loading Certificate (kube-apiserver-service-network-ca-bundle)... 
DEBUG     Loading Certificate (kube-apiserver-service-network-server)... 
DEBUG       Loading Certificate (kube-apiserver-service-network-signer)... 
DEBUG       Loading Install Config...              
DEBUG     Loading Certificate (kube-apiserver-service-network-signer)... 
DEBUG     Loading Certificate (kube-apiserver-complete-server-ca-bundle)... 
DEBUG     Loading Certificate (kube-apiserver-complete-client-ca-bundle)... 
DEBUG       Loading Certificate (admin-kubeconfig-ca-bundle)... 
DEBUG       Loading Certificate (kubelet-client-ca-bundle)... 
DEBUG         Loading Certificate (kubelet-signer)... 
DEBUG       Loading Certificate (kube-control-plane-ca-bundle)... 
DEBUG         Loading Certificate (kube-control-plane-signer)... 
DEBUG         Loading Certificate (kube-apiserver-lb-signer)... 
DEBUG         Loading Certificate (kube-apiserver-localhost-signer)... 
DEBUG         Loading Certificate (kube-apiserver-service-network-signer)... 
DEBUG       Loading Certificate (kube-apiserver-to-kubelet-ca-bundle)... 
DEBUG         Loading Certificate (kube-apiserver-to-kubelet-signer)... 
DEBUG       Loading Certificate (kubelet-bootstrap-kubeconfig-ca-bundle)... 
DEBUG         Loading Certificate (kubelet-bootstrap-kubeconfig-signer)... 
DEBUG     Loading Certificate (kube-apiserver-to-kubelet-ca-bundle)... 
DEBUG     Loading Certificate (kube-apiserver-to-kubelet-client)... 
DEBUG       Loading Certificate (kube-apiserver-to-kubelet-signer)... 
DEBUG     Loading Certificate (kube-apiserver-to-kubelet-signer)... 
DEBUG     Loading Certificate (kube-control-plane-ca-bundle)... 
DEBUG     Loading Certificate (kube-control-plane-kube-controller-manager-client)... 
DEBUG       Loading Certificate (kube-control-plane-signer)... 
DEBUG     Loading Certificate (kube-control-plane-kube-scheduler-client)... 
DEBUG       Loading Certificate (kube-control-plane-signer)... 
DEBUG     Loading Certificate (kube-control-plane-signer)... 
DEBUG     Loading Certificate (kubelet-bootstrap-kubeconfig-ca-bundle)... 
DEBUG     Loading Certificate (kubelet-client-ca-bundle)... 
DEBUG     Loading Certificate (kubelet-client)...  
DEBUG     Loading Certificate (kubelet-signer)...  
DEBUG     Loading Certificate (kubelet-serving-ca-bundle)... 
DEBUG       Loading Certificate (kubelet-signer)... 
DEBUG     Loading Certificate (mcs)...             
DEBUG     Loading Root CA...                       
DEBUG     Loading Key Pair (service-account.pub)... 
DEBUG     Loading Release Image Pull Spec...       
DEBUG     Loading Image...                         
DEBUG   Loading Master Ignition Config...          
DEBUG   Loading Master Machines...                 
DEBUG   Loading Worker Machines...                 
DEBUG   Fetching Cluster ID...                     
DEBUG   Reusing previously-fetched Cluster ID      
DEBUG   Fetching Install Config...                 
DEBUG   Reusing previously-fetched Install Config  
DEBUG   Fetching Image...                          
DEBUG     Fetching Install Config...               
DEBUG     Reusing previously-fetched Install Config 
DEBUG   Generating Image...                        
DEBUG   Fetching BootstrapImage...                 
DEBUG     Fetching Install Config...               
DEBUG     Reusing previously-fetched Install Config 
DEBUG   Generating BootstrapImage...               
DEBUG   Fetching Bootstrap Ignition Config...      
DEBUG     Fetching Install Config...               
DEBUG     Reusing previously-fetched Install Config 
DEBUG     Fetching Kubeconfig Admin Internal Client... 
DEBUG       Fetching Certificate (admin-kubeconfig-client)... 
DEBUG         Fetching Certificate (admin-kubeconfig-signer)... 
DEBUG         Generating Certificate (admin-kubeconfig-signer)... 
DEBUG       Generating Certificate (admin-kubeconfig-client)... 
DEBUG       Fetching Certificate (kube-apiserver-complete-server-ca-bundle)... 
DEBUG         Fetching Certificate (kube-apiserver-localhost-ca-bundle)... 
DEBUG           Fetching Certificate (kube-apiserver-localhost-signer)... 
DEBUG           Generating Certificate (kube-apiserver-localhost-signer)... 
DEBUG         Generating Certificate (kube-apiserver-localhost-ca-bundle)... 
DEBUG         Fetching Certificate (kube-apiserver-service-network-ca-bundle)... 
DEBUG           Fetching Certificate (kube-apiserver-service-network-signer)... 
DEBUG           Generating Certificate (kube-apiserver-service-network-signer)... 
DEBUG         Generating Certificate (kube-apiserver-service-network-ca-bundle)... 
DEBUG         Fetching Certificate (kube-apiserver-lb-ca-bundle)... 
DEBUG           Fetching Certificate (kube-apiserver-lb-signer)... 
DEBUG           Generating Certificate (kube-apiserver-lb-signer)... 
DEBUG         Generating Certificate (kube-apiserver-lb-ca-bundle)... 
DEBUG       Generating Certificate (kube-apiserver-complete-server-ca-bundle)... 
DEBUG       Fetching Install Config...             
DEBUG       Reusing previously-fetched Install Config 
DEBUG     Generating Kubeconfig Admin Internal Client... 
DEBUG     Fetching Kubeconfig Kubelet...           
DEBUG       Fetching Certificate (kube-apiserver-complete-server-ca-bundle)... 
DEBUG       Reusing previously-fetched Certificate (kube-apiserver-complete-server-ca-bundle) 
DEBUG       Fetching Certificate (kubelet-client)... 
DEBUG         Fetching Certificate (kubelet-bootstrap-kubeconfig-signer)... 
DEBUG         Generating Certificate (kubelet-bootstrap-kubeconfig-signer)... 
DEBUG       Generating Certificate (kubelet-client)... 
DEBUG       Fetching Install Config...             
DEBUG       Reusing previously-fetched Install Config 
DEBUG     Generating Kubeconfig Kubelet...         
DEBUG     Fetching Kubeconfig Admin Client (Loopback)... 
DEBUG       Fetching Certificate (admin-kubeconfig-client)... 
DEBUG       Reusing previously-fetched Certificate (admin-kubeconfig-client) 
DEBUG       Fetching Certificate (kube-apiserver-localhost-ca-bundle)... 
DEBUG       Reusing previously-fetched Certificate (kube-apiserver-localhost-ca-bundle) 
DEBUG       Fetching Install Config...             
DEBUG       Reusing previously-fetched Install Config 
DEBUG     Generating Kubeconfig Admin Client (Loopback)... 
DEBUG     Fetching Master Machines...              
DEBUG       Fetching Cluster ID...                 
DEBUG       Reusing previously-fetched Cluster ID  
DEBUG       Fetching Platform Credentials Check... 
DEBUG         Fetching Install Config...           
DEBUG         Reusing previously-fetched Install Config 
DEBUG       Generating Platform Credentials Check... 
DEBUG       Fetching Install Config...             
DEBUG       Reusing previously-fetched Install Config 
DEBUG       Fetching Image...                      
DEBUG       Reusing previously-fetched Image       
DEBUG       Fetching Master Ignition Config...     
DEBUG         Fetching Install Config...           
DEBUG         Reusing previously-fetched Install Config 
DEBUG         Fetching Root CA...                  
DEBUG         Generating Root CA...                
DEBUG       Generating Master Ignition Config...   
DEBUG     Generating Master Machines...            
DEBUG     Fetching Worker Machines...              
DEBUG       Fetching Cluster ID...                 
DEBUG       Reusing previously-fetched Cluster ID  
DEBUG       Fetching Platform Credentials Check... 
DEBUG       Reusing previously-fetched Platform Credentials Check 
DEBUG       Fetching Install Config...             
DEBUG       Reusing previously-fetched Install Config 
DEBUG       Fetching Image...                      
DEBUG       Reusing previously-fetched Image       
DEBUG       Fetching Worker Ignition Config...     
DEBUG         Fetching Install Config...           
DEBUG         Reusing previously-fetched Install Config 
DEBUG         Fetching Root CA...                  
DEBUG         Reusing previously-fetched Root CA   
DEBUG       Generating Worker Ignition Config...   
DEBUG     Generating Worker Machines...            
DEBUG     Fetching Common Manifests...             
DEBUG       Fetching Cluster ID...                 
DEBUG       Reusing previously-fetched Cluster ID  
DEBUG       Fetching Install Config...             
DEBUG       Reusing previously-fetched Install Config 
DEBUG       Fetching Ingress Config...             
DEBUG         Fetching Install Config...           
DEBUG         Reusing previously-fetched Install Config 
DEBUG       Generating Ingress Config...           
DEBUG       Fetching DNS Config...                 
DEBUG         Fetching Install Config...           
DEBUG         Reusing previously-fetched Install Config 
DEBUG         Fetching Cluster ID...               
DEBUG         Reusing previously-fetched Cluster ID 
DEBUG         Fetching Platform Credentials Check... 
DEBUG         Reusing previously-fetched Platform Credentials Check 
DEBUG       Generating DNS Config...               
DEBUG       Fetching Infrastructure Config...      
DEBUG         Fetching Cluster ID...               
DEBUG         Reusing previously-fetched Cluster ID 
DEBUG         Fetching Install Config...           
DEBUG         Reusing previously-fetched Install Config 
DEBUG         Fetching Cloud Provider Config...    
DEBUG           Fetching Install Config...         
DEBUG           Reusing previously-fetched Install Config 
DEBUG           Fetching Cluster ID...             
DEBUG           Reusing previously-fetched Cluster ID 
DEBUG           Fetching Platform Credentials Check... 
DEBUG           Reusing previously-fetched Platform Credentials Check 
DEBUG         Generating Cloud Provider Config...  
DEBUG         Fetching Additional Trust Bundle Config... 
DEBUG           Fetching Install Config...         
DEBUG           Reusing previously-fetched Install Config 
DEBUG         Generating Additional Trust Bundle Config... 
DEBUG       Generating Infrastructure Config...    
DEBUG       Fetching Network Config...             
DEBUG         Fetching Install Config...           
DEBUG         Reusing previously-fetched Install Config 
DEBUG         Fetching Network CRDs...             
DEBUG         Generating Network CRDs...           
DEBUG       Generating Network Config...           
DEBUG       Fetching Proxy Config...               
DEBUG         Fetching Install Config...           
DEBUG         Reusing previously-fetched Install Config 
DEBUG         Fetching Network Config...           
DEBUG         Reusing previously-fetched Network Config 
DEBUG       Generating Proxy Config...             
DEBUG       Fetching Scheduler Config...           
DEBUG         Fetching Install Config...           
DEBUG         Reusing previously-fetched Install Config 
DEBUG       Generating Scheduler Config...         
DEBUG       Fetching Image Content Source Policy... 
DEBUG         Fetching Install Config...           
DEBUG         Reusing previously-fetched Install Config 
DEBUG       Generating Image Content Source Policy... 
DEBUG       Fetching Root CA...                    
DEBUG       Reusing previously-fetched Root CA     
DEBUG       Fetching Certificate (etcd-signer)...  
DEBUG       Generating Certificate (etcd-signer)... 
DEBUG       Fetching Certificate (etcd-ca-bundle)... 
DEBUG         Fetching Certificate (etcd-signer)... 
DEBUG         Reusing previously-fetched Certificate (etcd-signer) 
DEBUG       Generating Certificate (etcd-ca-bundle)... 
DEBUG       Fetching Certificate (etcd-client)...  
DEBUG         Fetching Certificate (etcd-signer)... 
DEBUG         Reusing previously-fetched Certificate (etcd-signer) 
DEBUG       Generating Certificate (etcd-client)... 
DEBUG       Fetching Certificate (etcd-metric-ca-bundle)... 
DEBUG         Fetching Certificate (etcd-metric-signer)... 
DEBUG         Generating Certificate (etcd-metric-signer)... 
DEBUG       Generating Certificate (etcd-metric-ca-bundle)... 
DEBUG       Fetching Certificate (etcd-metric-signer)... 
DEBUG       Reusing previously-fetched Certificate (etcd-metric-signer) 
DEBUG       Fetching Certificate (etcd-metric-signer-client)... 
DEBUG         Fetching Certificate (etcd-metric-signer)... 
DEBUG         Reusing previously-fetched Certificate (etcd-metric-signer) 
DEBUG       Generating Certificate (etcd-metric-signer-client)... 
DEBUG       Fetching Certificate (mcs)...          
DEBUG         Fetching Root CA...                  
DEBUG         Reusing previously-fetched Root CA   
DEBUG         Fetching Install Config...           
DEBUG         Reusing previously-fetched Install Config 
DEBUG       Generating Certificate (mcs)...        
DEBUG       Fetching CVOOverrides...               
DEBUG       Generating CVOOverrides...             
DEBUG       Fetching EtcdCAConfigMap...            
DEBUG       Generating EtcdCAConfigMap...          
DEBUG       Fetching EtcdClientSecret...           
DEBUG       Generating EtcdClientSecret...         
DEBUG       Fetching EtcdHostServiceEndpoints...   
DEBUG       Generating EtcdHostServiceEndpoints... 
DEBUG       Fetching EtcdHostService...            
DEBUG       Generating EtcdHostService...          
DEBUG       Fetching EtcdMetricClientSecret...     
DEBUG       Generating EtcdMetricClientSecret...   
DEBUG       Fetching EtcdMetricServingCAConfigMap... 
DEBUG       Generating EtcdMetricServingCAConfigMap... 
DEBUG       Fetching EtcdMetricSignerSecret...     
DEBUG       Generating EtcdMetricSignerSecret...   
DEBUG       Fetching EtcdNamespace...              
DEBUG       Generating EtcdNamespace...            
DEBUG       Fetching EtcdService...                
DEBUG       Generating EtcdService...              
DEBUG       Fetching EtcdSignerSecret...           
DEBUG       Generating EtcdSignerSecret...         
DEBUG       Fetching KubeCloudConfig...            
DEBUG       Generating KubeCloudConfig...          
DEBUG       Fetching EtcdServingCAConfigMap...     
DEBUG       Generating EtcdServingCAConfigMap...   
DEBUG       Fetching KubeSystemConfigmapRootCA...  
DEBUG       Generating KubeSystemConfigmapRootCA... 
DEBUG       Fetching MachineConfigServerTLSSecret... 
DEBUG       Generating MachineConfigServerTLSSecret... 
DEBUG       Fetching OpenshiftConfigSecretPullSecret... 
DEBUG       Generating OpenshiftConfigSecretPullSecret... 
DEBUG       Fetching OpenshiftMachineConfigOperator... 
DEBUG       Generating OpenshiftMachineConfigOperator... 
DEBUG     Generating Common Manifests...           
DEBUG     Fetching Openshift Manifests...          
DEBUG       Fetching Install Config...             
DEBUG       Reusing previously-fetched Install Config 
DEBUG       Fetching Cluster ID...                 
DEBUG       Reusing previously-fetched Cluster ID  
DEBUG       Fetching Kubeadmin Password...         
DEBUG       Generating Kubeadmin Password...       
DEBUG       Fetching OpenShift Install (Manifests)... 
DEBUG       Generating OpenShift Install (Manifests)... 
DEBUG       Fetching CloudCredsSecret...           
DEBUG       Generating CloudCredsSecret...         
DEBUG       Fetching KubeadminPasswordSecret...    
DEBUG       Generating KubeadminPasswordSecret...  
DEBUG       Fetching RoleCloudCredsSecretReader... 
DEBUG       Generating RoleCloudCredsSecretReader... 
DEBUG       Fetching Private Cluster Outbound Service... 
DEBUG       Generating Private Cluster Outbound Service... 
DEBUG       Fetching Baremetal Config CR...        
DEBUG       Generating Baremetal Config CR...      
DEBUG       Fetching Image...                      
DEBUG       Reusing previously-fetched Image       
DEBUG     Generating Openshift Manifests...        
DEBUG     Fetching Proxy Config...                 
DEBUG     Reusing previously-fetched Proxy Config  
DEBUG     Fetching Certificate (admin-kubeconfig-ca-bundle)... 
DEBUG       Fetching Certificate (admin-kubeconfig-signer)... 
DEBUG       Reusing previously-fetched Certificate (admin-kubeconfig-signer) 
DEBUG     Generating Certificate (admin-kubeconfig-ca-bundle)... 
DEBUG     Fetching Certificate (aggregator)...     
DEBUG     Generating Certificate (aggregator)...   
DEBUG     Fetching Certificate (aggregator-ca-bundle)... 
DEBUG       Fetching Certificate (aggregator-signer)... 
DEBUG       Generating Certificate (aggregator-signer)... 
DEBUG     Generating Certificate (aggregator-ca-bundle)... 
DEBUG     Fetching Certificate (system:kube-apiserver-proxy)... 
DEBUG       Fetching Certificate (aggregator-signer)... 
DEBUG       Reusing previously-fetched Certificate (aggregator-signer) 
DEBUG     Generating Certificate (system:kube-apiserver-proxy)... 
DEBUG     Fetching Certificate (aggregator-signer)... 
DEBUG     Reusing previously-fetched Certificate (aggregator-signer) 
DEBUG     Fetching Certificate (system:kube-apiserver-proxy)... 
DEBUG       Fetching Certificate (aggregator)...   
DEBUG       Reusing previously-fetched Certificate (aggregator) 
DEBUG     Generating Certificate (system:kube-apiserver-proxy)... 
DEBUG     Fetching Bootstrap SSH Key Pair...       
DEBUG     Generating Bootstrap SSH Key Pair...     
DEBUG     Fetching Certificate (etcd-ca-bundle)... 
DEBUG     Reusing previously-fetched Certificate (etcd-ca-bundle) 
DEBUG     Fetching Certificate (etcd-metric-ca-bundle)... 
DEBUG     Reusing previously-fetched Certificate (etcd-metric-ca-bundle) 
DEBUG     Fetching Certificate (etcd-metric-signer)... 
DEBUG     Reusing previously-fetched Certificate (etcd-metric-signer) 
DEBUG     Fetching Certificate (etcd-metric-signer-client)... 
DEBUG     Reusing previously-fetched Certificate (etcd-metric-signer-client) 
DEBUG     Fetching Certificate (etcd-signer)...    
DEBUG     Reusing previously-fetched Certificate (etcd-signer) 
DEBUG     Fetching Certificate (etcd-client)...    
DEBUG     Reusing previously-fetched Certificate (etcd-client) 
DEBUG     Fetching Certificate (journal-gatewayd)... 
DEBUG       Fetching Root CA...                    
DEBUG       Reusing previously-fetched Root CA     
DEBUG     Generating Certificate (journal-gatewayd)... 
DEBUG     Fetching Certificate (kube-apiserver-lb-ca-bundle)... 
DEBUG     Reusing previously-fetched Certificate (kube-apiserver-lb-ca-bundle) 
DEBUG     Fetching Certificate (kube-apiserver-external-lb-server)... 
DEBUG       Fetching Certificate (kube-apiserver-lb-signer)... 
DEBUG       Reusing previously-fetched Certificate (kube-apiserver-lb-signer) 
DEBUG       Fetching Install Config...             
DEBUG       Reusing previously-fetched Install Config 
DEBUG     Generating Certificate (kube-apiserver-external-lb-server)... 
DEBUG     Fetching Certificate (kube-apiserver-internal-lb-server)... 
DEBUG       Fetching Certificate (kube-apiserver-lb-signer)... 
DEBUG       Reusing previously-fetched Certificate (kube-apiserver-lb-signer) 
DEBUG       Fetching Install Config...             
DEBUG       Reusing previously-fetched Install Config 
DEBUG     Generating Certificate (kube-apiserver-internal-lb-server)... 
DEBUG     Fetching Certificate (kube-apiserver-lb-signer)... 
DEBUG     Reusing previously-fetched Certificate (kube-apiserver-lb-signer) 
DEBUG     Fetching Certificate (kube-apiserver-localhost-ca-bundle)... 
DEBUG     Reusing previously-fetched Certificate (kube-apiserver-localhost-ca-bundle) 
DEBUG     Fetching Certificate (kube-apiserver-localhost-server)... 
DEBUG       Fetching Certificate (kube-apiserver-localhost-signer)... 
DEBUG       Reusing previously-fetched Certificate (kube-apiserver-localhost-signer) 
DEBUG     Generating Certificate (kube-apiserver-localhost-server)... 
DEBUG     Fetching Certificate (kube-apiserver-localhost-signer)... 
DEBUG     Reusing previously-fetched Certificate (kube-apiserver-localhost-signer) 
DEBUG     Fetching Certificate (kube-apiserver-service-network-ca-bundle)... 
DEBUG     Reusing previously-fetched Certificate (kube-apiserver-service-network-ca-bundle) 
DEBUG     Fetching Certificate (kube-apiserver-service-network-server)... 
DEBUG       Fetching Certificate (kube-apiserver-service-network-signer)... 
DEBUG       Reusing previously-fetched Certificate (kube-apiserver-service-network-signer) 
DEBUG       Fetching Install Config...             
DEBUG       Reusing previously-fetched Install Config 
DEBUG     Generating Certificate (kube-apiserver-service-network-server)... 
DEBUG     Fetching Certificate (kube-apiserver-service-network-signer)... 
DEBUG     Reusing previously-fetched Certificate (kube-apiserver-service-network-signer) 
DEBUG     Fetching Certificate (kube-apiserver-complete-server-ca-bundle)... 
DEBUG     Reusing previously-fetched Certificate (kube-apiserver-complete-server-ca-bundle) 
DEBUG     Fetching Certificate (kube-apiserver-complete-client-ca-bundle)... 
DEBUG       Fetching Certificate (admin-kubeconfig-ca-bundle)... 
DEBUG       Reusing previously-fetched Certificate (admin-kubeconfig-ca-bundle) 
DEBUG       Fetching Certificate (kubelet-client-ca-bundle)... 
DEBUG         Fetching Certificate (kubelet-signer)... 
DEBUG         Generating Certificate (kubelet-signer)... 
DEBUG       Generating Certificate (kubelet-client-ca-bundle)... 
DEBUG       Fetching Certificate (kube-control-plane-ca-bundle)... 
DEBUG         Fetching Certificate (kube-control-plane-signer)... 
DEBUG         Generating Certificate (kube-control-plane-signer)... 
DEBUG         Fetching Certificate (kube-apiserver-lb-signer)... 
DEBUG         Reusing previously-fetched Certificate (kube-apiserver-lb-signer) 
DEBUG         Fetching Certificate (kube-apiserver-localhost-signer)... 
DEBUG         Reusing previously-fetched Certificate (kube-apiserver-localhost-signer) 
DEBUG         Fetching Certificate (kube-apiserver-service-network-signer)... 
DEBUG         Reusing previously-fetched Certificate (kube-apiserver-service-network-signer) 
DEBUG       Generating Certificate (kube-control-plane-ca-bundle)... 
DEBUG       Fetching Certificate (kube-apiserver-to-kubelet-ca-bundle)... 
DEBUG         Fetching Certificate (kube-apiserver-to-kubelet-signer)... 
DEBUG         Generating Certificate (kube-apiserver-to-kubelet-signer)... 
DEBUG       Generating Certificate (kube-apiserver-to-kubelet-ca-bundle)... 
DEBUG       Fetching Certificate (kubelet-bootstrap-kubeconfig-ca-bundle)... 
DEBUG         Fetching Certificate (kubelet-bootstrap-kubeconfig-signer)... 
DEBUG         Reusing previously-fetched Certificate (kubelet-bootstrap-kubeconfig-signer) 
DEBUG       Generating Certificate (kubelet-bootstrap-kubeconfig-ca-bundle)... 
DEBUG     Generating Certificate (kube-apiserver-complete-client-ca-bundle)... 
DEBUG     Fetching Certificate (kube-apiserver-to-kubelet-ca-bundle)... 
DEBUG     Reusing previously-fetched Certificate (kube-apiserver-to-kubelet-ca-bundle) 
DEBUG     Fetching Certificate (kube-apiserver-to-kubelet-client)... 
DEBUG       Fetching Certificate (kube-apiserver-to-kubelet-signer)... 
DEBUG       Reusing previously-fetched Certificate (kube-apiserver-to-kubelet-signer) 
DEBUG     Generating Certificate (kube-apiserver-to-kubelet-client)... 
DEBUG     Fetching Certificate (kube-apiserver-to-kubelet-signer)... 
DEBUG     Reusing previously-fetched Certificate (kube-apiserver-to-kubelet-signer) 
DEBUG     Fetching Certificate (kube-control-plane-ca-bundle)... 
DEBUG     Reusing previously-fetched Certificate (kube-control-plane-ca-bundle) 
DEBUG     Fetching Certificate (kube-control-plane-kube-controller-manager-client)... 
DEBUG       Fetching Certificate (kube-control-plane-signer)... 
DEBUG       Reusing previously-fetched Certificate (kube-control-plane-signer) 
DEBUG     Generating Certificate (kube-control-plane-kube-controller-manager-client)... 
DEBUG     Fetching Certificate (kube-control-plane-kube-scheduler-client)... 
DEBUG       Fetching Certificate (kube-control-plane-signer)... 
DEBUG       Reusing previously-fetched Certificate (kube-control-plane-signer) 
DEBUG     Generating Certificate (kube-control-plane-kube-scheduler-client)... 
DEBUG     Fetching Certificate (kube-control-plane-signer)... 
DEBUG     Reusing previously-fetched Certificate (kube-control-plane-signer) 
DEBUG     Fetching Certificate (kubelet-bootstrap-kubeconfig-ca-bundle)... 
DEBUG     Reusing previously-fetched Certificate (kubelet-bootstrap-kubeconfig-ca-bundle) 
DEBUG     Fetching Certificate (kubelet-client-ca-bundle)... 
DEBUG     Reusing previously-fetched Certificate (kubelet-client-ca-bundle) 
DEBUG     Fetching Certificate (kubelet-client)... 
DEBUG     Reusing previously-fetched Certificate (kubelet-client) 
DEBUG     Fetching Certificate (kubelet-signer)... 
DEBUG     Reusing previously-fetched Certificate (kubelet-signer) 
DEBUG     Fetching Certificate (kubelet-serving-ca-bundle)... 
DEBUG       Fetching Certificate (kubelet-signer)... 
DEBUG       Reusing previously-fetched Certificate (kubelet-signer) 
DEBUG     Generating Certificate (kubelet-serving-ca-bundle)... 
DEBUG     Fetching Certificate (mcs)...            
DEBUG     Reusing previously-fetched Certificate (mcs) 
DEBUG     Fetching Root CA...                      
DEBUG     Reusing previously-fetched Root CA       
DEBUG     Fetching Key Pair (service-account.pub)... 
DEBUG     Generating Key Pair (service-account.pub)... 
DEBUG     Fetching Release Image Pull Spec...      
DEBUG     Generating Release Image Pull Spec...    
DEBUG Using internal constant for release image quay.io/openshift-release-dev/ocp-release@sha256:95cfe9273aecb9a0070176210477491c347f8e69e41759063642edf8bb8aceb6 
DEBUG     Fetching Image...                        
DEBUG     Reusing previously-fetched Image         
DEBUG   Generating Bootstrap Ignition Config...    
DEBUG   Fetching Master Ignition Config...         
DEBUG   Reusing previously-fetched Master Ignition Config 
DEBUG   Fetching Master Machines...                
DEBUG   Reusing previously-fetched Master Machines 
DEBUG   Fetching Worker Machines...                
DEBUG   Reusing previously-fetched Worker Machines 
DEBUG Generating Terraform Variables...            
INFO Obtaining RHCOS image file from 'https://releases-art-rhcos.svc.ci.openshift.org/art/storage/releases/rhcos-4.5/45.82.202008010929-0/x86_64/rhcos-45.82.202008010929-0-vmware.x86_64.ova?sha256=e7fc00902051d3933bde11fe5e25fa6d252f5388a6d6daa54107a756aadcf899' 
INFO The file was found in cache: /root/.cache/openshift-installer/image_cache/abc7fccbe43d10b0fa665c80e3865ac7. Reusing... 
DEBUG Fetching Kubeconfig Admin Client...          
DEBUG Loading Kubeconfig Admin Client...           
DEBUG   Loading Certificate (admin-kubeconfig-client)... 
DEBUG   Loading Certificate (kube-apiserver-complete-server-ca-bundle)... 
DEBUG   Loading Install Config...                  
DEBUG   Fetching Certificate (admin-kubeconfig-client)... 
DEBUG   Reusing previously-fetched Certificate (admin-kubeconfig-client) 
DEBUG   Fetching Certificate (kube-apiserver-complete-server-ca-bundle)... 
DEBUG   Reusing previously-fetched Certificate (kube-apiserver-complete-server-ca-bundle) 
DEBUG   Fetching Install Config...                 
DEBUG   Reusing previously-fetched Install Config  
DEBUG Generating Kubeconfig Admin Client...        
DEBUG Fetching Kubeadmin Password...               
DEBUG Reusing previously-fetched Kubeadmin Password 
DEBUG Fetching Certificate (journal-gatewayd)...   
DEBUG Reusing previously-fetched Certificate (journal-gatewayd) 
DEBUG Fetching Cluster...                          
DEBUG Loading Cluster...                           
DEBUG   Loading Cluster ID...                      
DEBUG   Loading Install Config...                  
DEBUG   Loading Platform Credentials Check...      
DEBUG   Loading Platform Permissions Check...      
DEBUG     Loading Install Config...                
DEBUG   Loading Platform Provisioning Check...     
DEBUG     Loading Install Config...                
DEBUG   Loading Terraform Variables...             
DEBUG   Loading Kubeadmin Password...              
DEBUG   Fetching Cluster ID...                     
DEBUG   Reusing previously-fetched Cluster ID      
DEBUG   Fetching Install Config...                 
DEBUG   Reusing previously-fetched Install Config  
DEBUG   Fetching Platform Credentials Check...     
DEBUG   Reusing previously-fetched Platform Credentials Check 
DEBUG   Fetching Platform Permissions Check...     
DEBUG     Fetching Install Config...               
DEBUG     Reusing previously-fetched Install Config 
DEBUG   Generating Platform Permissions Check...   
DEBUG   Fetching Platform Provisioning Check...    
DEBUG     Fetching Install Config...               
DEBUG     Reusing previously-fetched Install Config 
DEBUG   Generating Platform Provisioning Check...  
DEBUG   Fetching Terraform Variables...            
DEBUG   Reusing previously-fetched Terraform Variables 
DEBUG   Fetching Kubeadmin Password...             
DEBUG   Reusing previously-fetched Kubeadmin Password 
DEBUG Generating Cluster...                        
INFO Creating infrastructure resources...         
DEBUG Symlinking plugin terraform-provider-azurerm src: "/opt/openshift/openshift-install" dst: "/tmp/openshift-install-987352622/plugins/terraform-provider-azurerm" 
DEBUG Symlinking plugin terraform-provider-azureprivatedns src: "/opt/openshift/openshift-install" dst: "/tmp/openshift-install-987352622/plugins/terraform-provider-azureprivatedns" 
DEBUG Symlinking plugin terraform-provider-google src: "/opt/openshift/openshift-install" dst: "/tmp/openshift-install-987352622/plugins/terraform-provider-google" 
DEBUG Symlinking plugin terraform-provider-random src: "/opt/openshift/openshift-install" dst: "/tmp/openshift-install-987352622/plugins/terraform-provider-random" 
DEBUG Symlinking plugin terraform-provider-aws src: "/opt/openshift/openshift-install" dst: "/tmp/openshift-install-987352622/plugins/terraform-provider-aws" 
DEBUG Symlinking plugin terraform-provider-ignition src: "/opt/openshift/openshift-install" dst: "/tmp/openshift-install-987352622/plugins/terraform-provider-ignition" 
DEBUG Symlinking plugin terraform-provider-local src: "/opt/openshift/openshift-install" dst: "/tmp/openshift-install-987352622/plugins/terraform-provider-local" 
DEBUG Symlinking plugin terraform-provider-openstack src: "/opt/openshift/openshift-install" dst: "/tmp/openshift-install-987352622/plugins/terraform-provider-openstack" 
DEBUG Symlinking plugin terraform-provider-ovirt src: "/opt/openshift/openshift-install" dst: "/tmp/openshift-install-987352622/plugins/terraform-provider-ovirt" 
DEBUG Symlinking plugin terraform-provider-vsphere src: "/opt/openshift/openshift-install" dst: "/tmp/openshift-install-987352622/plugins/terraform-provider-vsphere" 
DEBUG Symlinking plugin terraform-provider-vsphereprivate src: "/opt/openshift/openshift-install" dst: "/tmp/openshift-install-987352622/plugins/terraform-provider-vsphereprivate" 
DEBUG Initializing modules...                      
DEBUG - bootstrap in ../../tmp/openshift-install-987352622/bootstrap 
DEBUG - master in ../../tmp/openshift-install-987352622/master 
DEBUG                                              
DEBUG Initializing the backend...                  
DEBUG                                              
DEBUG Initializing provider plugins...             
DEBUG                                              
DEBUG Terraform has been successfully initialized! 
DEBUG                                              
DEBUG You may now begin working with Terraform. Try running "terraform plan" to see 
DEBUG any changes that are required for your infrastructure. All Terraform commands 
DEBUG should now work.                             
DEBUG                                              
DEBUG If you ever set or change modules or backend configuration for Terraform, 
DEBUG rerun this command to reinitialize your working directory. If you forget, other 
DEBUG commands will detect it and remind you to do so if necessary. 
DEBUG data.vsphere_datacenter.datacenter: Refreshing state... 
DEBUG data.vsphere_datastore.datastore: Refreshing state... 
DEBUG data.vsphere_compute_cluster.cluster: Refreshing state... 
DEBUG data.vsphere_network.network: Refreshing state... 
DEBUG vsphere_tag_category.category: Creating...   
DEBUG vsphere_tag_category.category: Creation complete after 0s [id=urn:vmomi:InventoryServiceCategory:e3d6d9f9-6159-424d-bdbc-1942b095e9bc:GLOBAL] 
DEBUG vsphere_tag.tag: Creating...                 
DEBUG vsphere_tag.tag: Creation complete after 0s [id=urn:vmomi:InventoryServiceTag:c573b8d2-514b-494f-bfe8-a9c4c8be7f0a:GLOBAL] 
DEBUG vsphere_folder.folder[0]: Creating...        
DEBUG vsphere_folder.folder[0]: Creation complete after 0s [id=group-v135] 
DEBUG vsphereprivate_import_ova.import: Creating... 
ERROR                                              
ERROR Error: failed to lease wait: The operation is not allowed in the current state. 
ERROR                                              
ERROR   on ../../tmp/openshift-install-987352622/main.tf line 43, in resource "vsphereprivate_import_ova" "import": 
ERROR   43: resource "vsphereprivate_import_ova" "import" { 
ERROR                                              
ERROR                                              
FATAL failed to fetch Cluster: failed to generate asset "Cluster": failed to create cluster: failed to apply Terraform: failed to complete the change


What did you expect to happen?

I expect either a documentation stating it or a more debug way to understand that message from Terraform side. If we can simply not proceed with any workaround(somehow possible), then I will just open a documentation bugzilla so we can document specific scenarios where ova can fail.

Comment 1 Matthew Staebler 2020-10-30 14:54:39 UTC
This is a deficiency in the vshpereprivate terraform provider that the installer owns. The provider should be ensuring that the host that it selects for the import is not in maintenance mode.

Comment 7 jima 2021-01-29 07:33:07 UTC
Verified on 4.7.0-0.nightly-2021-01-28-042352 and passed.

According to code update, test below two scenarios:
host1: datastore1 | "VM Network", and in maintenance mode 
host2: esxi, WorkloadDatastore | "internal",....,"VM Network"

Run command "openshift-install create cluster --dir ipi --log-level"
1. choose datastore1 as default datastore, "VM Network" as network. installer exited with error "failed to find a host in the cluster that contains the provided datastore and network" as expected.
2. choose WorkloadDatastore as default datastore, "internal" as network, installer find host2 and imported ova template on that host as expected.

Comment 10 errata-xmlrpc 2021-02-24 15:29:18 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:5633


Note You need to log in before you can comment on or make changes to this bug.