Bug 1756484

Created attachment 1620234 [details]
Contents of "oc adm must-gather" and ".openshift_install.log"

Description of problem:
Openshift installation on azure fails as follows:

[root@dhcp-8-33-74 ~]# openshift-install create cluster --dir ~/azure-badr
? SSH Public Key /root/.ssh/cluster--key.pub
? Platform azure
? azure subscription id ***
? azure tenant id ***
? azure service principal client id ***
? azure service principal client secret [? for help] ************************************
INFO Saving user credentials to "/root/.azure/osServicePrincipal.json" 
? Region centralus
? Base Domain css-qe.com
? Cluster Name csi-ocp42
? Pull Secret [? for help] ***********************************************************************************************************INFO Creating infrastructure resources...         *********************************************
INFO Waiting up to 30m0s for the Kubernetes API at https://api.csi-ocp42.css-qe.com:6443... 
INFO API v1.14.0+1682e38 up                       
INFO Waiting up to 30m0s for bootstrapping to complete... 
INFO Destroying the bootstrap resources...        
ERROR                                              
ERROR Error: Error waiting for removal of Backend Address Pool Association for NIC "csi-ocp42-5frp6-bootstrap-nic" (Resource Group "csi-ocp42-5frp6-rg"): Code="OperationNotAllowed" Message="Operation 'startTenantUpdate' is not allowed on VM 'csi-ocp42-5frp6-bootstrap' since the VM is marked for deletion. You can only retry the Delete operation (or wait for an ongoing one to complete)." Details=[] 
ERROR                                              
ERROR                                              
FATAL Terraform destroy: failed to destroy using Terraform 

The cluster however eventually becomes available and seems to have installed successfully:

[root@dhcp-8-33-74 auth]# oc get csr
NAME        AGE   REQUESTOR                                                                   CONDITION
csr-5sj2r   38m   system:node:csi-ocp42-5frp6-worker-centralus1-58q74                         Approved,Issued
csr-69879   46m   system:node:csi-ocp42-5frp6-master-1                                        Approved,Issued
csr-dcrfq   39m   system:serviceaccount:openshift-machine-config-operator:node-bootstrapper   Approved,Issued
csr-fn2qd   46m   system:node:csi-ocp42-5frp6-master-2                                        Approved,Issued
csr-fx5qd   39m   system:serviceaccount:openshift-machine-config-operator:node-bootstrapper   Approved,Issued
csr-ggm7d   46m   system:node:csi-ocp42-5frp6-master-0                                        Approved,Issued
csr-h5hqz   39m   system:node:csi-ocp42-5frp6-worker-centralus2-f5v45                         Approved,Issued
csr-h95fp   47m   system:serviceaccount:openshift-machine-config-operator:node-bootstrapper   Approved,Issued
csr-j2jcv   47m   system:serviceaccount:openshift-machine-config-operator:node-bootstrapper   Approved,Issued
csr-qwzjn   39m   system:node:csi-ocp42-5frp6-worker-centralus3-c864h                         Approved,Issued
csr-rlv25   47m   system:serviceaccount:openshift-machine-config-operator:node-bootstrapper   Approved,Issued
csr-rlx5p   39m   system:serviceaccount:openshift-machine-config-operator:node-bootstrapper   Approved,Issued


[root@dhcp-8-33-74 azure-badr]# openshift-install --dir=/root/azure-badr/ wait-for install-complete
INFO Waiting up to 30m0s for the cluster at https://api.csi-ocp42.css-qe.com:6443 to initialize... 
INFO Waiting up to 10m0s for the openshift-console route to be created... 
INFO Install complete!                            
INFO To access the cluster as the system:admin user when using 'oc', run 'export KUBECONFIG=/root/azure-badr/auth/kubeconfig' 
INFO Access the OpenShift web-console here: https://console-openshift-console.apps.csi-ocp42.css-qe.com 
INFO Login to the console with user: kubeadmin, password: *** 


Version-Release number of the following components:

[root@dhcp-8-33-74 azure-badr]# oc version
Client Version: version.Info{Major:"", Minor:"", GitVersion:"v0.0.0-alpha.0-34-g0d02ccfc", GitCommit:"0d02ccfccbfbabe8be161db4dbb1f42bf00cf7c2", GitTreeState:"clean", BuildDate:"2019-07-28T20:08:21Z", GoVersion:"go1.12.6", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"14+", GitVersion:"v1.14.0+1682e38", GitCommit:"1682e38", GitTreeState:"clean", BuildDate:"2019-07-28T20:08:40Z", GoVersion:"go1.12.6", Compiler:"gc", Platform:"linux/amd64"}
OpenShift Version: 4.2.0-0.nightly-2019-07-30-073644
[root@dhcp-8-33-74 azure-badr]# openshift-install version
openshift-install v4.2.0-201907300219-dirty
built from commit 9323138c52b773e31b7c98d8a934f9ddaa6ccaee
release image quay.io/openshift-release-dev/ocp-release-nightly@sha256:63a230163f19a17d7ebb5bc877fbb4fe74713678292c12b4d8407bbc5397b378

How reproducible:
I have reproduced this within two consecutive attempts.

Steps to Reproduce:
1.Run openshift-install create cluster commands
2.
3.

Actual results:
Please include the entire output from the last TASK line through the end of output if an error is generated

INFO Destroying the bootstrap resources...        
ERROR                                              
ERROR Error: Error waiting for removal of Backend Address Pool Association for NIC "csi-ocp42-5frp6-bootstrap-nic" (Resource Group "csi-ocp42-5frp6-rg"): Code="OperationNotAllowed" Message="Operation 'startTenantUpdate' is not allowed on VM 'csi-ocp42-5frp6-bootstrap' since the VM is marked for deletion. You can only retry the Delete operation (or wait for an ongoing one to complete)." Details=[] 
ERROR                                              
ERROR                                              
FATAL Terraform destroy: failed to destroy using Terraform 

Expected results:
Successful installation

Additional info:

Please see attachment