Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1878009

Summary: OCP 4.5.8 - Installing a cluster on Azure using ARM templates - "OSProvisioningTimedOut" at control plane machines step
Product: OpenShift Container Platform Reporter: Pravesh <pravesh>
Component: RHCOSAssignee: Ben Howard <behoward>
Status: CLOSED DUPLICATE QA Contact: Michael Nguyen <mnguyen>
Severity: high Docs Contact:
Priority: low    
Version: 4.5CC: adahiya, bbreard, imcleod, jligon, miabbott, nstielau, wking
Target Milestone: ---   
Target Release: 4.7.0   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-09-16 20:13:57 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
error screenshot none

Description Pravesh 2020-09-11 04:45:26 UTC 
Description of problem:

Version-Release number of the following components:

Openshift Container Platform installer 4.5.8 on Azure using ARM Templates


How reproducible:
Always

Steps to Reproduce:
1. Following steps to deploy a UPI Cluster from
https://docs.openshift.com/container-platform/4.5/installing/installing_azure/installing-azure-user-infra.html

2. At the step of "Creating the control plane machines in Azure" - VM OS Provisioning times out and fails.

3. command used:
az deployment group create -g ${RESOURCE_GROUP} --template-file "/home/ocp/05_masters.json" --parameters masterIgnition="${MASTER_IGNITION}" --parameters sshKeyData="${SSH_KEY}" --parameters privateDNSZoneName="${CLUSTER_NAME}.${BASE_DOMAIN}" --parameters baseName="${INFRA_ID}"



Actual results:

Deployment failed. Correlation ID: 548a099a-b973-40db-ad8f-9e15f60e800b. {  "status": "Failed",  "error": {    "code": "ResourceDeploymentFailure",    "message": "The resource operation completed with terminal provisioning state 'Failed'.",    "details": [      {        "code": "OSProvisioningTimedOut",        "message": "OS Provisioning for VM 'upicluster-vsmrc-master-2' did not finish in the allotted time. The VM may still finish provisioning successfully. Please check provisioning state later. Also, make sure the image has been properly prepared (generalized).\r\n * Instructions for Windows: https://azure.microsoft.com/documentation/articles/virtual-machines-windows-upload-image/ \r\n * Instructions for Linux: https://azure.microsoft.com/documentation/articles/virtual-machines-linux-capture-image/ \r\n * If you are deploying more than 20 Virtual Machines concurrently, consider moving your custom image to shared image gallery. Please refer to https://aka.ms/movetosig for the same."      }
    ]
  }
}

Expected results:
Control plane machines should be created successfully.


Additional info:
Comment 1 Pravesh 2020-09-11 04:49:26 UTC 
Created attachment 1714504 [details]
error screenshot
Comment 2 Abhinav Dahiya 2020-09-11 17:16:42 UTC 
OSProvisioningTimedOut is caused when the machine is created in azure but the OS hasn't booted up. This can be due to
- boot failure
- ignition failing to get config from bootstrap host.

Please attach the console logs for the control-plane and also the bootstrap-gather log bundle.

Moving to RHCOS team because this error in any case if best triaged by that team.
Comment 3 Micah Abbott 2020-09-11 17:44:44 UTC 
We've seen similar timeouts before in Azure; see https://bugzilla.redhat.com/show_bug.cgi?id=1779375

Without console logs from the systems failing to boot, we are unable to diagnose further.

Setting low priority and targeting 4.7
Comment 4 W. Trevor King 2020-09-11 20:13:38 UTC 
Is RHCOS planning to keep this one open?  Bug 1779375 was kicked back and forth a few times before it was closed DEFERRED.  I'd have expected this one to be closed as a dup, and the RHCOS folks to pick back up on analysis after there has been some progress on [1] or [2].

[1]: https://bugzilla.redhat.com/show_bug.cgi?id=1779375#c27
[2]: https://bugzilla.redhat.com/show_bug.cgi?id=1779375#c28
Comment 5 Micah Abbott 2020-09-14 17:23:36 UTC 
(In reply to W. Trevor King from comment #4)
> Is RHCOS planning to keep this one open?  Bug 1779375 was kicked back and
> forth a few times before it was closed DEFERRED.  I'd have expected this one
> to be closed as a dup, and the RHCOS folks to pick back up on analysis after
> there has been some progress on [1] or [2].
> 
> [1]: https://bugzilla.redhat.com/show_bug.cgi?id=1779375#c27
> [2]: https://bugzilla.redhat.com/show_bug.cgi?id=1779375#c28

I didn't want to immediately close it without a chance to get some console logs from the failing node(s).  If we are unable to get those, then it will likely be closed as a dupe of 1779375 as expected.
Comment 6 Pravesh 2020-09-14 18:58:11 UTC 
Retrying the deployment to get console logs. Thank you!
Comment 7 Pravesh 2020-09-15 19:48:34 UTC 
Tried installation on 4.5.9 with RHCOS 4.5 and provisioning was successful.
The installation was failing (with OSProvisioningTimedOut) on 4.5.8 with RHCOS 4.5. I see that the documentation now points to RHCOS image 4.5 whereas it was 4.4 before.

I think we can close this issue now.
Comment 8 Micah Abbott 2020-09-16 20:13:57 UTC 
Closing as a dupe of BZ#1779375 

If you experience this problem again, please get console logs from the failing nodes and attach them to BZ#1779375

*** This bug has been marked as a duplicate of bug 1779375 ***
Comment 9 Pravesh 2020-09-16 20:18:52 UTC 
(In reply to Micah Abbott from comment #8)
> Closing as a dupe of BZ#1779375 
> 
> If you experience this problem again, please get console logs from the
> failing nodes and attach them to BZ#1779375
> 
> *** This bug has been marked as a duplicate of bug 1779375 ***

Yes, sure.