Bug 1843314

Summary:	Increase the timeout waiting for ironic node deployment
Product:	OpenShift Container Platform	Reporter:	Alexander Chuzhoy <sasha>
Component:	Installer	Assignee:	Stephen Benjamin <stbenjam>
Installer sub component:	OpenShift on Bare Metal IPI	QA Contact:	Amit Ugol <augol>
Status:	CLOSED ERRATA	Docs Contact:
Severity:	medium
Priority:	medium	CC:	dhellmann, stbenjam
Version:	4.5	Keywords:	Triaged
Target Milestone:	---
Target Release:	4.6.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-10-27 16:04:37 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1846187

Description Alexander Chuzhoy 2020-06-03 02:19:27 UTC

Cluster version: 4.4.4

In our testbeds we occasionally see timeouts occuring during the phase when ironic is being brought up.

time="2020-06-02T20:56:12Z" level=debug msg="module.masters.ironic_node_v1.openshift-master-host[2]: Still creating... [34m0s elapsed]"
time="2020-06-02T20:56:12Z" level=debug msg="module.masters.ironic_node_v1.openshift-master-host[0]: Still creating... [34m0s elapsed]"
time="2020-06-02T20:56:17Z" level=error
time="2020-06-02T20:56:17Z" level=error msg="Error: could not contact API: timeout reached"

When I check the bootstrap VM - I see that it works fine, just took it a bit longer than 30 minutes to boot with the ignition config and to get the rhcos image.

Would it be possible to increase the timeout to say 1 hour.

Thanks.

Comment 1 Alexander Chuzhoy 2020-06-04 20:52:50 UTC

Just did some tests today on somewhat slow testbeds.
Successfully deployed 4.4.5 today.

As you can see from below it took a bit less than 27 mins:
time="2020-06-04T07:14:52Z" level=debug msg="module.masters.ironic_node_v1.openshift-master-host[0]: Creation complete after 26m49s [id=4d8b98b2-ff8e-400d-bf07-ef8778e6183b]"
time="2020-06-04T07:14:53Z" level=debug msg="module.masters.ironic_node_v1.openshift-master-host[2]: Creation complete after 26m50s [id=85acbf97-106e-4bee-83fa-4b855d69069c]"
time="2020-06-04T07:14:53Z" level=debug msg="module.masters.ironic_node_v1.openshift-master-host[1]: Creation complete after 26m50s [id=7b522fe0-91a8-4234-898f-3a837ea5ccd9]"


Then tried to install 4.5.0-0.nightly-2020-06-04-025914 it was timing out around 25 mins.

Comment 4 Alexander Chuzhoy 2020-06-17 03:48:04 UTC

Version:4.5.0-0.nightly-2020-06-16-014907


time="2020-06-17T00:40:08Z" level=debug msg="module.masters.ironic_node_v1.openshift-master-host[2]: Still creating... [25m10s elapsed]"
time="2020-06-17T00:40:08Z" level=debug msg="module.masters.ironic_node_v1.openshift-master-host[0]: Still creating... [25m10s elapsed]"
time="2020-06-17T00:40:18Z" level=debug msg="module.masters.ironic_node_v1.openshift-master-host[0]: Still creating... [25m20s elapsed]"
time="2020-06-17T00:40:18Z" level=debug msg="module.masters.ironic_node_v1.openshift-master-host[1]: Still creating... [25m20s elapsed]"
time="2020-06-17T00:40:18Z" level=debug msg="module.masters.ironic_node_v1.openshift-master-host[2]: Still creating... [25m20s elapsed]"
time="2020-06-17T00:40:28Z" level=debug msg="module.masters.ironic_node_v1.openshift-master-host[0]: Still creating... [25m30s elapsed]"
time="2020-06-17T00:40:28Z" level=debug msg="module.masters.ironic_node_v1.openshift-master-host[1]: Still creating... [25m30s elapsed]"
time="2020-06-17T00:40:28Z" level=debug msg="module.masters.ironic_node_v1.openshift-master-host[2]: Still creating... [25m30s elapsed]"
time="2020-06-17T00:40:34Z" level=debug msg="module.masters.ironic_node_v1.openshift-master-host[0]: Creation complete after 25m36s [id=3d0a33bb-f652-45cd-a7e3-2f3b6a5f3586]"
time="2020-06-17T00:40:35Z" level=debug msg="module.masters.ironic_node_v1.openshift-master-host[2]: Creation complete after 25m37s [id=6a8403e0-fd4b-4053-86c1-b2a2694215d2]"
time="2020-06-17T00:40:37Z" level=debug msg="module.masters.ironic_node_v1.openshift-master-host[1]: Creation complete after 25m38s [id=31c107c5-0df7-4ecc-94ee-92e6227ff4c

Thus it didn't time out after 25 mins.

Comment 6 errata-xmlrpc 2020-10-27 16:04:37 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4196