Bug 2057582

Summary:	e2e-metal jobs are failing to install
Product:	OpenShift Container Platform	Reporter:	Stephen Benjamin <stbenjam>
Component:	Installer	Assignee:	sdasu
Installer sub component:	openshift-installer	QA Contact:	Gaoyun Pei <gpei>
Status:	CLOSED WONTFIX	Docs Contact:
Severity:	medium
Priority:	high	CC:	bgilbert, dornelas, jhixson, jligon, mrussell, nstielau, padillon, sdasu
Version:	4.10
Target Milestone:	---
Target Release:	4.11.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:	job=release-openshift-ocp-installer-e2e-metal-4.10=all job=release-openshift-ocp-installer-e2e-metal-compact-4.10=all job=release-openshift-ocp-installer-e2e-metal-serial-4.10=all job=release-openshift-ocp-installer-e2e-metal-4.11=all job=release-openshift-ocp-installer-e2e-metal-compact-4.11=all job=release-openshift-ocp-installer-e2e-metal-serial-4.11=all
Last Closed:	2022-08-05 20:15:25 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Stephen Benjamin 2022-02-23 16:34:48 UTC

These jobs are all failing to install:


release-openshift-ocp-installer-e2e-metal-4.10
release-openshift-ocp-installer-e2e-metal-compact-4.10
release-openshift-ocp-installer-e2e-metal-serial-4.10

release-openshift-ocp-installer-e2e-metal-4.11
release-openshift-ocp-installer-e2e-metal-compact-4.11
release-openshift-ocp-installer-e2e-metal-serial-4.11

See:
  https://sippy.ci.openshift.org/sippy-ng/jobs/4.11/runs?filters=%7B%22items%22%3A%5B%7B%22columnField%22%3A%22job%22%2C%22operatorValue%22%3A%22starts%20with%22%2C%22value%22%3A%22release-%22%7D%2C%7B%22columnField%22%3A%22job%22%2C%22operatorValue%22%3A%22contains%22%2C%22value%22%3A%22metal%22%7D%5D%2C%22linkOperator%22%3A%22and%22%7D

https://sippy.ci.openshift.org/sippy-ng/jobs/4.10/runs?filters={%22items%22%3A[{%22columnField%22%3A%22job%22%2C%22operatorValue%22%3A%22starts%20with%22%2C%22value%22%3A%22release-%22}%2C{%22columnField%22%3A%22job%22%2C%22operatorValue%22%3A%22contains%22%2C%22value%22%3A%22metal%22}]%2C%22linkOperator%22%3A%22and%22}

Comment 1 Patrick Dillon 2022-03-22 17:25:53 UTC

We are working on getting access to the equinix/packet environment to diagnose bootstrap issues. We're also working to convert this job to steps.

Comment 2 Stephen Benjamin 2022-05-10 12:42:15 UTC

Hi, just checking in - is there any update on these jobs? Is there anything TRT can do to help support you getting these working again?

Comment 3 Kiran Thyagaraja 2022-05-12 16:17:15 UTC

(In reply to Stephen Benjamin from comment #2)
> Hi, just checking in - is there any update on these jobs? Is there anything
> TRT can do to help support you getting these working again?

Hi Stephen, Any help is welcome. I started taking a look at this BZ after a break and the issue now seems to be with TF updated in the UPI image. Our plan was to combine the fixing this bug with moving the UPI metal jobs to steps and its taking time.

Comment 8 Gaoyun Pei 2022-06-15 14:26:46 UTC

Hi, I'm still seeing the e2e-metal-4.11 jobs failing as:

[31m│[0m [0m[1m[31mError: [0m[0m[1mfailed to create Matchbox client or connect to a3558a943132041b48b20a67aa291d99-23796056.us-east-1.elb.amazonaws.com:8081: context deadline exceeded[0m
[31m│[0m [0m
[31m│[0m [0m[0m  with provider["registry.terraform.io/poseidon/matchbox"],
[31m│[0m [0m  on main.tf line 38, in provider "matchbox":
[31m│[0m [0m  38: provider "matchbox" [4m{[0m[0m
[31m│[0m [0m
[31m╵[0m[0m
failed to create the infra resources 

such as https://storage.googleapis.com/origin-ci-test/logs/release-openshift-ocp-installer-e2e-metal-4.11/1537010341815259136/build-log.txt