Bug 2019438

Summary: operator.Run template e2e-metal - e2e-metal container setup
Product: OpenShift Container Platform Reporter: Devan Goodwin <dgoodwin>
Component: InstallerAssignee: Beth White <beth.white>
Installer sub component: openshift-installer QA Contact: Amit Ugol <augol>
Status: CLOSED DUPLICATE Docs Contact:
Severity: medium    
Priority: unspecified CC: bfournie, mstaeble, padillon, sippy
Version: 4.10   
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
job=release-openshift-ocp-installer-e2e-metal-compact-4.10
Last Closed: 2021-11-30 03:56:27 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Devan Goodwin 2021-11-02 14:38:58 UTC
operator.Run template e2e-metal - e2e-metal container setup

is failing frequently in CI for some configurations. I am specifically chasing metal compact: https://sippy.ci.openshift.org/sippy-ng/jobs/4.10/analysis?filters=%7B%22items%22%3A%5B%7B%22columnField%22%3A%22name%22%2C%22operatorValue%22%3A%22equals%22%2C%22value%22%3A%22release-openshift-ocp-installer-e2e-metal-compact-4.10%22%7D%5D%7D

The error most recently has been: 

level=info msg=Ignition-Configs created in: /tmp/shared/installer and /tmp/shared/installer/auth
jq: error: Could not open file /var/lib/openshift-install/rhcos.json: No such file or directory

However this message is also showing up across a smattering of UPI jobs:

https://search.ci.openshift.org/?search=Could+not+open+file+%2Fvar%2Flib%2Fopenshift-install%2Frhcos.jso&maxAge=48h&context=1&type=bug%2Bjunit&name=&excludeName=&maxMatches=5&maxBytes=20971520&groupBy=job

But for release-openshift-ocp-installer-e2e-metal-compact-4.10 specifically it is showing 80% impact for the past 2 days of failures. (of a 100% fail rate)

Specific prow job example:

https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/release-openshift-ocp-installer-e2e-metal-compact-4.10/1455259382496366592

Comment 1 Patrick Dillon 2021-11-02 15:22:29 UTC
It looks like this was broken by https://github.com/openshift/installer/pull/5252 so we will need to come up with a backwards-compatible fix.

I just added https://issues.redhat.com/browse/CORS-1822 to the installer backlog to fix problems of this nature.

Comment 2 Bob Fournier 2021-11-02 17:06:38 UTC
This looks like a UPI bug, not IPI. Changing subcomponent to openshiftinstaller.

Comment 3 Matthew Staebler 2021-11-30 03:56:27 UTC

*** This bug has been marked as a duplicate of bug 2024190 ***