Bug 2106866

Summary: Test Flake - Using OLM descriptor components successfully creates operand using form
Product: OpenShift Container Platform Reporter: Jon Jackson <jonjacks>
Component: Management ConsoleAssignee: Jakub Hadvig <jhadvig>
Status: CLOSED ERRATA QA Contact: Yadan Pei <yapei>
Severity: low Docs Contact:
Priority: unspecified    
Version: 4.12CC: rhamilto, yapei
Target Milestone: ---   
Target Release: 4.12.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-01-17 19:52:40 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
flaky OLM tests
none
Screenshot of prow.ci.openshift status on Jul 28
none
Screenshot of prow.ci.openshift status on Jul 28
none
Screen Recording of the descriptor.sepc.ts none

Description Jon Jackson 2022-07-13 17:51:01 UTC
Description of problem:
OLM descriptor e2e test case is flaking

Version-Release number of selected component (if applicable):
4.12

How reproducible:
About 25% of pull-ci-openshift-console-master-e2e-gcp-console jobs

Steps to Reproduce:
CI Search: https://search.ci.openshift.org/?search=Using+OLM+descriptor+components+successfully+creates+operand+using+form&maxAge=24h&context=1&type=junit&name=pull-ci-openshift-console-master-e2e-gcp-console&excludeName=&maxMatches=5&maxBytes=&groupBy=job

Additional info:

# Using OLM descriptor components.Using OLM descriptor components successfully creates operand using form
AssertionError: Timed out retrying after 30000ms: Expected to find element: `[data-test="create-dynamic-form"]`, but never found it.

After investigation, I found that the above failure message is actually from the 2nd retry. The first failure and root of the problem occurs earlier in the test case where cypress tries to click on an operand name, but the element has sometimes been detached from the DOM.

Comment 7 xiangyli 2022-07-28 06:07:07 UTC
Created attachment 1899854 [details]
Screenshot of prow.ci.openshift status on Jul 28

Comment 8 xiangyli 2022-07-28 06:21:46 UTC
Since last comment on Jul 19, in the past 7 days from Jul 21 to Jul 28, there are still 10 runs, 80% failed, 88% of failures match = 70% impact

https://search.ci.openshift.org/?search=Using+OLM+descriptor+components+successfully+creates+operand+using+form&maxAge=168h&context=1&type=junit&name=&excludeName=&maxMatches=5&maxBytes=20971520&groupBy=job

Comment 9 Jon Jackson 2022-07-28 13:22:01 UTC
I'm only seeing one failure in the linked search results, and that's on the `periodic-ci-openshift-release-master-nightly-4.11-console-aws` job. That job runs on the 4.11 branch, which I don't think has this fix.

Comment 10 xiangyli 2022-07-28 14:26:48 UTC
Created attachment 1899977 [details]
Screenshot of prow.ci.openshift status on Jul 28

Sorry if I was not clear enough about the search result for the logs for this bug. But in the `periodic-ci-openshift-release-master-nightly-4.12-console-aws` job there are still 21 runs, 67% failed, 7% of failures match = 5% impact in the last 7 days, which I think is not verifiable. Sorry again about the unclearness caused. 

https://search.ci.openshift.org/?search=Using+OLM+descriptor+components+successfully+creates+operand+using+form&maxAge=168h&context=1&type=junit&name=periodic-ci-openshift-release-master-nightly-4.12-console-aws&excludeName=&maxMatches=5&maxBytes=20971520&groupBy=job

Comment 12 Jon Jackson 2022-07-28 15:16:56 UTC
Just investigated the one failure listed in the search results above, and it appears to be a different flake. The "Using OLM descriptor components deletes operand" case is failing, which causes the subsequent creation case to fail as well. See attached log snippet.

Comment 17 errata-xmlrpc 2023-01-17 19:52:40 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.12.0 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:7399