Bug 2106866 - Test Flake - Using OLM descriptor components successfully creates operand using form
Summary: Test Flake - Using OLM descriptor components successfully creates operand usi...
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Management Console
Version: 4.12
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: 4.12.0
Assignee: Jakub Hadvig
QA Contact: Yadan Pei
Depends On:
TreeView+ depends on / blocked
Reported: 2022-07-13 17:51 UTC by Jon Jackson
Modified: 2023-01-17 19:53 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Last Closed: 2023-01-17 19:52:40 UTC
Target Upstream Version:

Attachments (Terms of Use)
flaky OLM tests (537.93 KB, image/png)
2022-07-19 01:22 UTC, Yadan Pei
no flags Details
Screenshot of prow.ci.openshift status on Jul 28 (652.64 KB, image/png)
2022-07-28 06:07 UTC, xiangyli
no flags Details
Screenshot of prow.ci.openshift status on Jul 28 (369.03 KB, image/png)
2022-07-28 14:26 UTC, xiangyli
no flags Details
Screen Recording of the descriptor.sepc.ts (3.47 MB, video/mp4)
2022-07-29 08:04 UTC, xiangyli
no flags Details

System ID Private Priority Status Summary Last Updated
Github openshift console pull 11832 0 None open Bug 2106866: Fix flaky OLM test 2022-07-15 11:39:18 UTC
Github openshift console pull 11856 0 None open Bug 2106866: Fix flaky OLM descriptor test 2022-07-20 15:48:24 UTC
Red Hat Product Errata RHSA-2022:7399 0 None None None 2023-01-17 19:53:03 UTC

Description Jon Jackson 2022-07-13 17:51:01 UTC
Description of problem:
OLM descriptor e2e test case is flaking

Version-Release number of selected component (if applicable):

How reproducible:
About 25% of pull-ci-openshift-console-master-e2e-gcp-console jobs

Steps to Reproduce:
CI Search: https://search.ci.openshift.org/?search=Using+OLM+descriptor+components+successfully+creates+operand+using+form&maxAge=24h&context=1&type=junit&name=pull-ci-openshift-console-master-e2e-gcp-console&excludeName=&maxMatches=5&maxBytes=&groupBy=job

Additional info:

# Using OLM descriptor components.Using OLM descriptor components successfully creates operand using form
AssertionError: Timed out retrying after 30000ms: Expected to find element: `[data-test="create-dynamic-form"]`, but never found it.

After investigation, I found that the above failure message is actually from the 2nd retry. The first failure and root of the problem occurs earlier in the test case where cypress tries to click on an operand name, but the element has sometimes been detached from the DOM.

Comment 7 xiangyli 2022-07-28 06:07:07 UTC
Created attachment 1899854 [details]
Screenshot of prow.ci.openshift status on Jul 28

Comment 8 xiangyli 2022-07-28 06:21:46 UTC
Since last comment on Jul 19, in the past 7 days from Jul 21 to Jul 28, there are still 10 runs, 80% failed, 88% of failures match = 70% impact


Comment 9 Jon Jackson 2022-07-28 13:22:01 UTC
I'm only seeing one failure in the linked search results, and that's on the `periodic-ci-openshift-release-master-nightly-4.11-console-aws` job. That job runs on the 4.11 branch, which I don't think has this fix.

Comment 10 xiangyli 2022-07-28 14:26:48 UTC
Created attachment 1899977 [details]
Screenshot of prow.ci.openshift status on Jul 28

Sorry if I was not clear enough about the search result for the logs for this bug. But in the `periodic-ci-openshift-release-master-nightly-4.12-console-aws` job there are still 21 runs, 67% failed, 7% of failures match = 5% impact in the last 7 days, which I think is not verifiable. Sorry again about the unclearness caused. 


Comment 12 Jon Jackson 2022-07-28 15:16:56 UTC
Just investigated the one failure listed in the search results above, and it appears to be a different flake. The "Using OLM descriptor components deletes operand" case is failing, which causes the subsequent creation case to fail as well. See attached log snippet.

Comment 17 errata-xmlrpc 2023-01-17 19:52:40 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.12.0 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.