Hide Forgot
Description of problem: OLM descriptor e2e test case is flaking Version-Release number of selected component (if applicable): 4.12 How reproducible: About 25% of pull-ci-openshift-console-master-e2e-gcp-console jobs Steps to Reproduce: CI Search: https://search.ci.openshift.org/?search=Using+OLM+descriptor+components+successfully+creates+operand+using+form&maxAge=24h&context=1&type=junit&name=pull-ci-openshift-console-master-e2e-gcp-console&excludeName=&maxMatches=5&maxBytes=&groupBy=job Additional info: # Using OLM descriptor components.Using OLM descriptor components successfully creates operand using form AssertionError: Timed out retrying after 30000ms: Expected to find element: `[data-test="create-dynamic-form"]`, but never found it. After investigation, I found that the above failure message is actually from the 2nd retry. The first failure and root of the problem occurs earlier in the test case where cypress tries to click on an operand name, but the element has sometimes been detached from the DOM.
Created attachment 1898028 [details] flaky OLM tests PR was merged three days ago but we still see it failed in 26 runs, 69% failed, 44% of failures match = 31% impact https://search.ci.openshift.org/?search=Using+OLM+descriptor+components+successfully+creates+operand+using+form&maxAge=24h&context=1&type=junit&name=pull-ci-openshift-console-master-e2e-gcp-console&excludeName=&maxMatches=5&maxBytes=&groupBy=job
Created attachment 1899854 [details] Screenshot of prow.ci.openshift status on Jul 28
Since last comment on Jul 19, in the past 7 days from Jul 21 to Jul 28, there are still 10 runs, 80% failed, 88% of failures match = 70% impact https://search.ci.openshift.org/?search=Using+OLM+descriptor+components+successfully+creates+operand+using+form&maxAge=168h&context=1&type=junit&name=&excludeName=&maxMatches=5&maxBytes=20971520&groupBy=job
I'm only seeing one failure in the linked search results, and that's on the `periodic-ci-openshift-release-master-nightly-4.11-console-aws` job. That job runs on the 4.11 branch, which I don't think has this fix.
Created attachment 1899977 [details] Screenshot of prow.ci.openshift status on Jul 28 Sorry if I was not clear enough about the search result for the logs for this bug. But in the `periodic-ci-openshift-release-master-nightly-4.12-console-aws` job there are still 21 runs, 67% failed, 7% of failures match = 5% impact in the last 7 days, which I think is not verifiable. Sorry again about the unclearness caused. https://search.ci.openshift.org/?search=Using+OLM+descriptor+components+successfully+creates+operand+using+form&maxAge=168h&context=1&type=junit&name=periodic-ci-openshift-release-master-nightly-4.12-console-aws&excludeName=&maxMatches=5&maxBytes=20971520&groupBy=job
Just investigated the one failure listed in the search results above, and it appears to be a different flake. The "Using OLM descriptor components deletes operand" case is failing, which causes the subsequent creation case to fail as well. See attached log snippet.
Created attachment 1900110 [details] Screen Recording of the descriptor.sepc.ts Since there is only one failure: https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-nightly-4.12-console-aws/1552114282458517504 https://search.ci.openshift.org/?search=Using+OLM+descriptor+components+successfully+creates+operand+using+form&maxAge=168h&context=1&type=junit&name=periodic-ci-openshift-release-master-nightly-4.12-console-aws&excludeName=&maxMatches=5&maxBytes=20971520&groupBy=job And from the recording (1:45), it is caused by `data-test-action="Delete App"` failure, which is a different issue. Thus the bug can be verified.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.12.0 bug fix and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:7399