Bug 1918068 - Flaky olm integration tests
Summary: Flaky olm integration tests
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Management Console
Version: 4.7
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: 4.7.0
Assignee: Jon Jackson
QA Contact: Yadan Pei
Depends On:
TreeView+ depends on / blocked
Reported: 2021-01-20 00:10 UTC by Jon Jackson
Modified: 2021-02-24 15:54 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: CatalogSource detail view integration tests was looking for the filter toolbar on the CatalogSource details page Operators tab right after the CatalogSource was created. It takes a few seconds for the PackageManifests that drive that page to be created on the cluster, and PackageManifests don't support k8s watches. Consequence: If no PackageManifest has been created yet when you initially visit the CatlogSource detail Operators tab, the filter toolbar will not be present and it will never appear without reloading the page. This causes the test to time out and fail sometimes. Fix: Make our k8s watch helper fall back to polling every 15 seconds if watching is not supported. This allows the PackageManifest page to update as the resources are created on the cluster without a page reload. Update the integration test to remove conditional logic and retries that were there because the page was static. Result: This test case passes reliably.
Clone Of:
Last Closed: 2021-02-24 15:54:42 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Github openshift console pull 7874 0 None closed Bug 1918068: Fix flaky OLM Cypress tests 2021-02-08 16:18:57 UTC
Red Hat Product Errata RHSA-2020:5633 0 None None None 2021-02-24 15:54:59 UTC

Description Jon Jackson 2021-01-20 00:10:57 UTC
Description of problem:
OLM integration tests related to operator installation workflow are flaking frequently.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. Run `yarn test-cypress-olm` in frontend working dir

Actual results:
OLM operator install integration test suites fail frequently

Expected results:
OLM operator install integration test suites always pass

Comment 3 Yadan Pei 2021-01-26 09:04:33 UTC
Search co-clusterserviceversion-logo__name__clusterserviceversion in CI logs within 7 days, we can see only one failure which is mentioned in the PR comment, no more related failures


Moving to VERIFIED

Comment 6 errata-xmlrpc 2021-02-24 15:54:42 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.