Description of problem: See https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_release/11529/rehearse-11529-pull-ci-operator-framework-operator-marketplace-master-okd-e2e-aws-operator/1308505582092685312 ``` --- FAIL: TestMarketplace/no-setup-test-group/operatorhub-test-suite/disable-all-enable-one (5.29s) operatorhubtests.go:97: Error Trace: operatorhubtests.go:97 Error: Received unexpected error: default CatalogSource still present on the cluster Test: TestMarketplace/no-setup-test-group/operatorhub-test-suite/disable-all-enable-one Messages: Default CatalogSource has not been disabled ``` This is probably happening due to a different default config used in OKD - https://github.com/openshift/installer/blob/fcos/data/data/manifests/openshift/community-operators.yaml
@Vadim, the test is supposed to override whatever config the cluster has before the test start: https://github.com/operator-framework/operator-marketplace/blob/2d6d7189243754f3ab8da86b3e7aa39c6fcdbbab/test/testsuites/operatorhubtests.go#L82 As long as all the defaults are present on the cluster, even if some are disabled, this test shouldn't be affected. How often is this happening?
(In reply to Anik from comment #1) > How often is this happening? I've seen it once so far, as OKD tests are not being run regularly until https://github.com/operator-framework/operator-marketplace/pull/332 is merged.
Wrong PR number - https://github.com/openshift/release/pull/11529 would add optional okd test to marketplace PRs
This test is likely to flip flop between passing and failing because of https://github.com/operator-framework/operator-marketplace/blob/2d6d7189243754f3ab8da86b3e7aa39c6fcdbbab/test/testsuites/operatorhubtests.go#L93 https://github.com/operator-framework/operator-marketplace/blob/2d6d7189243754f3ab8da86b3e7aa39c6fcdbbab/test/testsuites/operatorhubtests.go#L449 this does not poll for the correct status of the cluster. https://github.com/operator-framework/operator-marketplace/blob/2d6d7189243754f3ab8da86b3e7aa39c6fcdbbab/test/testsuites/operatorhubtests.go#L407 should be called instead to ensure higher success rate. Setting the priority for this as low for now.
Looks like this might be blocking a 4.6 blocker bug https://bugzilla.redhat.com/show_bug.cgi?id=1874751. Changing priority and target release so that the fix for this can unblock 1874751
It works well, LGTM, verify it.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:4196