Bug 1752988

Summary: e2e test does not verify all OLM resources are available
Product: OpenShift Container Platform Reporter: Clayton Coleman <ccoleman>
Component: OLMAssignee: Clayton Coleman <ccoleman>
OLM sub component: OLM QA Contact: Bruno Andrade <bandrade>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: unspecified CC: bandrade
Version: 4.2.0   
Target Milestone: ---   
Target Release: 4.2.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-10-16 06:41:28 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1752983    

Description Clayton Coleman 2019-09-17 17:47:34 UTC
This bug was initially created as a copy of Bug #1752983

I am copying this bug because: it turns out it's also broken in 4.2

The 4.1 OLM e2e test incorrectly loops over a closure variable and thus doesn't test all of the resources. We are going to be running the 4.1 e2e tests against 4.2 clusters to verify we don't regress, and this blocks us turning on the tests.

Comment 2 Bruno Andrade 2019-09-18 20:21:21 UTC
LGTM, tests passed as expected. Marking as VERIFIED

Steps used to reproduce

oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.2.0-0.nightly-2019-09-18-114152   True        False         3h33m   Cluster version is 4.2.0-0.nightly-2019-09-18-114152


 ./openshift-tests run all --dry-run | grep "OLM should" | ./openshift-tests run -f -
started: (0/1/7) "[Feature:Platform] OLM should be installed with operatorgroups at version v1 [Suite:openshift/conformance/parallel]"

started: (0/2/7) "[Feature:Platform] OLM should have imagePullPolicy:IfNotPresent on thier deployments [Suite:openshift/conformance/parallel]"

started: (0/3/7) "[Feature:Platform] OLM should be installed with installplans at version v1alpha1 [Suite:openshift/conformance/parallel]"

started: (0/4/7) "[Feature:Platform] OLM should be installed with packagemanifests at version v1 [Suite:openshift/conformance/parallel]"

started: (0/5/7) "[Feature:Platform] OLM should be installed with subscriptions at version v1alpha1 [Suite:openshift/conformance/parallel]"

started: (0/6/7) "[Feature:Platform] OLM should be installed with catalogsources at version v1alpha1 [Suite:openshift/conformance/parallel]"

started: (0/7/7) "[Feature:Platform] OLM should be installed with clusterserviceversions at version v1alpha1 [Suite:openshift/conformance/parallel]"

passed: (36.1s) 2019-09-18T20:17:22 "[Feature:Platform] OLM should be installed with operatorgroups at version v1 [Suite:openshift/conformance/parallel]"

passed: (35.9s) 2019-09-18T20:17:22 "[Feature:Platform] OLM should be installed with subscriptions at version v1alpha1 [Suite:openshift/conformance/parallel]"

passed: (36.3s) 2019-09-18T20:17:22 "[Feature:Platform] OLM should have imagePullPolicy:IfNotPresent on thier deployments [Suite:openshift/conformance/parallel]"

passed: (37.1s) 2019-09-18T20:17:23 "[Feature:Platform] OLM should be installed with catalogsources at version v1alpha1 [Suite:openshift/conformance/parallel]"

passed: (37.7s) 2019-09-18T20:17:24 "[Feature:Platform] OLM should be installed with packagemanifests at version v1 [Suite:openshift/conformance/parallel]"

passed: (38s) 2019-09-18T20:17:24 "[Feature:Platform] OLM should be installed with clusterserviceversions at version v1alpha1 [Suite:openshift/conformance/parallel]"

passed: (38.5s) 2019-09-18T20:17:24 "[Feature:Platform] OLM should be installed with installplans at version v1alpha1 [Suite:openshift/conformance/parallel]"

7 pass, 0 skip (38.7s)

Comment 3 errata-xmlrpc 2019-10-16 06:41:28 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2922