Bug 1752983

Summary: e2e test does not verify all OLM resources are available
Product: OpenShift Container Platform Reporter: Clayton Coleman <ccoleman>
Component: OLMAssignee: Clayton Coleman <ccoleman>
OLM sub component: OLM QA Contact: Bruno Andrade <bandrade>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: unspecified CC: alegrand, anpicker, erooth, ffranz, jiazha, juzhao, lcosic, mloibl, pkrupa, surbania
Version: 4.1.z   
Target Milestone: ---   
Target Release: 4.1.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1740372 Environment:
Last Closed: 2019-09-27 00:33:52 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1752988    
Bug Blocks:    

Description Clayton Coleman 2019-09-17 17:40:27 UTC
The 4.1 OLM e2e test incorrectly loops over a closure variable and thus doesn't test all of the resources. We are going to be running the 4.1 e2e tests against 4.2 clusters to verify we don't regress, and this blocks us turning on the tests.

This is only for 4.1.z - the test in 4.2 was refactored to not have this problem, but was not back ported.  Must be in 4.1.z.

Comment 2 Bruno Andrade 2019-09-23 15:00:01 UTC
LGTM, tests passed as expected. Marking as VERIFIED

Steps used to reproduce:

git status
# On branch release-4.1

oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.1.0-0.nightly-2019-09-21-195243   True        False         144m    Cluster version is 4.1.0-0.nightly-2019-09-21-195243


 ./openshift-tests run all --dry-run | grep "OLM should" | ./openshift-tests run -f -
started: (0/1/6) "[Feature:Platform] OLM should be installed with catalogsources at version v1alpha1 [Suite:openshift/conformance/parallel]"

started: (0/2/6) "[Feature:Platform] OLM should be installed with installplans at version v1alpha1 [Suite:openshift/conformance/parallel]"

started: (0/3/6) "[Feature:Platform] OLM should be installed with clusterserviceversions at version v1alpha1 [Suite:openshift/conformance/parallel]"

started: (0/4/6) "[Feature:Platform] OLM should be installed with operatorgroups at version v1 [Suite:openshift/conformance/parallel]"

started: (0/5/6) "[Feature:Platform] OLM should be installed with subscriptions at version v1alpha1 [Suite:openshift/conformance/parallel]"

started: (0/6/6) "[Feature:Platform] OLM should be installed with packagemanifests at version v1 [Suite:openshift/conformance/parallel]"

passed: (21s) 2019-09-23T14:53:57 "[Feature:Platform] OLM should be installed with operatorgroups at version v1 [Suite:openshift/conformance/parallel]"

passed: (20.9s) 2019-09-23T14:53:57 "[Feature:Platform] OLM should be installed with clusterserviceversions at version v1alpha1 [Suite:openshift/conformance/parallel]"

passed: (20.9s) 2019-09-23T14:53:57 "[Feature:Platform] OLM should be installed with subscriptions at version v1alpha1 [Suite:openshift/conformance/parallel]"

passed: (20.9s) 2019-09-23T14:53:57 "[Feature:Platform] OLM should be installed with packagemanifests at version v1 [Suite:openshift/conformance/parallel]"

passed: (21.2s) 2019-09-23T14:53:57 "[Feature:Platform] OLM should be installed with catalogsources at version v1alpha1 [Suite:openshift/conformance/parallel]"

passed: (20.9s) 2019-09-23T14:53:57 "[Feature:Platform] OLM should be installed with installplans at version v1alpha1 [Suite:openshift/conformance/parallel]"

6 pass, 0 skip (21.3s)

Comment 4 errata-xmlrpc 2019-09-27 00:33:52 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2856