Bug 1648215

Summary: The CSV cannot be created successfully via subscription
Product: OpenShift Container Platform Reporter: Jian Zhang <jiazha>
Component: OLMAssignee: Evan Cordell <ecordell>
Status: CLOSED ERRATA QA Contact: Jian Zhang <jiazha>
Severity: high Docs Contact:
Priority: high    
Version: 4.1.0CC: chezhang, dyan, ecordell, jfan, jpeeler, zitang
Target Milestone: ---   
Target Release: 4.1.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-06-04 10:40:51 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jian Zhang 2018-11-09 07:31:49 UTC
Description of problem:
After creating a subscription, the PHASE of the csv is empty.
[jzhang@dhcp-140-18 installer]$ oc get csv -n jian
NAME                  DISPLAY   VERSION   REPLACES              PHASE
etcdoperator.v0.9.2   etcd      0.9.2     etcdoperator.v0.9.0 

Check the olm-operator logs, got below errors:
E1109 06:02:06.241510       1 reflector.go:205] github.com/operator-framework/operator-lifecycle-manager/pkg/lib/queueinformer/queueinformer_operator.go:108: Failed to list *v1alpha2.OperatorGroup: the server could not find the requested resource (get operatorgroups.operators.coreos.com)

Check the catalog-operator logs, got below:
time="2018-11-09T05:58:51Z" level=info msg="jian/install-etcdoperator.v0.9.2-99hnn updated"
time="2018-11-09T05:58:51Z" level=info msg="error fetching CSV etcdoperator.v0.9.2 via k8s api: clusterserviceversions.operators.coreos.com \"etcdoperator.v0.9.2\" not found"

I confirm that the "etcdoperator.v0.9.2" csv exists in the ConfigMap.
  packages: "
    \ \n- #! package-manifest: ./deploy/chart/catalog_resources/rh-operators/etcdoperator.v0.9.2.clusterserviceversion.yaml\n
    \ packageName: etcd\n  channels:\n  - name: alpha\n    currentCSV: etcdoperator.v0.9.2\n


Version-Release number of selected component (if applicable):
      image: registry.svc.ci.openshift.org/openshift/origin-v4.0-20181109054332@sha256:bd42b78b320f01a3c5b9181d5581b77a9103c4bcf1e9c4d7bbfe1c1b098b6ba8

[jzhang@dhcp-140-18 installer]$ oc exec olm-operator-67f6b68849-tglt5 -- olm -version
OLM version: 0.7.4
git commit: c82dca5

How reproducible:
always

Steps to Reproduce:
1. Install the OCP 4.0 via openshift-installer by following the doc: https://github.com/openshift/installer/blob/master/docs/dev/libvirt-howto.md

2. Create a subscription, like below:
[jzhang@dhcp-140-18 installer]$ cat subscription.yaml 
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  generateName: etcd-
  namespace: jian
spec:
  source: rh-operators
  name: etcd
  startingCSV: etcdoperator.v0.9.2
  channel: alpha

[jzhang@dhcp-140-18 installer]$ oc create -f subscription.yaml

3. Check the Etcd pod in project "jian".

[jzhang@dhcp-140-18 installer]$ oc get subscription -n jian
NAME         PACKAGE   SOURCE         CHANNEL
etcd-7r662   etcd      rh-operators   alpha
[jzhang@dhcp-140-18 installer]$ oc get installplan -n jian
NAME                                CSV                   SOURCE         APPROVAL    APPROVED
install-etcdoperator.v0.9.2-99hnn   etcdoperator.v0.9.2   rh-operators   Automatic   false
[jzhang@dhcp-140-18 installer]$ oc get csv -n jian
NAME                  DISPLAY   VERSION   REPLACES              PHASE
etcdoperator.v0.9.2   etcd      0.9.2     etcdoperator.v0.9.0   
[jzhang@dhcp-140-18 installer]$ oc get pods -n jian
No resources found.

Actual results:
The status of the csv was incorrect, and no pod generated.
[jzhang@dhcp-140-18 installer]$ oc get csv -n jian
NAME                  DISPLAY   VERSION   REPLACES              PHASE
etcdoperator.v0.9.2   etcd      0.9.2     etcdoperator.v0.9.0   
[jzhang@dhcp-140-18 installer]$ oc get pods -n jian
No resources found.

Expected results:
Subscription successfully. No errors and the Etcd pod can be generated.

Additional info:

Comment 1 Zihan Tang 2018-11-19 08:16:22 UTC
When I test TSB and ASB operator, I also hit this issue, subscription can not create successfully.
olm-operator log: 
E1119 08:08:26.899521       1 reflector.go:205] github.com/operator-framework/operator-lifecycle-manager/pkg/lib/queueinformer/queueinformer_operator.go:108: Failed to list *v1alpha2.OperatorGroup: the server could not find the requested resource (get operatorgroups.operators.coreos.com)

OLM version: 0.8.0
git commit: ff2325e

Comment 2 Jian Zhang 2018-11-21 10:03:42 UTC
I used the latest version for having a try. It works well. So, remove the TestBlocker label.

[jzhang@dhcp-140-18 installer]$ oc exec olm-operator-585c885c55-2zcr9 -- olm -version
OLM version: 0.8.0
git commit: 08ea39b

Actually, the version is "git commit: 08ea39b", I highly suggest that we fix bug 1626434 first so that the users can know the version clearly. @Evan What do you think? For now, you can move this bug to "ON_QA" status so that I can verify it. It's better if you can post the fixed PR in here.

[jzhang@dhcp-140-18 installer]$ cat subscription.yaml 
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  generateName: etcd-
spec:
  source: rh-operators
  name: etcd
  startingCSV: etcdoperator.v0.9.2
  channel: alpha

[jzhang@dhcp-140-18 installer]$ oc get pods
NAME                                READY     STATUS    RESTARTS   AGE
catalog-operator-75c8fcc485-md5xn   1/1       Running   0          6h
etcd-operator-7b49974f5b-gdd5m      3/3       Running   0          8m
olm-operator-585c885c55-2zcr9       1/1       Running   0          6h
package-server-7d685ccff7-6sb9j     1/1       Running   0          6h

[jzhang@dhcp-140-18 installer]$ oc get csv
NAME                  DISPLAY   VERSION   REPLACES              PHASE
etcdoperator.v0.9.2   etcd      0.9.2     etcdoperator.v0.9.0   Succeeded
[jzhang@dhcp-140-18 installer]$ oc get installplan
NAME                                CSV                   SOURCE         APPROVAL    APPROVED
install-etcdoperator.v0.9.2-sx4kp   etcdoperator.v0.9.2   rh-operators   Automatic   false
[jzhang@dhcp-140-18 installer]$ oc get subscription
NAME         PACKAGE   SOURCE         CHANNEL
etcd-kxcgh   etcd      rh-operators   alpha

Comment 3 Jeff Peeler 2018-11-27 17:15:10 UTC
Bug 1626434, comment 20 outlines a suitable way for QE to keep track of what upstream code was built. I think this bug can be moved to ON_QA.

Comment 4 Evan Cordell 2018-11-27 18:57:19 UTC
This is fixed in the latest version.

Comment 5 Jian Zhang 2018-11-28 05:33:36 UTC
Verify it per comment 2.

Comment 8 errata-xmlrpc 2019-06-04 10:40:51 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0758