Bug 1913822

Summary: Install plan not created
Product: OpenShift Container Platform Reporter: Neil Girard <ngirard>
Component: OLMAssignee: Vu Dinh <vdinh>
OLM sub component: OLM QA Contact: Jian Zhang <jiazha>
Status: CLOSED WORKSFORME Docs Contact:
Severity: high    
Priority: high CC: ankithom, apjagtap, bluddy, krizza, vdinh
Version: 4.6   
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-02-19 17:23:43 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Neil Girard 2021-01-07 16:57:54 UTC 
Description of problem:
When installing a set of operators, one which has 2 other operators and a dependency / required, the cluster in 4.6 has issues with the install plan.  it is either being removed during install or is not being generated.  When it fails, there are messages in the catalog operator logs stating:

2021-01-07T14:13:36.892538053Z time="2021-01-07T14:13:36Z" level=warning msg="unable to get installplan from cache" channel=v1.4 id=RCDFn installplan=install-lxtkp namespace=e2e-operator-testing-promotion pkg=couchdb-operator-certified source=certified-operators sub=couchdb-operator-certified-v1.4-certified-operators-openshift-marketplace
2021-01-07T14:13:36.892583048Z E0107 14:13:36.892539       1 queueinformer_operator.go:290] sync "e2e-operator-testing-promotion" failed: installplans.operators.coreos.com "install-lxtkp" not found


Customer states that these operators install fine in their 4.5.x clusters, but when running in 4.6, it works in one namespace but fails when installing into a second.

Version-Release number of selected component (if applicable):
4.6.8

How reproducible:
Constantly for customer, unable to recreate yet in lab cluster.

Steps to Reproduce:
1.
2.
3.

Actual results:
Install plan is not generated

Expected results:
Install plan is generated

Additional info:
Comment 2 Neil Girard 2021-01-07 17:11:45 UTC 
This is the subscription that was created for one of the 3 operators to show install plan generated in metadata/annotations:

$ cat couchdb-operator-certified-v1.4-certified-operators-openshift-marketplace.yaml 
---
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  annotations:
    olm.generated-by: install-lxtkp
  creationTimestamp: "2021-01-06T11:14:07Z"
  generation: 1
  labels:
    operators.coreos.com/couchdb-operator-certified.e2e-operator-testing-promotion: ""
  managedFields:
  - apiVersion: operators.coreos.com/v1alpha1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:labels:
          .: {}
          f:operators.coreos.com/couchdb-operator-certified.e2e-operator-testing-promotion: {}
    manager: olm
    operation: Update
    time: "2021-01-06T11:14:07Z"
  - apiVersion: operators.coreos.com/v1alpha1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .: {}
          f:olm.generated-by: {}
      f:spec:
        .: {}
        f:channel: {}
        f:config:
          .: {}
          f:resources: {}
        f:installPlanApproval: {}
        f:name: {}
        f:source: {}
        f:sourceNamespace: {}
        f:startingCSV: {}
      f:status:
        .: {}
        f:catalogHealth: {}
        f:conditions: {}
        f:installedCSV: {}
        f:lastUpdated: {}
    manager: catalog
    operation: Update
    time: "2021-01-06T11:27:28Z"
  name: couchdb-operator-certified-v1.4-certified-operators-openshift-marketplace
  namespace: e2e-operator-testing-promotion
  resourceVersion: "101616"
  selfLink: /apis/operators.coreos.com/v1alpha1/namespaces/e2e-operator-testing-promotion/subscriptions/couchdb-operator-certified-v1.4-certified-operators-openshift-marketplace
  uid: bb4a6719-54a9-4bfb-ad4f-1e6d266de3d8
spec:
  channel: v1.4
  config:
    resources: {}
  installPlanApproval: Automatic
  name: couchdb-operator-certified
  source: certified-operators
  sourceNamespace: openshift-marketplace
  startingCSV: couchdb-operator.v1.4.2
status:
  catalogHealth:
  - catalogSourceRef:
      apiVersion: operators.coreos.com/v1alpha1
      kind: CatalogSource
      name: appconnect-operator-catalogsource
      namespace: openshift-marketplace
      resourceVersion: "100575"
      uid: 07ba164a-c8d1-403c-abbf-0abae10bc5cf
    healthy: true
    lastUpdated: "2021-01-06T11:27:27Z"
  - catalogSourceRef:
      apiVersion: operators.coreos.com/v1alpha1
      kind: CatalogSource
      name: automation-base-pak-operators
      namespace: openshift-marketplace
      resourceVersion: "100725"
      uid: d369f9bb-841d-4ee3-acc3-ce9517313d1a
    healthy: true
    lastUpdated: "2021-01-06T11:27:27Z"
  - catalogSourceRef:
      apiVersion: operators.coreos.com/v1alpha1
      kind: CatalogSource
      name: certified-operators
      namespace: openshift-marketplace
      resourceVersion: "97844"
      uid: 16c690b8-4e09-4c87-8437-236a3e2ca911
    healthy: true
    lastUpdated: "2021-01-06T11:27:27Z"
  - catalogSourceRef:
      apiVersion: operators.coreos.com/v1alpha1
      kind: CatalogSource
      name: cloud-pak-for-data-operator
      namespace: openshift-marketplace
      resourceVersion: "97826"
      uid: 192d164c-9c9d-4615-9f40-44e9b0ad72bc
    healthy: true
    lastUpdated: "2021-01-06T11:27:27Z"
  - catalogSourceRef:
      apiVersion: operators.coreos.com/v1alpha1
      kind: CatalogSource
      name: community-operators
      namespace: openshift-marketplace
      resourceVersion: "97476"
      uid: 041a83e4-16db-4e66-adee-c45af353633b
    healthy: true
    lastUpdated: "2021-01-06T11:27:27Z"
  - catalogSourceRef:
      apiVersion: operators.coreos.com/v1alpha1
      kind: CatalogSource
      name: opencloud-operators
      namespace: openshift-marketplace
      resourceVersion: "97822"
      uid: 26e19e19-7bd9-4733-968c-38f3efd0afea
    healthy: true
    lastUpdated: "2021-01-06T11:27:27Z"
  - catalogSourceRef:
      apiVersion: operators.coreos.com/v1alpha1
      kind: CatalogSource
      name: redhat-marketplace
      namespace: openshift-marketplace
      resourceVersion: "97978"
      uid: f7cdcd50-8913-402c-8e1a-dd5ba4e97bcf
    healthy: true
    lastUpdated: "2021-01-06T11:27:27Z"
  - catalogSourceRef:
      apiVersion: operators.coreos.com/v1alpha1
      kind: CatalogSource
      name: redhat-operators
      namespace: openshift-marketplace
      resourceVersion: "97825"
      uid: 495d5009-fe0c-4fbb-8b57-b0f8832160ac
    healthy: true
    lastUpdated: "2021-01-06T11:27:27Z"
  conditions:
  - lastTransitionTime: "2021-01-06T11:27:27Z"
    message: all available catalogsources are healthy
    reason: AllCatalogSourcesHealthy
    status: "False"
    type: CatalogSourcesUnhealthy
  installedCSV: couchdb-operator.v1.4.2
  lastUpdated: "2021-01-06T11:27:27Z"