Bug 1908431

Summary: When the marketplace-operator pod get's restarted, the custom catalogsources are gone, as well as the pods
Product: OpenShift Container Platform Reporter: Andy Bartlett <andbartl>
Component: OLMAssignee: Evan Cordell <ecordell>
OLM sub component: OperatorHub QA Contact: Salvatore Colangelo <scolange>
Status: CLOSED ERRATA Docs Contact:
Severity: urgent    
Priority: urgent CC: bluddy, hchatter, jiazha, krizza, nhale, openshift-bugs-escalate, pneedle, scolange, scuppett
Version: 4.6Keywords: Triaged, UpcomingSprint
Target Milestone: ---   
Target Release: 4.7.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause and consequence: When a default catalog source in the `openshift-marketplace` namespace is disabled through the OperatorHub API, you can a create custom catalog source with the same name as that default. Previously, custom catalog sources with the same name as a default catalog source were deleted by the Marketplace Operator when the marketplace was restarted. Fix: An annotation has been added to the default catalog sources that are created by the Marketplace Operator. Result: Now the Operator only deletes the catalog sources that contain the annotation when the marketplace is restarted. Custom catalog sources created with the same name as the default catalog sources are not deleted.
Story Points: ---
Clone Of:
: 1915905 (view as bug list) Environment:
Last Closed: 2021-02-24 15:45:23 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1915905    

Description Andy Bartlett 2020-12-16 17:12:01 UTC
Description of problem:

We use custom catalogsources created with the same name. When the openshift marketplace pod gets Terminated and started again, the catalogsources are removed.

We noticed this behaviour yesterday when the cluster was upgraded from 4.6.6. to 4.6.8 yesterday.

Version-Release number of selected component (if applicable):

OCP 4.6.8


How reproducible:

100%


Steps to Reproduce:

$ oc get clusterversion
NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.6.8     True        False         28h     Cluster version is 4.6.8

$ oc get pods -n openshift-marketplace
NAME                                    READY   STATUS    RESTARTS   AGE
certified-operators-29b59               1/1     Running   0          2m22s
community-operators-fjltj               1/1     Running   0          2m22s
marketplace-operator-86d777b646-w74dq   1/1     Running   0          8m20s
redhat-operators-2vbjb                  1/1     Running   0          2m24s

$ oc get operatorhubs.config.openshift.io cluster -o jsonpath="{.spec}"
{"disableAllDefaultSources":true}

$ oc -n openshift-marketplace delete pod marketplace-operator-86d777b646-w74dq
pod "marketplace-operator-86d777b646-w74dq" deleted

$ oc -n openshift-marketplace get pods
NAME                                    READY   STATUS    RESTARTS   AGE
marketplace-operator-86d777b646-vx9qg   1/1     Running   0          24s

$ oc -n openshift-marketplace get catalogsources.operators.coreos.com
No resources found in openshift-marketplace namespace.

Actual results:

custom catalogsources are deleted

Expected results:

custom catalogsources should be maintained, not removed during a deletion or restart of the marketplace pods


Additional info:

Comment 8 Salvatore Colangelo 2021-01-15 14:36:42 UTC
[scolange@scolange ~]$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.7.0-0.nightly-2021-01-14-211319   True        False         5h23m   Cluster version is 4.7.0-0.nightly-2021-01-14-211319

1. Created ConfigMap

[scolange@scolange BUG-1732914]$ oc create -f 1.configmap_no_ns_etcd-v5.yaml -n openshift-marketplace
configmap/installed-community-global-operators created

2. Created CatalogSource

[scolange@scolange BUG-1732914]$ oc create -f 2.catalogsource.yaml -n openshift-marketplace
catalogsource.operators.coreos.com/installed-community-global-operators-test created

3. Wait the catalog source go READY

[scolange@scolange BUG-1732914]$ oc get catalogsource installed-community-global-operators-test -o yaml
apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  creationTimestamp: "2021-01-15T14:16:33Z"
  generation: 1
  managedFields:
  - apiVersion: operators.coreos.com/v1alpha1
    fieldsType: FieldsV1
    fieldsV1:
      f:spec:
        .: {}
        f:configMap: {}
        f:displayName: {}
        f:icon:
          .: {}
          f:base64data: {}
          f:mediatype: {}
        f:publisher: {}
        f:sourceType: {}
    manager: oc
    operation: Update
    time: "2021-01-15T14:16:33Z"
  - apiVersion: operators.coreos.com/v1alpha1
    fieldsType: FieldsV1
    fieldsV1:
      f:status:
        .: {}
        f:configMapReference:
          .: {}
          f:lastUpdateTime: {}
          f:name: {}
          f:namespace: {}
          f:resourceVersion: {}
          f:uid: {}
        f:connectionState:
          .: {}
          f:address: {}
          f:lastConnect: {}
          f:lastObservedState: {}
        f:registryService:
          .: {}
          f:createdAt: {}
          f:port: {}
          f:protocol: {}
          f:serviceName: {}
          f:serviceNamespace: {}
    manager: catalog
    operation: Update
    time: "2021-01-15T14:16:35Z"
  name: installed-community-global-operators-test
  namespace: openshift-marketplace
  resourceVersion: "134064"
  selfLink: /apis/operators.coreos.com/v1alpha1/namespaces/openshift-marketplace/catalogsources/installed-community-global-operators-test
  uid: c1d576f8-4c0c-4871-997c-14527a633f4f
spec:
  configMap: installed-community-global-operators
  displayName: Community Operators Test
  icon:
    base64data: ""
    mediatype: ""
  publisher: Community
  sourceType: internal
status:
  configMapReference:
    lastUpdateTime: "2021-01-15T14:16:33Z"
    name: installed-community-global-operators
    namespace: openshift-marketplace
    resourceVersion: "133897"
    uid: 1a56a459-1a26-43f4-bf1c-bc52bc7bd859
  connectionState:
    address: installed-community-global-operators-test.openshift-marketplace.svc:50051
    lastConnect: "2021-01-15T14:16:54Z"
    lastObservedState: READY
  registryService:
    createdAt: "2021-01-15T14:16:35Z"
    port: "50051"
    protocol: grpc
    serviceName: installed-community-global-operators-test
    serviceNamespace: openshift-marketplace

5. Verify the properties of OperatorHub

[scolange@scolange BUG-1732914]$ oc get operatorhubs.config.openshift.io cluster -o jsonpath="{.spec}"
map[disableAllDefaultSources:true]

6.Verify all pod in openshift-marketplace

[scolange@scolange BUG-1732914]$ oc get pod
NAME                                                              READY   STATUS      RESTARTS   AGE
installed-community-global-operators-test-p8rf7                   1/1     Running     0          9m6s
marketplace-operator-868847c68c-nsxfz                             1/1     Running     0          18m
qe-app-registry-gxkbh  

7. Delete the main pod of marketplace
[scolange@scolange BUG-1732914]$ oc delete pod marketplace-operator-868847c68c-nsxfz
pod "marketplace-operator-868847c68c-nsxfz" deleted

7. Verify if it is cancelled
[scolange@scolange BUG-1732914]$ oc get pod
NAME                                                              READY   STATUS      RESTARTS   AGE
installed-community-global-operators-test-p8rf7                   1/1     Running     0          9m31s
marketplace-operator-868847c68c-5lqff                             1/1     Running     0          12s
qe-app-registry-gxkbh                                             1/1     Running     0          4h43m


8. Verify if the custuom catalogSource
[scolange@scolange BUG-1732914]$ oc get catalogsources.operators.coreos.com 
NAME                                        DISPLAY                    TYPE       PUBLISHER      AGE
installed-community-global-operators-test   Community Operators Test   internal   Community      9m56s
qe-app-registry                             Production Operators       grpc       OpenShift QE   5h47m


LGMT

Comment 13 errata-xmlrpc 2021-02-24 15:45:23 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:5633