Bug 1831818
| Summary: | Switching an OpearatorSource to a CatalogSource as default for marketplace does not clean old OperatorSource | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Anik <anbhatta> |
| Component: | OLM | Assignee: | Anik <anbhatta> |
| OLM sub component: | OLM | QA Contact: | Jian Zhang <jiazha> |
| Status: | CLOSED ERRATA | Docs Contact: | |
| Severity: | medium | ||
| Priority: | medium | CC: | ecordell, kuiwang |
| Version: | 4.5 | ||
| Target Milestone: | --- | ||
| Target Release: | 4.5.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2020-07-13 17:35:12 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Hi Anik, I'm sorry, but I'm confused. I have some questions below: > 4. Build and push an image to a registry with the above changes Do you mean this CatalogSource image: quay.io/operator-framework/upstream-community-operators:latest? > 5. Point to the image built and pushed above here https://github.com/operator-framework/operator-marketplace/blob/master/manifests/09_operator.yaml#L36 > 6. oc apply -f manifests/ Seems like we're reinstalling the marketplace-operator itself, why not recover it by using the CVO? $ oc patch clusterversion version --type=merge -p '{"spec": {"overrides":[{"kind": "Deployment", "name": "marketplace-operator", "namespace": "openshift-marketplace", "managed": true, "group": "apps"}]}}' One more thing, even if we didn't do any actions for the marketplace-operator itself, the old operator-source still active, as I described in https://issues.redhat.com/browse/OLM-1615. Is it the same issue? Thanks! Hi Jian, Yes it's the same issue you mentioned in JIRA. This bz is to fix that issue. As for the patching and restarting of Marketplace, apologizes if the way I framed it is confusing. Basically, you just have to switch one of the operatorsource to a catsrc, and see that the old operatorsource was not being removed. With this PR in, the old operatorsource will be removed. Hope that clarifies things. Hi Anik, Thanks for your clarification! Test steps as follows: Cluster version is 4.5.0-0.nightly-2020-05-13-221558 Marketplace version: mac:~ jianzhang$ oc exec marketplace-operator-85fbdccd7-llcnx -- marketplace-operator --version time="2020-05-15T10:00:25Z" level=info msg="Go Version: go1.13.4" time="2020-05-15T10:00:25Z" level=info msg="Go OS/Arch: linux/amd64" time="2020-05-15T10:00:25Z" level=info msg="operator-sdk Version: v0.8.0" Marketplace source git commit: b1ba97ff4cf3999fd8fcdc2c97700d5291dca1f0 1, Check the default OperatorSource, they worked well. mac:~ jianzhang$ oc get operatorsource NAME TYPE ENDPOINT REGISTRY DISPLAYNAME PUBLISHER STATUS MESSAGE AGE certified-operators appregistry https://quay.io/cnr certified-operators Certified Operators Red Hat Succeeded The object has been successfully reconciled 28h community-operators appregistry https://quay.io/cnr community-operators Community Operators Red Hat Succeeded The object has been successfully reconciled 28h redhat-marketplace appregistry https://quay.io/cnr redhat-marketplace Red Hat Marketplace Red Hat Succeeded The object has been successfully reconciled 28h redhat-operators appregistry https://quay.io/cnr redhat-operators Red Hat Operators Red Hat Succeeded The object has been successfully reconciled 28h mac:~ jianzhang$ oc get catalogsource NAME DISPLAY TYPE PUBLISHER AGE certified-operators Certified Operators grpc Red Hat 28h community-operators Community Operators grpc Red Hat 28h redhat-marketplace Red Hat Marketplace grpc Red Hat 28h redhat-operators Red Hat Operators grpc Red Hat 28h mac:~ jianzhang$ oc get pods NAME READY STATUS RESTARTS AGE certified-operators-68f8c69857-hmrc9 1/1 Running 0 23h community-operators-5f77c8cf4d-t5rpv 1/1 Running 0 23h marketplace-operator-85fbdccd7-llcnx 1/1 Running 0 28h redhat-marketplace-cc46cd9bb-cq4d6 1/1 Running 0 23h redhat-operators-57dbd6fcb-b58xw 1/1 Running 0 28h 2, Convert the one of OperatorSource (community-operator) to CatalogSource. mac:~ jianzhang$ cat cs-community-45.yaml apiVersion: operators.coreos.com/v1alpha1 kind: CatalogSource metadata: name: operatorhubio-catalog namespace: openshift-marketplace spec: sourceType: grpc image: quay.io/operator-framework/upstream-community-operators:latest displayName: Community Test Operators publisher: OperatorHub.io mac:~ jianzhang$ oc create -f cs-community-45.yaml catalogsource.operators.coreos.com/operatorhubio-catalog created 3, Check the old community-operator OperatorSource status. But, it still exists. [root@preserve-olm-env ~]# oc get catalogsource NAME DISPLAY TYPE PUBLISHER AGE certified-operators Certified Operators grpc Red Hat 29h community-operators Community Operators grpc Red Hat 29h operatorhubio-catalog Community Test Operators grpc OperatorHub.io 6m47s redhat-marketplace Red Hat Marketplace grpc Red Hat 29h redhat-operators Red Hat Operators grpc Red Hat 29h [root@preserve-olm-env ~]# oc get pods NAME READY STATUS RESTARTS AGE certified-operators-68f8c69857-hmrc9 1/1 Running 0 24h community-operators-5f77c8cf4d-t5rpv 1/1 Running 0 24h marketplace-operator-85fbdccd7-llcnx 1/1 Running 0 29h operatorhubio-catalog-hd7zn 1/1 Running 0 6m53s redhat-marketplace-cc46cd9bb-cq4d6 1/1 Running 0 24h redhat-operators-57dbd6fcb-b58xw 1/1 Running 0 29h [root@preserve-olm-env ~]# oc get operatorsource NAME TYPE ENDPOINT REGISTRY DISPLAYNAME PUBLISHER STATUS MESSAGE AGE certified-operators appregistry https://quay.io/cnr certified-operators Certified Operators Red Hat Succeeded The object has been successfully reconciled 29h community-operators appregistry https://quay.io/cnr community-operators Community Operators Red Hat Succeeded The object has been successfully reconciled 29h redhat-marketplace appregistry https://quay.io/cnr redhat-marketplace Red Hat Marketplace Red Hat Succeeded The object has been successfully reconciled 29h redhat-operators appregistry https://quay.io/cnr redhat-operators Red Hat Operators Red Hat Succeeded The object has been successfully reconciled 29h The test case listed above is incorrect. Please see the description of the problem above for the correct steps to test this. Please use this doc as reference: https://github.com/operator-framework/operator-marketplace/blob/master/CONTRIBUTING.md#testing-changes-to-rbac-policy NOTE: You need to rebuild and redeploy marketplace operator. The defaults are bootstrapped by marketplace operator from here: https://github.com/operator-framework/operator-marketplace/blob/master/defaults/ Simply changing an operatorsource to a catsrc in cluster will not work since marketplace will maintain the state of the cluster it received during bootstrap. Hi Anik, Thanks! Sorry, I misunderstood it. For this bug, the test steps as follows: 1, Remove the default community-operator OperatorSource YAML file and add a CatalogSource YAML file, as follows: https://github.com/jianzhangbjz/operator-marketplace/blob/cs/defaults/03_community_operators.yaml 2, Build the marketplace-operator image. [root@preserve-olm-env operator-marketplace]# docker build . --tag=quay.io/olmqe/marketplace-operator Sending build context to Docker daemon 261.4MB Step 1/12 : FROM registry.svc.ci.openshift.org/openshift/release:golang-1.10 AS builder golang-1.10: Pulling from openshift/release 524b0c1e57f8: Already exists 7201192efb82: Pull complete Digest: sha256:0378b72464970785d0b83b400d5a83b7c4cb2d0a997fe35d087f77ce3997ad0f Status: Downloaded newer image for registry.svc.ci.openshift.org/openshift/release:golang-1.10 ---> eae4d37ce64f Step 2/12 : WORKDIR /go/src/github.com/operator-framework/operator-marketplace ---> Running in 1f65310a0ce9 Removing intermediate container 1f65310a0ce9 ---> 5a1a88298e0f Step 3/12 : COPY . . ---> 0cb0bd22a678 Step 4/12 : RUN make osbs-build ---> Running in af4f869c29a1 # hack/build.sh ./build/build.sh building marketplace-operator... Removing intermediate container af4f869c29a1 ---> 09cfa76fe23a Step 5/12 : FROM registry.svc.ci.openshift.org/openshift/origin-v4.0:base base: Pulling from openshift/origin-v4.0 c2340472a0fa: Pull complete 6e55351c18ff: Pull complete b2d2704dda6c: Pull complete a2c10be042b9: Pull complete Digest: sha256:cbd0e2931e6ae8cbd1522cd7226cff89629c21a3652f6fd57d84f405d432c07a Status: Downloaded newer image for registry.svc.ci.openshift.org/openshift/origin-v4.0:base ---> f779ab8af41a Step 6/12 : RUN useradd marketplace-operator ---> Running in 7396c74faf02 Removing intermediate container 7396c74faf02 ---> 9e808118e7b3 Step 7/12 : USER marketplace-operator ---> Running in 7a9fe184e9f6 Removing intermediate container 7a9fe184e9f6 ---> a1297ffa8ddd Step 8/12 : COPY --from=builder /go/src/github.com/operator-framework/operator-marketplace/build/_output/bin/marketplace-operator /usr/bin ---> 82a3cd539038 Step 9/12 : ADD manifests /manifests ---> 70e616c6a11b Step 10/12 : ADD defaults /defaults ---> 98f2702dc2b6 Step 11/12 : LABEL io.k8s.display-name="OpenShift Marketplace Operator" io.k8s.description="This is a component of OpenShift Container Platform and manages the OpenShift Marketplace." io.openshift.tags="openshift,marketplace" io.openshift.release.operator=true maintainer="AOS Marketplace <aos-marketplace>" ---> Running in 52904f1fd7c9 Removing intermediate container 52904f1fd7c9 ---> fc5c91dd2785 Step 12/12 : CMD ["/usr/bin/marketplace-operator"] ---> Running in 6d7514d7c51c Removing intermediate container 6d7514d7c51c ---> 25cdb6fe6af2 Successfully built 25cdb6fe6af2 Successfully tagged quay.io/olmqe/marketplace-operator:latest And, retag and push it to the Quay. [root@preserve-olm-env operator-marketplace]# docker tag quay.io/olmqe/marketplace-operator quay.io/olmqe/marketplace-operator:cs [root@preserve-olm-env operator-marketplace]# docker push quay.io/olmqe/marketplace-operator:cs The push refers to repository [quay.io/olmqe/marketplace-operator] 8a4d09641909: Pushed cc5492997673: Pushed 302efb89260f: Pushed f65c5c3b30fc: Pushed 4ef6a7eaf5a6: Pushed 762eded89aa4: Pushed e18335b29b2e: Pushed 4f25b188f48c: Pushed cs: digest: sha256:d9fad855cebb41038567fa1b80e5cf9a8abe83997bc6ac3e4c2cc3c1d1ef1bd1 size: 1990 3, Set this image in the marketplace-operator manifest, as follows: https://github.com/jianzhangbjz/operator-marketplace/blob/cs/manifests/09_operator.yaml 4, Set the marketplace-operator to `Unmanaged` for the CVO. And, delete the marketplace-operator. mac:operator-marketplace jianzhang$ oc patch clusterversion version --type=merge -p '{"spec": {"overrides":[{"kind": "Deployment", "name": "marketplace-operator", "namespace": "openshift-marketplace", "unmanaged": true, "group": "apps"}]}}' mac:operator-marketplace jianzhang$ oc delete deployment marketplace-operator -n openshift-marketplace deployment.apps "marketplace-operator" deleted 5, Redeploy the marketplace-operator. mac:operator-marketplace jianzhang$ oc create -f manifests/09_operator.yaml deployment.apps/marketplace-operator created mac:operator-marketplace jianzhang$ oc get deployment NAME READY UP-TO-DATE AVAILABLE AGE certified-operators 1/1 1 1 4h18m marketplace-operator 1/1 1 1 26s qe-app-registry 1/1 1 1 4h4m redhat-marketplace 1/1 1 1 4h18m redhat-operators 1/1 1 1 4h18m mac:operator-marketplace jianzhang$ oc get pods NAME READY STATUS RESTARTS AGE certified-operators-5c65f7c4c5-lmkdc 1/1 Running 0 2m15s community-operators-699ps 1/1 Running 0 2m20s marketplace-operator-5cc9b978d-qkl5m 1/1 Running 0 2m39s qe-app-registry-589b449c66-b2z6d 1/1 Running 0 2m13s redhat-marketplace-bb99b89f7-g9h99 1/1 Running 0 2m18s redhat-operators-5f865d76f6-5cbvb 1/1 Running 0 2m17s The old community operator resources had been removed. Looks good to me, verify it. mac:operator-marketplace jianzhang$ oc get operatorsource NAME TYPE ENDPOINT REGISTRY DISPLAYNAME PUBLISHER STATUS MESSAGE AGE certified-operators appregistry https://quay.io/cnr certified-operators Certified Operators Red Hat Succeeded The object has been successfully reconciled 4h20m qe-app-registry appregistry https://quay.io/cnr openshift-qe-optional-operators Succeeded The object has been successfully reconciled 4h7m redhat-marketplace appregistry https://quay.io/cnr redhat-marketplace Red Hat Marketplace Red Hat Succeeded The object has been successfully reconciled 4h20m redhat-operators appregistry https://quay.io/cnr redhat-operators Red Hat Operators Red Hat Succeeded The object has been successfully reconciled 4h20m mac:operator-marketplace jianzhang$ oc get catalogsource NAME DISPLAY TYPE PUBLISHER AGE certified-operators Certified Operators grpc Red Hat 4h21m community-operators Community Operators grpc Red Hat 2m52s qe-app-registry grpc 4h7m redhat-marketplace Red Hat Marketplace grpc Red Hat 4h20m redhat-operators Red Hat Operators grpc Red Hat 4h20m mac:operator-marketplace jianzhang$ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.5.0-0.nightly-2020-05-18-225907 True False 6h17m Cluster version is 4.5.0-0.nightly-2020-05-18-225907 mac:operator-marketplace jianzhang$ oc exec marketplace-operator-5cc9b978d-qkl5m -- marketplace-operator --version Marketplace source git commit: time="2020-05-19T09:37:28Z" level=info msg="Go Version: go1.10.8" time="2020-05-19T09:37:28Z" level=info msg="Go OS/Arch: linux/amd64" time="2020-05-19T09:37:28Z" level=info msg="operator-sdk Version: v0.8.0" Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:2409 |
Description of problem: In OCP 4.5, the OperatorHub API has been enhanced to accept both OperatorSource and CatalogSource as default operator sources for marketplace. However, when an OperatorSource is switched to a CatalogSource, the old OperatorSource is not cleared. Version-Release number of selected component (if applicable): 4.5 How reproducible: Always Steps to Reproduce: 1. Launch OCP 4.5 cluster 2. Patch and delete marketplace operator $ oc patch clusterversion version --type=merge -p '{"spec": {"overrides":[{"kind": "Deployment", "name": "marketplace-operator", "namespace": "openshift-marketplace", "unmanaged": true, "group": "apps"}]}}' $ oc delete deployment marketplace-operator -n openshift-marketplace 3. Switch https://github.com/operator-framework/operator-marketplace/blob/master/defaults/03_community_operators.yaml to the following Catsrc: ```catsrc.yaml apiVersion: "operators.coreos.com/v1" kind: "CatalogSource" metadata: name: "community-operators" namespace: "openshift-marketplace" spec: sourceType: grpc image: quay.io/operator-framework/upstream-community-operators:latest displayName: "Community Operators" publisher: "Red Hat" updateStrategy: registryPoll: interval: 45m 4. Build and push an image to a registry with the above changes 5. Point to the image built and pushed above here https://github.com/operator-framework/operator-marketplace/blob/master/manifests/09_operator.yaml#L36 6. oc apply -f manifests/ Actual results: There's 6 registry pods: $ oc get pods NAME READY STATUS RESTARTS AGE certified-operators-674686b9cd-ll7h9 1/1 Running 0 42m community-operators-66469948f8-pv6h7 1/1 Running 0 41m marketplace-operator-57fd59469f-tb85r 1/1 Running 0 42m redhat-marketplace-565549d5cc-kxddh 1/1 Running 0 42m redhat-operators-6f6bf8bcf7-5tnvw 1/1 Running 0 42m Expected results: There's 3 opsrc, 1 catalogSource, and 4 registry pods, one for each source. Additional info: https://github.com/operator-framework/operator-marketplace/pull/300