Bug 2070792
| Summary: | service "openshift-marketplace/marketplace-operator-metrics" is not annotated with capability | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Yang Yang <yanyang> |
| Component: | OLM | Assignee: | Per da Silva <pegoncal> |
| OLM sub component: | OperatorHub | QA Contact: | Jian Zhang <jiazha> |
| Status: | CLOSED ERRATA | Docs Contact: | |
| Severity: | medium | ||
| Priority: | urgent | CC: | wking |
| Version: | 4.11 | ||
| Target Milestone: | --- | ||
| Target Release: | 4.11.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2022-08-10 11:03:06 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Auditing a recent CI build: $ oc adm release extract --to manifests registry.ci.openshift.org/ocp/release:4.11.0-0.ci-2022-04-04-041224 Looks pretty good to me: $ grep -cr 'capability.openshift.io/name: "marketplace"' manifests | grep 0000_50_operator-marketplace_ manifests/0000_50_operator-marketplace_01_namespace.yaml:1 manifests/0000_50_operator-marketplace_04_service_account.yaml:1 manifests/0000_50_operator-marketplace_05_role.yaml:3 manifests/0000_50_operator-marketplace_06_role_binding.yaml:2 manifests/0000_50_operator-marketplace_07_configmap.yaml:1 manifests/0000_50_operator-marketplace_08_service.yaml:1 manifests/0000_50_operator-marketplace_09_operator-ibm-cloud-managed.yaml:1 manifests/0000_50_operator-marketplace_09_operator.yaml:1 manifests/0000_50_operator-marketplace_10_clusteroperator.yaml:1 manifests/0000_50_operator-marketplace_11_service_monitor.yaml:3 manifests/0000_50_operator-marketplace_12_prometheus_rule.yaml:1 $ diff -ru <(grep -cr 'capability.openshift.io/name: "marketplace"' manifests | grep 0000_50_operator-marketplace_) <(grep -cr '^kind: ' manifests | grep 0000_50_operator-marketplace_) ...no difference... I expect the next run linked from [1] to show the fix working. [2] uses commit 372ad0a7ffd90cde146c36c31d49ef930c302a8e , which is too old to contain [3]. [1]: https://testgrid.k8s.io/redhat-openshift-ocp-release-4.11-informing#periodic-ci-openshift-release-master-ci-4.11-e2e-aws-no-capabilities [2]: https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.11-e2e-aws-no-capabilities/1510235989061144576 [3]: https://github.com/operator-framework/operator-marketplace/pull/465#event-6355531345 [1] doesn't hit this issue, so I'm moving back to ON_QA for another pass at verification. [1]: https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.11-e2e-aws-no-capabilities/1510960797176565760 Hi W.Trevor, Yes, thanks! I retest it and details as follows: 1, Create an OCP with the below configs to disable the marketplace-operator, for example, https://mastern-jenkins-csb-openshift-qe.apps.ocp-c1.prod.psi.redhat.com/job/ocp-common/job/Flexy-install/91472/parameters/ baselineCapabilitySet: None mac:~ jianzhang$ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.11.0-0.nightly-2022-04-06-000911 True False 16m Cluster version is 4.11.0-0.nightly-2022-04-06-000911 mac:~ jianzhang$ oc get ns|grep marketplace mac:~ jianzhang$ oc get catalogsource No openshift-marketplace project found, looks good. 2, Check if service `marketplace-operator-metrics` disabled. mac:~ jianzhang$ oc get pods -n openshift-cluster-version NAME READY STATUS RESTARTS AGE cluster-version-operator-5b88bfbcb4-fp5n7 1/1 Running 0 46m mac:~ jianzhang$ oc -n openshift-cluster-version logs cluster-version-operator-5b88bfbcb4-fp5n7 > cvo.log mac:~ jianzhang$ grep "marketplace-operator-metrics" cvo.log I0406 02:55:20.354733 1 sync_worker.go:860] Running sync for service "openshift-marketplace/marketplace-operator-metrics" (609 of 786) I0406 02:55:20.354745 1 sync_worker.go:868] Skipping precreation of service "openshift-marketplace/marketplace-operator-metrics" (609 of 786): disabled capabilities: marketplace The "openshift-marketplace/marketplace-operator-metrics" service was skipped. Looks good to me, verify it. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Important: OpenShift Container Platform 4.11.0 bug fix and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:5069 |
Description of problem: Openshift-marketplace was made optional in [1]. But service "openshift-marketplace/marketplace-operator-metrics" does not have the annotation [2]. When installing a cluster with openshift-marketplace disabled, CVO loops on service "openshift-marketplace/marketplace-operator-metrics" creation for a long while because namespace is not present. $ oc get ns | grep openshift-marketplace Nothing returns $ grep EnabledCapabilities cvo.log I0331 10:15:53.003275 1 status.go:168] Synchronizing status errs=field.ErrorList(nil) status=&cvo.SyncWorkerStatus{Generation:2, Failure:(*payload.UpdateError)(0xc0027f5140), Done:728, Total:784, Completed:0, Reconciling:false, Initial:true, VersionHash:"GlrgMJKYA7o=", LastProgress:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Actual:v1.Release{Version:"4.11.0-0.ci.test-2022-03-31-085516-ci-ln-fhbly7b-latest", Image:"registry.build01.ci.openshift.org/ci-ln-fhbly7b/release@sha256:61f4433e35cdb00893b97af8a06c93390533c7fe00c13a2495976701d1c50f51", URL:"", Channels:[]string(nil)}, Verified:false, loadPayloadStatus:cvo.LoadPayloadStatus{Step:"PayloadLoaded", Message:"Payload loaded version=\"4.11.0-0.ci.test-2022-03-31-085516-ci-ln-fhbly7b-latest\" image=\"registry.build01.ci.openshift.org/ci-ln-fhbly7b/release@sha256:61f4433e35cdb00893b97af8a06c93390533c7fe00c13a2495976701d1c50f51\"", Failure:error(nil), Release:v1.Release{Version:"4.11.0-0.ci.test-2022-03-31-085516-ci-ln-fhbly7b-latest", Image:"registry.build01.ci.openshift.org/ci-ln-fhbly7b/release@sha256:61f4433e35cdb00893b97af8a06c93390533c7fe00c13a2495976701d1c50f51", URL:"", Channels:[]string(nil)}, Verified:false, LastTransitionTime:time.Time{wall:0xc0897ab9402be75b, ext:665050712046, loc:(*time.Location)(0x2c58160)}}, CapabilitiesStatus:cvo.CapabilityStatus{Status:v1.ClusterVersionCapabilitiesStatus{EnabledCapabilities:[]v1.ClusterVersionCapability(nil), KnownCapabilities:[]v1.ClusterVersionCapability{"baremetal", "marketplace", "openshift-samples"}}, ImplicitlyEnabledCaps:[]v1.ClusterVersionCapability(nil)}} $ grep openshift-marketplace cvo.log I0331 09:24:51.726733 1 payload.go:199] excluding 0000_50_operator-marketplace_09_operator-ibm-cloud-managed.yaml group=apps kind=Deployment namespace=openshift-marketplace name=marketplace-operator: include.release.openshift.io/self-managed-high-availability unset I0331 09:24:52.860177 1 payload.go:199] excluding 0000_50_operator-marketplace_09_operator-ibm-cloud-managed.yaml group=apps kind=Deployment namespace=openshift-marketplace name=marketplace-operator: include.release.openshift.io/self-managed-high-availability unset I0331 09:24:53.045988 1 sync_worker.go:843] Skipping precreation of clusteroperator "openshift-marketplace/marketplace" (608 of 784): disabled capabilities: marketplace I0331 09:24:53.046101 1 sync_worker.go:860] Running sync for namespace "openshift-marketplace" (598 of 784) I0331 09:24:53.046144 1 sync_worker.go:868] Skipping precreation of namespace "openshift-marketplace" (598 of 784): disabled capabilities: marketplace I0331 09:24:53.046185 1 sync_worker.go:860] Running sync for serviceaccount "openshift-marketplace/marketplace-operator" (599 of 784) I0331 09:24:53.046223 1 sync_worker.go:868] Skipping precreation of serviceaccount "openshift-marketplace/marketplace-operator" (599 of 784): disabled capabilities: marketplace I0331 09:24:53.046325 1 sync_worker.go:860] Running sync for role "openshift-marketplace/marketplace-operator" (601 of 784) I0331 09:24:53.046364 1 sync_worker.go:868] Skipping precreation of role "openshift-marketplace/marketplace-operator" (601 of 784): disabled capabilities: marketplace I0331 09:24:53.046541 1 sync_worker.go:860] Running sync for rolebinding "openshift-marketplace/marketplace-operator" (604 of 784) I0331 09:24:53.046698 1 sync_worker.go:868] Skipping precreation of rolebinding "openshift-marketplace/marketplace-operator" (604 of 784): disabled capabilities: marketplace I0331 09:24:53.046732 1 sync_worker.go:860] Running sync for configmap "openshift-marketplace/marketplace-trusted-ca" (605 of 784) I0331 09:24:53.046766 1 sync_worker.go:868] Skipping precreation of configmap "openshift-marketplace/marketplace-trusted-ca" (605 of 784): disabled capabilities: marketplace I0331 09:24:53.046804 1 sync_worker.go:860] Running sync for service "openshift-marketplace/marketplace-operator-metrics" (606 of 784) I0331 09:24:53.103502 1 core.go:54] Service openshift-marketplace/marketplace-operator-metrics not found, creating E0331 09:24:55.072305 1 task.go:112] error running apply for service "openshift-marketplace/marketplace-operator-metrics" (606 of 784): namespaces "openshift-marketplace" not found I0331 09:24:59.415652 1 core.go:54] Service openshift-marketplace/marketplace-operator-metrics not found, creating E0331 09:25:00.542617 1 task.go:112] error running apply for service "openshift-marketplace/marketplace-operator-metrics" (606 of 784): namespaces "openshift-marketplace" not found I0331 09:25:03.542482 1 core.go:54] Service openshift-marketplace/marketplace-operator-metrics not found, creating E0331 09:25:04.322189 1 task.go:112] error running apply for service "openshift-marketplace/marketplace-operator-metrics" (606 of 784): namespaces "openshift-marketplace" not found I0331 09:25:08.593262 1 core.go:54] Service openshift-marketplace/marketplace-operator-metrics not found, creating E0331 09:25:08.919615 1 task.go:112] error running apply for service "openshift-marketplace/marketplace-operator-metrics" (606 of 784): namespaces "openshift-marketplace" not found I0331 09:25:11.891484 1 core.go:54] Service openshift-marketplace/marketplace-operator-metrics not found, creating E0331 09:25:12.001655 1 task.go:112] error running apply for service "openshift-marketplace/marketplace-operator-metrics" (606 of 784): namespaces "openshift-marketplace" not found ... I0331 10:15:16.711865 1 core.go:54] Service openshift-marketplace/marketplace-operator-metrics not found, creating E0331 10:15:16.768916 1 task.go:112] error running apply for service "openshift-marketplace/marketplace-operator-metrics" (606 of 784): namespaces "openshift-marketplace" not found I0331 10:15:19.980141 1 core.go:54] Service openshift-marketplace/marketplace-operator-metrics not found, creating E0331 10:15:20.037475 1 task.go:112] error running apply for service "openshift-marketplace/marketplace-operator-metrics" (606 of 784): namespaces "openshift-marketplace" not found I0331 10:15:21.007423 1 task_graph.go:546] Result of work: [Could not update service "openshift-marketplace/marketplace-operator-metrics" (606 of 784): resource may have been deleted] I0331 10:15:21.007455 1 sync_worker.go:1032] Update error 606 of 784: UpdatePayloadResourceNotFound Could not update service "openshift-marketplace/marketplace-operator-metrics" (606 of 784): resource may have been deleted (*errors.StatusError: namespaces "openshift-marketplace" not found) E0331 10:15:21.007489 1 sync_worker.go:555] unable to synchronize image (waiting 3m15.534447376s): Could not update service "openshift-marketplace/marketplace-operator-metrics" (606 of 784): resource may have been deleted [1] https://github.com/operator-framework/operator-marketplace/pull/462 [2] https://github.com/operator-framework/operator-marketplace/blob/master/manifests/08_service.yaml Version-Release number of selected component (if applicable): How reproducible: 1/1 Steps to Reproduce: 1. Install a cluster with below capabilities set in install-config.yaml capabilities: baselineCapabilitySet: None 2. 3. Actual results: Marketplace is disabled, but service "openshift-marketplace/marketplace-operator-metrics" is not disabled. Expected results: service "openshift-marketplace/marketplace-operator-metrics" is made optional Additional info: