Created attachment 1870818 [details] provider-add-on install Description of problem: =========================== Recently the name of the provider qe addon was changed. Since then, facing issues with deployment of add-on : 1. smtp secret uses wrong name and toolbox pod fails to come up Chat details - https://chat.google.com/room/AAAASHA9vWs/lvl_LKqt2LA 2022-04-05T06:40:33.144Z ERROR controllers.ManagedOCS An error was encountered during reconcilePhases {"error": "Unable to get SMTP secret: Secret \"ocs-provider-qe-smtp\" not found"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler /tmp/go/ocs-osd-deployer/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:298 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem /tmp/go/ocs-osd-deployer/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:253 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2 /tmp/go/ocs-osd-deployer/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:214 2022-04-05T06:40:33.150Z ERROR controller-runtime.manager.controller.managedocs Reconciler error {"reconciler group": "ocs.openshift.io", "reconciler kind": "ManagedOCS", "name": "managedocs", "namespace": "openshift-storage", "error": "Unable to get SMTP secret: Secret \"ocs-provider-qe-smtp\" not found"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem /tmp/go/ocs-osd-deployer/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:253 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2 /tmp/go/ocs-osd-deployer/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:214 oc get secret|grep smtp ocs-converged-qe-smtp Opaque 5 23m Version-Release number of selected component (if applicable): ============================================================= oc describe csv ocs-osd-deployer.v2.0.0 |grep -i image Mediatype: image/svg+xml Image: quay.io/openshift/origin-kube-rbac-proxy:4.10.0 Image: quay.io/osd-addons/ocs-osd-deployer:2.0.0-8 Image: quay.io/osd-addons/ocs-osd-deployer:2.0.0-8 ➜ p oc get csv -n openshift-storage -o json ocs-operator.v4.10.0 | jq '.metadata.labels["full_version"]' "4.10.0-219" ➜ p oc describe catsrc redhat-operators |grep -i image f:image: Image: quay.io/rhceph-dev/odf4-odf-compose-operator-bundle-index:4.10.0-219 How reproducible: ===================== Always Steps to Reproduce: =================== 1. Install ROSA cluster - provider Openshift cluster 2. Add the security group for ceph pods 3. Install ocs-provider-qe add-on from OCM UI Actual results: ================= Even though cephcluster is in Health_OK, the add-on stays in Installing state for long and the manager pod shows smtp secret related issues Expected results: ================== No errors in deployment Additional info: =====================
The issue was that the SMTP service that integrates with OCM uses the Addon name and not the Addon ID. We have recently changed the addon names as per the official documentation which made the Addon name of converged offering similar to the addon name for provider addon. This caused the issue and the secret ocs-converged-qe-smtp was getting created for the provider-qe addon.
As the addon name and SMTP service config was updated, the changes needs to be tested on a new cluster. I verified changes on the new cluster and everything seems to be working fine.