Description of problem ====================== The ocs-operator should not add `reclaimspace.csiaddons.openshift.io/schedule` annotation to all openshift-* namespaces, because it could get into conflict with another operator which doesn't expect this annotation to be there. One such conflict arises when ODF is installed on ARO cluster: ARO operator is trying to get rid of this annotation on ARO namespaces (openshift-azure-loggin or openshift-azure), so that it effectively prevents ODF to be installed on ARO. The change was introduced via https://github.com/red-hat-storage/ocs-operator/pull/2074 The annotation itself seems to control ODF "reclaim space operation"[1], so there is no reason to label all openshift namespaces in the first place. [1] https://access.redhat.com/documentation/en-us/red_hat_openshift_data_foundation/4.10/html/managing_and_allocating_storage_resources/reclaiming-space-on-target-volumes_rhodf Version-Release number of selected component ============================================ ODF 4.14 How reproducible ================ 100% Steps to Reproduce ================== 1. Install ODF on ARO 4.14 cluster Actual results ============== ARO Operator is removing `reclaimspace.csiaddons.openshift.io/schedule` annotation from ARO namespaces. Example from aro-operator-master log: ``` time=2024-04-19 16:43:10.000000 level=info msg="" Update Namespace/openshift-azure-logging: u00a0u00a0&v1.Namespace{ u00a0u00a0 TypeMeta: {}, u00a0u00a0 ObjectMeta: v1.ObjectMeta{ u00a0u00a0 ... // 9 identical fields u00a0u00a0 DeletionGracePeriodSeconds: nil, u00a0u00a0 Labels: { "kubernetes.io/metadata.name": "openshift-azure-logging", "pod-security.kubernetes.io/audit": "privileged", "pod-security.kubernetes.io/enforce": "privileged", "pod-security.kubernetes.io/warn": "privileged" }, u00a0u00a0 Annotations: map[string]string{ u00a0u00a0 ... // 2 identical entries u00a0u00a0 "openshift.io/sa.scc.supplemental-groups": "1000680000/10000", u00a0u00a0 "openshift.io/sa.scc.uid-range": "1000680000/10000", -u00a0 "reclaimspace.csiaddons.openshift.io/schedule": "@weekly", u00a0u00a0 }, u00a0u00a0 OwnerReferences: { { APIVersion: "aro.openshift.io/v1alpha1", Kind: "Cluster", Name: "cluster", UID: "1b821b4f-2dcc-4ae7-ab57-452df1ad678b", ... } }, u00a0u00a0 Finalizers: nil, u00a0u00a0 ManagedFields: { { Manager: "Go-http-client", Operation: "Update", APIVersion: "v1", Time: s"2024-04-19 08:41:38 +0000 UTC", ... }, { Manager: "cluster-policy-controller", Operation: "Update", APIVersion: "v1", Time: s"2024-04-19 08:41:38 +0000 UTC", ... }, { Manager: "ocs-operator", Operation: "Update", APIVersion: "v1", Time: s"2024-04-19 16:43:10 +0000 UTC", ... } }, u00a0u00a0 }, u00a0u00a0 Spec: { Finalizers: {"kubernetes"}}, u00a0u00a0 Status: {Phase: "Active"}, u00a0u00a0 } ``` Expected results ================ ODF doesn't annotate all openshift namespaces, but only those which needs that annotation only.
ARO namespaces are: - openshift-azure-logging - openshift-azure-operator That said I need to point out that the design choice selected in RHSTOR-4468 is the root cause here. The proper solution is to add the annotation *only* to namespaces which are using ODF backed rbd storage.
A clarification - This BZ is Azure specific bug. It was not seen on any of the other platform. For verification : the following command should be run on some ( no need to test all) of the openshift-* namespaces: oc get ns openshift-storage -o yaml | grep "reclaimspace" same BZ for 4.14 : https://bugzilla.redhat.com/show_bug.cgi?id=2278642 same BZ for 4.15.3 : https://bugzilla.redhat.com/show_bug.cgi?id=2278641 In order to verify this BZ - an ARO cluster should be deployed and the verification should be performed on the ARO cluster.
Hi Yuli, Yes the problem was seen in ARO but for verification, we don't need ARO, we just need to ensure that we are not annotating the namespaces anymore in any clusters.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Important: Red Hat OpenShift Data Foundation 4.16.0 security, enhancement & bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2024:4591