Bug 2276135 - ocs-operator should not be annotating all openshift-* namespaces
Summary: ocs-operator should not be annotating all openshift-* namespaces
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenShift Data Foundation
Classification: Red Hat Storage
Component: ocs-operator
Version: 4.14
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: ODF 4.16.0
Assignee: Nikhil Ladha
QA Contact: Yuli Persky
URL:
Whiteboard:
Depends On:
Blocks: 2278568 2278641 2278642
TreeView+ depends on / blocked
 
Reported: 2024-04-19 19:17 UTC by Martin Bukatovic
Modified: 2024-07-17 13:20 UTC (History)
5 users (show)

Fixed In Version: 4.16.0-92
Doc Type: No Doc Update
Doc Text:
Clone Of:
: 2278568 2278641 2278642 (view as bug list)
Environment:
Last Closed: 2024-07-17 13:20:03 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github red-hat-storage ocs-operator pull 2577 0 None open skip update events for namespace controller 2024-04-22 08:14:28 UTC
Github red-hat-storage ocs-operator pull 2593 0 None open Bug 2276135: [release-4.16] remove namespace controller 2024-05-02 07:54:31 UTC
Red Hat Product Errata RHSA-2024:4591 0 None None None 2024-07-17 13:20:21 UTC

Description Martin Bukatovic 2024-04-19 19:17:17 UTC
Description of problem
======================

The ocs-operator should not add `reclaimspace.csiaddons.openshift.io/schedule`
annotation to all openshift-* namespaces, because it could get into conflict
with another operator which doesn't expect this annotation to be there.

One such conflict arises when ODF is installed on ARO cluster: ARO operator is
trying to get rid of this annotation on ARO namespaces (openshift-azure-loggin
or openshift-azure), so that it effectively prevents ODF to be installed on ARO.

The change was introduced via
https://github.com/red-hat-storage/ocs-operator/pull/2074

The annotation itself seems to control ODF "reclaim space operation"[1], so
there is no reason to label all openshift namespaces in the first place.

[1] https://access.redhat.com/documentation/en-us/red_hat_openshift_data_foundation/4.10/html/managing_and_allocating_storage_resources/reclaiming-space-on-target-volumes_rhodf

Version-Release number of selected component
============================================

ODF 4.14

How reproducible
================

100%

Steps to Reproduce
==================

1. Install ODF on ARO 4.14 cluster

Actual results
==============

ARO Operator is removing `reclaimspace.csiaddons.openshift.io/schedule`
annotation from ARO namespaces.

Example from aro-operator-master log:

```
time=2024-04-19 16:43:10.000000 level=info msg=""
Update Namespace/openshift-azure-logging: u00a0u00a0&v1.Namespace{
    u00a0u00a0 TypeMeta: {},
    u00a0u00a0 ObjectMeta: v1.ObjectMeta{
        u00a0u00a0             ... // 9 identical fields
        u00a0u00a0             DeletionGracePeriodSeconds: nil,
        u00a0u00a0             Labels:                     {
            "kubernetes.io/metadata.name": "openshift-azure-logging",
            "pod-security.kubernetes.io/audit": "privileged",
            "pod-security.kubernetes.io/enforce": "privileged",
            "pod-security.kubernetes.io/warn": "privileged"
        },
        u00a0u00a0             Annotations: map[string]string{
            u00a0u00a0                 ... // 2 identical entries
            u00a0u00a0                 "openshift.io/sa.scc.supplemental-groups":	"1000680000/10000",
            u00a0u00a0                 "openshift.io/sa.scc.uid-range":                "1000680000/10000",
            -u00a0                     "reclaimspace.csiaddons.openshift.io/schedule": "@weekly",
            u00a0u00a0
        },
        u00a0u00a0             OwnerReferences: {
                {
          	 APIVersion: "aro.openshift.io/v1alpha1",
                Kind: "Cluster",
                Name: "cluster",	
                UID: "1b821b4f-2dcc-4ae7-ab57-452df1ad678b",                     	
                ...
            }
        },
        u00a0u00a0             Finalizers:	 nil,
        u00a0u00a0             ManagedFields:   {
                {
                Manager: "Go-http-client",
                Operation: "Update",
                APIVersion: "v1",
                Time: s"2024-04-19 08:41:38 +0000 UTC",
                ...
            },
          	 {
                Manager: "cluster-policy-controller",
                Operation: "Update",
                APIVersion: "v1",
                Time: s"2024-04-19 08:41:38 +0000 UTC",
                ...
            },
                {
                Manager: "ocs-operator",
                Operation: "Update",
                APIVersion: "v1",
                Time: s"2024-04-19 16:43:10 +0000 UTC",
                ...
            }
        },	
        u00a0u00a0
    },
    u00a0u00a0 Spec:   {
        Finalizers: {"kubernetes"}},
    u00a0u00a0 Status: {Phase: "Active"},
    u00a0u00a0
}       
```

Expected results
================

ODF doesn't annotate all openshift namespaces, but only those which needs that annotation only.

Comment 7 Martin Bukatovic 2024-04-22 07:39:11 UTC
ARO namespaces are:

- openshift-azure-logging
- openshift-azure-operator

That said I need to point out that the design choice selected in RHSTOR-4468 is the root cause here.

The proper solution is to add the annotation *only* to namespaces which are using ODF backed rbd storage.

Comment 13 Yuli Persky 2024-06-03 07:05:51 UTC
A clarification - This BZ is Azure specific bug. It was not seen on any of the other platform. 

For verification : the following command should be run on some ( no need to test all) of the openshift-* namespaces: 

oc get ns openshift-storage -o yaml | grep "reclaimspace"

same BZ for 4.14 : https://bugzilla.redhat.com/show_bug.cgi?id=2278642

same BZ for 4.15.3 : https://bugzilla.redhat.com/show_bug.cgi?id=2278641


In order to verify this BZ - an ARO cluster should be deployed and the verification should be performed on the ARO cluster.

Comment 14 Madhu Rajanna 2024-06-03 07:39:06 UTC
Hi Yuli,
Yes the problem was seen in ARO but for verification, we don't need ARO, we just need to ensure that we are not annotating the namespaces anymore in any clusters.

Comment 16 errata-xmlrpc 2024-07-17 13:20:03 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Red Hat OpenShift Data Foundation 4.16.0 security, enhancement & bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2024:4591


Note You need to log in before you can comment on or make changes to this bug.