Bug 2283653

Summary: Uninstalling client operator leaves behind certain resources
Product: [Red Hat Storage] Red Hat OpenShift Data Foundation Reporter: Jilju Joy <jijoy>
Component: ocs-client-operatorAssignee: Leela Venkaiah Gangavarapu <lgangava>
Status: CLOSED NOTABUG QA Contact: Neha Berry <nberry>
Severity: high Docs Contact:
Priority: unspecified    
Version: 4.16CC: lgangava, muagarwa, odf-bz-bot, omitrani
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: isf-provider
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2024-09-13 03:14:43 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jilju Joy 2024-05-28 16:00:36 UTC 
Description of problem:
Following https://bugzilla.redhat.com/show_bug.cgi?id=2251308#c19 , I tried to uninstall client operator. Subs, CSVs deleted. Deployments, daemonsets, csiaddonsnodes are removed. But certain resources including CRDs are not deleted.


% oc get operators
NAME                                          AGE
ocs-client-operator.openshift-operators       5d1h
odf-csi-addons-operator.openshift-operators   5d1h


% oc get all
Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
NAME                                  TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
service/ocs-client-operator-console   ClusterIP   172.31.8.212   <none>        9001/TCP   4d21h


% oc get service ocs-client-operator-console -o yaml
apiVersion: v1
kind: Service
metadata:
  annotations:
    service.alpha.openshift.io/serving-cert-secret-name: ocs-client-operator-console-serving-cert
  creationTimestamp: "2024-05-23T18:30:46Z"
  labels:
    app.kubernetes.io/name: ocs-client-operator-console
  name: ocs-client-operator-console
  namespace: openshift-operators
  resourceVersion: "68607"
  uid: 33ef8e9f-af46-470f-b671-c2dd9f761d09
spec:
  clusterIP: 172.31.8.212
  clusterIPs:
  - 172.31.8.212
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - name: console-port
    port: 9001
    protocol: TCP
    targetPort: 9001
  selector:
    app.kubernetes.io/name: ocs-client-operator-console
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}


% oc get crd | grep -e storageclaim -e storageclient -e reclaimspace
reclaimspacecronjobs.csiaddons.openshift.io                       2024-05-23T14:19:58Z
reclaimspacejobs.csiaddons.openshift.io                           2024-05-23T14:19:59Z
storageclaims.ocs.openshift.io                                    2024-05-23T14:20:01Z
storageclients.ocs.openshift.io                                   2024-05-23T14:20:01Z

Able to delete service ocs-client-operator-console manually.

% oc delete service ocs-client-operator-console
service "ocs-client-operator-console" deleted

% oc get service ocs-client-operator-console
Error from server (NotFound): services "ocs-client-operator-console" not found

Manually deleting the operators did not work. Those were created again.

% oc delete operator ocs-client-operator.openshift-operators odf-csi-addons-operator.openshift-operators
operator.operators.coreos.com "ocs-client-operator.openshift-operators" deleted
operator.operators.coreos.com "odf-csi-addons-operator.openshift-operators" deleted


% oc get operator                                                                                       
NAME                                          AGE
ocs-client-operator.openshift-operators       4s
odf-csi-addons-operator.openshift-operators   4s

% oc get operators -o yaml
apiVersion: v1
items:
- apiVersion: operators.coreos.com/v1
  kind: Operator
  metadata:
    creationTimestamp: "2024-05-28T15:56:37Z"
    generation: 1
    name: ocs-client-operator.openshift-operators
    resourceVersion: "2094063"
    uid: 79f93eaa-b209-42f7-a303-4a555c771b16
  spec: {}
  status:
    components:
      labelSelector:
        matchExpressions:
        - key: operators.coreos.com/ocs-client-operator.openshift-operators
          operator: Exists
      refs:
      - apiVersion: apiextensions.k8s.io/v1
        conditions:
        - lastTransitionTime: "2024-05-23T14:20:01Z"
          message: no conflicts found
          reason: NoConflicts
          status: "True"
          type: NamesAccepted
        - lastTransitionTime: "2024-05-23T14:20:01Z"
          message: the initial names have been accepted
          reason: InitialNamesAccepted
          status: "True"
          type: Established
        kind: CustomResourceDefinition
        name: storageclaims.ocs.openshift.io
      - apiVersion: apiextensions.k8s.io/v1
        conditions:
        - lastTransitionTime: "2024-05-23T14:20:01Z"
          message: no conflicts found
          reason: NoConflicts
          status: "True"
          type: NamesAccepted
        - lastTransitionTime: "2024-05-23T14:20:01Z"
          message: the initial names have been accepted
          reason: InitialNamesAccepted
          status: "True"
          type: Established
        kind: CustomResourceDefinition
        name: storageclients.ocs.openshift.io
- apiVersion: operators.coreos.com/v1
  kind: Operator
  metadata:
    creationTimestamp: "2024-05-28T15:56:38Z"
    generation: 1
    name: odf-csi-addons-operator.openshift-operators
    resourceVersion: "2094066"
    uid: 5d9835f0-5bf4-4966-8c00-f67d850349d7
  spec: {}
  status:
    components:
      labelSelector:
        matchExpressions:
        - key: operators.coreos.com/odf-csi-addons-operator.openshift-operators
          operator: Exists
      refs:
      - apiVersion: apiextensions.k8s.io/v1
        conditions:
        - lastTransitionTime: "2024-05-23T14:19:58Z"
          message: no conflicts found
          reason: NoConflicts
          status: "True"
          type: NamesAccepted
        - lastTransitionTime: "2024-05-23T14:19:58Z"
          message: the initial names have been accepted
          reason: InitialNamesAccepted
          status: "True"
          type: Established
        kind: CustomResourceDefinition
        name: csiaddonsnodes.csiaddons.openshift.io
      - apiVersion: apiextensions.k8s.io/v1
        conditions:
        - lastTransitionTime: "2024-05-23T14:19:58Z"
          message: no conflicts found
          reason: NoConflicts
          status: "True"
          type: NamesAccepted
        - lastTransitionTime: "2024-05-23T14:19:58Z"
          message: the initial names have been accepted
          reason: InitialNamesAccepted
          status: "True"
          type: Established
        kind: CustomResourceDefinition
        name: networkfences.csiaddons.openshift.io
      - apiVersion: apiextensions.k8s.io/v1
        conditions:
        - lastTransitionTime: "2024-05-23T14:19:58Z"
          message: no conflicts found
          reason: NoConflicts
          status: "True"
          type: NamesAccepted
        - lastTransitionTime: "2024-05-23T14:19:58Z"
          message: the initial names have been accepted
          reason: InitialNamesAccepted
          status: "True"
          type: Established
        kind: CustomResourceDefinition
        name: reclaimspacecronjobs.csiaddons.openshift.io
      - apiVersion: apiextensions.k8s.io/v1
        conditions:
        - lastTransitionTime: "2024-05-23T14:19:59Z"
          message: no conflicts found
          reason: NoConflicts
          status: "True"
          type: NamesAccepted
        - lastTransitionTime: "2024-05-23T14:19:59Z"
          message: the initial names have been accepted
          reason: InitialNamesAccepted
          status: "True"
          type: Established
        kind: CustomResourceDefinition
        name: reclaimspacejobs.csiaddons.openshift.io
      - apiVersion: apiextensions.k8s.io/v1
        conditions:
        - lastTransitionTime: "2024-05-23T14:19:58Z"
          message: no conflicts found
          reason: NoConflicts
          status: "True"
          type: NamesAccepted
        - lastTransitionTime: "2024-05-23T14:19:58Z"
          message: the initial names have been accepted
          reason: InitialNamesAccepted
          status: "True"
          type: Established
        kind: CustomResourceDefinition
        name: volumereplicationclasses.replication.storage.openshift.io
      - apiVersion: apiextensions.k8s.io/v1
        conditions:
        - lastTransitionTime: "2024-05-23T14:19:58Z"
          message: no conflicts found
          reason: NoConflicts
          status: "True"
          type: NamesAccepted
        - lastTransitionTime: "2024-05-23T14:19:58Z"
          message: the initial names have been accepted
          reason: InitialNamesAccepted
          status: "True"
          type: Established
        kind: CustomResourceDefinition
        name: volumereplications.replication.storage.openshift.io
kind: List
metadata:
  resourceVersion: ""



=========================================

Version-Release number of selected component (if applicable):
% oc get csv
NAME                                         DISPLAY                            VERSION             REPLACES   PHASE
ocs-client-operator.v4.16.0-101.stable       OpenShift Data Foundation Client   4.16.0-101.stable              Succeeded
odf-csi-addons-operator.v4.16.0-101.stable   CSI Addons                         4.16.0-101.stable              Succeeded

% oc get clusterversion
NAME      VERSION       AVAILABLE   PROGRESSING   SINCE   STATUS

===========================================

How reproducible:
2/2

Steps to Reproduce:
Follow the steps from https://bugzilla.redhat.com/show_bug.cgi?id=2251308#c19
============================================
Actual results:
Some resources are not deleted after uninstalling client from CLI.

Expected results:
All resources created by client operator should be removed.
============================================
Additional info:
Openshift data Foundation Client operator was installed from UI selecting the default "all namespaces" installation mode.
Comment 3 Leela Venkaiah Gangavarapu 2024-05-28 16:06:18 UTC 
I don't believe this BZ is a blocker for current release and can be taken in next patch cycle. Please reasses once.