Bug 1975456 - Uninstalling kube-descheduler clusterkubedescheduleroperator.4.6.0-202106010807.p0.git.5db84c5 removes some clusterrolebindings
Summary: Uninstalling kube-descheduler clusterkubedescheduleroperator.4.6.0-2021060108...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: OLM
Version: 4.6
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ---
: 4.6.z
Assignee: Vu Dinh
QA Contact: RamaKasturi
URL:
Whiteboard:
: 1974401 (view as bug list)
Depends On: 1975453
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-06-23 17:16 UTC by Vu Dinh
Modified: 2022-10-11 06:38 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-07-14 07:16:34 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github operator-framework operator-lifecycle-manager pull 2209 0 None open [release-4.6] Bug 1975456: Handle invalid label during resource cleanup 2021-06-23 18:21:53 UTC
Red Hat Knowledge Base (Solution) 6126771 0 None None None 2022-01-07 02:04:18 UTC
Red Hat Product Errata RHBA-2021:2641 0 None None None 2021-07-14 07:16:47 UTC

Description Vu Dinh 2021-06-23 17:16:55 UTC
This bug was initially created as a copy of Bug #1975453

I am copying this bug because: 



This bug was initially created as a copy of Bug #1974414

I am copying this bug because: 



This bug was initially created as a copy of Bug #1970910

I am copying this bug because: 



Description of problem:
Uninstalling kube-descheduler clusterkubedescheduleroperator.4.6.0-202106010807.p0.git.5db84c5 removes some clusterrolebindings causing the cluster to be unusable.

Version-Release number of selected component (if applicable):
clusterkubedescheduleroperator.4.6.0-202106010807.p0.git.5db84c5

How reproducible:
Always. 

Steps to Reproduce:
1. Create a fresh installation of OCP 4.6
2. oc create -f aio-cluster-kube-descheduler-operator.yaml
3. oc create -f kubedescheduler-cluster.yaml
4. check csv and rolebindings:
oc get clusterrolebinding -A | wc -l
oc get csv
NAME                                                               DISPLAY                     VERSION                             REPLACES   PHASE
clusterkubedescheduleroperator.4.6.0-202106010807.p0.git.5db84c5   Kube Descheduler Operator   4.6.0-202106010807.p0.git.5db84c5              Pending
5. oc delete csv clusterkubedescheduleroperator.4.6.0-202106010807.p0.git.5db84c5
6. Wait for OLM to remove clusterrolebindings
7. oc get clusterrolebinding -A | wc -l

Actual results:
Number of clusterrolebindings reduced severely


Expected results:
Just the clusterrolebindings of the namespace been removed

Additional info:
Adding yaml files mentioned in reproducer steps.

Comment 2 RamaKasturi 2021-06-24 08:42:33 UTC
Verified bug with the payload below and i see that CRB's were not deleted when kubedescheduler csv is deleted.

Reproduced in a env where fix is not present & below are the steps i followed to reproduce the issue:
1) Install 4.6 cluster with the payload which has the fix
2) create namespace called 'openshift-kube-descheduler-operator'
3) create operatorgroup using the yaml below
[knarra@knarra ~]$ cat /tmp/operatorgroup.yaml 
apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
  name: openshift-kube-descheduler-operator
  namespace: openshift-kube-descheduler-operator
spec:
  targetNamespaces:
    - openshift-kube-descheduler-operator
4) create catalogsource with index image using the yaml below
[knarra@knarra ~]$ cat /tmp/catalogsource.yaml 
apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: qe-app-registry
  namespace: openshift-kube-descheduler-operator
spec:
  sourceType: grpc
  image: docker.io/dinhxuanvu/descheduler-index:v1

5) create subscription using the yaml file below

[knarra@knarra ~]$ cat /tmp/subscription.yaml 
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: cluster-kube-descheduler-operator
  namespace: openshift-kube-descheduler-operator
spec:
  channel: stable
  name: cluster-kube-descheduler-operator
  source: qe-app-registry
  sourceNamespace: openshift-kube-descheduler-operator

Now you can see that csv is in pending state with error "one or more requirements couldn't be found"


Events:
  Type    Reason               Age   From                        Message
  ----    ------               ----  ----                        -------
  Normal  RequirementsUnknown  34s   operator-lifecycle-manager  requirements not yet checked
  Normal  RequirementsNotMet   34s   operator-lifecycle-manager  one or more requirements couldn't be found

[knarra@knarra openshift-client-linux-4.6.0-0.nightly-2021-06-24-012628]$ ./oc get clusterrolebinding -A | wc -l
187
[knarra@knarra openshift-client-linux-4.6.0-0.nightly-2021-06-24-012628]$ ./oc get csv
NAME                                                               DISPLAY                     VERSION                             REPLACES   PHASE
clusterkubedescheduleroperator.4.6.0-202106010807.p0.git.5db84c5   Kube Descheduler Operator   4.6.0-202106010807.p0.git.5db84c5              Pending
[knarra@knarra openshift-client-linux-4.6.0-0.nightly-2021-06-24-012628]$ ./oc delete csv clusterkubedescheduleroperator.4.6.0-202106010807.p0.git.5db84c5
clusterserviceversion.operators.coreos.com "clusterkubedescheduleroperator.4.6.0-202106010807.p0.git.5db84c5" deleted
[knarra@knarra openshift-client-linux-4.6.0-0.nightly-2021-06-24-012628]$ ./oc get clusterrolebinding -A | wc -l
92
[knarra@knarra openshift-client-linux-4.6.0-0.nightly-2021-06-24-012628]$ ./oc get clusterrolebinding -A | wc -l
81
[knarra@knarra openshift-client-linux-4.6.0-0.nightly-2021-06-24-012628]$ ./oc get clusterrolebinding -A | wc -l
81
[knarra@knarra openshift-client-linux-4.6.0-0.nightly-2021-06-24-012628]$ ./oc get clusterrolebinding -A | wc -l
81


Below are the steps i followed to verify the bug :
=================================================
1) Install 4.6 cluster with the payload which has the fix
2) create namespace called 'openshift-kube-descheduler-operator'
3) create operatorgroup using the yaml below
[knarra@knarra ~]$ cat /tmp/operatorgroup.yaml 
apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
  name: openshift-kube-descheduler-operator
  namespace: openshift-kube-descheduler-operator
spec:
  targetNamespaces:
    - openshift-kube-descheduler-operator
4) create catalogsource with index image using the yaml below
[knarra@knarra ~]$ cat /tmp/catalogsource.yaml 
apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: qe-app-registry
  namespace: openshift-kube-descheduler-operator
spec:
  sourceType: grpc
  image: docker.io/dinhxuanvu/descheduler-index:v1

5) create subscription using the yaml file below

[knarra@knarra ~]$ cat /tmp/subscription.yaml 
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: cluster-kube-descheduler-operator
  namespace: openshift-kube-descheduler-operator
spec:
  channel: stable
  name: cluster-kube-descheduler-operator
  source: qe-app-registry
  sourceNamespace: openshift-kube-descheduler-operator

Now you can see that csv is in pending state with error "one or more requirements couldn't be found"

Events:
  Type    Reason               Age                From                        Message
  ----    ------               ----               ----                        -------
  Normal  RequirementsUnknown  13s (x2 over 13s)  operator-lifecycle-manager  requirements not yet checked
  Normal  RequirementsNotMet   13s (x2 over 13s)  operator-lifecycle-manager  one or more requirements couldn't be found
[knarra@knarra openshift-client-linux-4.6.0-0.nightly-2021-06-24-012628]$ ./oc get clusterrolebinding -A | wc -l
187
[knarra@knarra openshift-client-linux-4.6.0-0.nightly-2021-06-24-012628]$ ./oc get csv
NAME                                                               DISPLAY                     VERSION                             REPLACES   PHASE
clusterkubedescheduleroperator.4.6.0-202106010807.p0.git.5db84c5   Kube Descheduler Operator   4.6.0-202106010807.p0.git.5db84c5              Pending
[knarra@knarra openshift-client-linux-4.6.0-0.nightly-2021-06-24-012628]$ ./oc delete csv clusterkubedescheduleroperator.4.6.0-202106010807.p0.git.5db84c5
clusterserviceversion.operators.coreos.com "clusterkubedescheduleroperator.4.6.0-202106010807.p0.git.5db84c5" deleted
[knarra@knarra openshift-client-linux-4.6.0-0.nightly-2021-06-24-012628]$ ./oc get csv
No resources found in openshift-kube-descheduler-operator namespace.
[knarra@knarra openshift-client-linux-4.6.0-0.nightly-2021-06-24-012628]$ ./oc get clusterrolebinding -A | wc -l
187
[knarra@knarra openshift-client-linux-4.6.0-0.nightly-2021-06-24-012628]$ ./oc get clusterrolebinding -A | wc -l
187

Based on the above moving bug to verified state.

Comment 8 errata-xmlrpc 2021-07-14 07:16:34 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6.38 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:2641

Comment 9 Per da Silva 2022-01-07 02:04:18 UTC
*** Bug 1974401 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.