Bug 1975456

Summary:	Uninstalling kube-descheduler clusterkubedescheduleroperator.4.6.0-202106010807.p0.git.5db84c5 removes some clusterrolebindings
Product:	OpenShift Container Platform	Reporter:	Vu Dinh <vdinh>
Component:	OLM	Assignee:	Vu Dinh <vdinh>
OLM sub component:	OLM	QA Contact:	RamaKasturi <knarra>
Status:	CLOSED ERRATA	Docs Contact:
Severity:	urgent
Priority:	urgent	CC:	jiazha, rkshirsa, talessio
Version:	4.6	Keywords:	FastFix
Target Milestone:	---
Target Release:	4.6.z
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2021-07-14 07:16:34 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1975453
Bug Blocks:

Description Vu Dinh 2021-06-23 17:16:55 UTC

This bug was initially created as a copy of Bug #1975453

I am copying this bug because:

This bug was initially created as a copy of Bug #1974414

I am copying this bug because:

This bug was initially created as a copy of Bug #1970910

I am copying this bug because:

Description of problem:
Uninstalling kube-descheduler clusterkubedescheduleroperator.4.6.0-202106010807.p0.git.5db84c5 removes some clusterrolebindings causing the cluster to be unusable.

Version-Release number of selected component (if applicable):
clusterkubedescheduleroperator.4.6.0-202106010807.p0.git.5db84c5

How reproducible:
Always.

Steps to Reproduce:
1. Create a fresh installation of OCP 4.6
2. oc create -f aio-cluster-kube-descheduler-operator.yaml
3. oc create -f kubedescheduler-cluster.yaml
4. check csv and rolebindings:
oc get clusterrolebinding -A | wc -l
oc get csv
NAME DISPLAY VERSION REPLACES PHASE
clusterkubedescheduleroperator.4.6.0-202106010807.p0.git.5db84c5 Kube Descheduler Operator 4.6.0-202106010807.p0.git.5db84c5 Pending
5. oc delete csv clusterkubedescheduleroperator.4.6.0-202106010807.p0.git.5db84c5
6. Wait for OLM to remove clusterrolebindings
7. oc get clusterrolebinding -A | wc -l

Actual results:
Number of clusterrolebindings reduced severely

Expected results:
Just the clusterrolebindings of the namespace been removed

Additional info:
Adding yaml files mentioned in reproducer steps.

Comment 2 RamaKasturi 2021-06-24 08:42:33 UTC

Verified bug with the payload below and i see that CRB's were not deleted when kubedescheduler csv is deleted.

Reproduced in a env where fix is not present & below are the steps i followed to reproduce the issue:
1) Install 4.6 cluster with the payload which has the fix
2) create namespace called 'openshift-kube-descheduler-operator'
3) create operatorgroup using the yaml below
[knarra@knarra ~]$ cat /tmp/operatorgroup.yaml 
apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
  name: openshift-kube-descheduler-operator
  namespace: openshift-kube-descheduler-operator
spec:
  targetNamespaces:
    - openshift-kube-descheduler-operator
4) create catalogsource with index image using the yaml below
[knarra@knarra ~]$ cat /tmp/catalogsource.yaml 
apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: qe-app-registry
  namespace: openshift-kube-descheduler-operator
spec:
  sourceType: grpc
  image: docker.io/dinhxuanvu/descheduler-index:v1

5) create subscription using the yaml file below

[knarra@knarra ~]$ cat /tmp/subscription.yaml 
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: cluster-kube-descheduler-operator
  namespace: openshift-kube-descheduler-operator
spec:
  channel: stable
  name: cluster-kube-descheduler-operator
  source: qe-app-registry
  sourceNamespace: openshift-kube-descheduler-operator

Now you can see that csv is in pending state with error "one or more requirements couldn't be found"


Events:
  Type    Reason               Age   From                        Message
  ----    ------               ----  ----                        -------
  Normal  RequirementsUnknown  34s   operator-lifecycle-manager  requirements not yet checked
  Normal  RequirementsNotMet   34s   operator-lifecycle-manager  one or more requirements couldn't be found

[knarra@knarra openshift-client-linux-4.6.0-0.nightly-2021-06-24-012628]$ ./oc get clusterrolebinding -A | wc -l
187
[knarra@knarra openshift-client-linux-4.6.0-0.nightly-2021-06-24-012628]$ ./oc get csv
NAME                                                               DISPLAY                     VERSION                             REPLACES   PHASE
clusterkubedescheduleroperator.4.6.0-202106010807.p0.git.5db84c5   Kube Descheduler Operator   4.6.0-202106010807.p0.git.5db84c5              Pending
[knarra@knarra openshift-client-linux-4.6.0-0.nightly-2021-06-24-012628]$ ./oc delete csv clusterkubedescheduleroperator.4.6.0-202106010807.p0.git.5db84c5
clusterserviceversion.operators.coreos.com "clusterkubedescheduleroperator.4.6.0-202106010807.p0.git.5db84c5" deleted
[knarra@knarra openshift-client-linux-4.6.0-0.nightly-2021-06-24-012628]$ ./oc get clusterrolebinding -A | wc -l
92
[knarra@knarra openshift-client-linux-4.6.0-0.nightly-2021-06-24-012628]$ ./oc get clusterrolebinding -A | wc -l
81
[knarra@knarra openshift-client-linux-4.6.0-0.nightly-2021-06-24-012628]$ ./oc get clusterrolebinding -A | wc -l
81
[knarra@knarra openshift-client-linux-4.6.0-0.nightly-2021-06-24-012628]$ ./oc get clusterrolebinding -A | wc -l
81


Below are the steps i followed to verify the bug :
=================================================
1) Install 4.6 cluster with the payload which has the fix
2) create namespace called 'openshift-kube-descheduler-operator'
3) create operatorgroup using the yaml below
[knarra@knarra ~]$ cat /tmp/operatorgroup.yaml 
apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
  name: openshift-kube-descheduler-operator
  namespace: openshift-kube-descheduler-operator
spec:
  targetNamespaces:
    - openshift-kube-descheduler-operator
4) create catalogsource with index image using the yaml below
[knarra@knarra ~]$ cat /tmp/catalogsource.yaml 
apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: qe-app-registry
  namespace: openshift-kube-descheduler-operator
spec:
  sourceType: grpc
  image: docker.io/dinhxuanvu/descheduler-index:v1

5) create subscription using the yaml file below

[knarra@knarra ~]$ cat /tmp/subscription.yaml 
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: cluster-kube-descheduler-operator
  namespace: openshift-kube-descheduler-operator
spec:
  channel: stable
  name: cluster-kube-descheduler-operator
  source: qe-app-registry
  sourceNamespace: openshift-kube-descheduler-operator

Now you can see that csv is in pending state with error "one or more requirements couldn't be found"

Events:
  Type    Reason               Age                From                        Message
  ----    ------               ----               ----                        -------
  Normal  RequirementsUnknown  13s (x2 over 13s)  operator-lifecycle-manager  requirements not yet checked
  Normal  RequirementsNotMet   13s (x2 over 13s)  operator-lifecycle-manager  one or more requirements couldn't be found
[knarra@knarra openshift-client-linux-4.6.0-0.nightly-2021-06-24-012628]$ ./oc get clusterrolebinding -A | wc -l
187
[knarra@knarra openshift-client-linux-4.6.0-0.nightly-2021-06-24-012628]$ ./oc get csv
NAME                                                               DISPLAY                     VERSION                             REPLACES   PHASE
clusterkubedescheduleroperator.4.6.0-202106010807.p0.git.5db84c5   Kube Descheduler Operator   4.6.0-202106010807.p0.git.5db84c5              Pending
[knarra@knarra openshift-client-linux-4.6.0-0.nightly-2021-06-24-012628]$ ./oc delete csv clusterkubedescheduleroperator.4.6.0-202106010807.p0.git.5db84c5
clusterserviceversion.operators.coreos.com "clusterkubedescheduleroperator.4.6.0-202106010807.p0.git.5db84c5" deleted
[knarra@knarra openshift-client-linux-4.6.0-0.nightly-2021-06-24-012628]$ ./oc get csv
No resources found in openshift-kube-descheduler-operator namespace.
[knarra@knarra openshift-client-linux-4.6.0-0.nightly-2021-06-24-012628]$ ./oc get clusterrolebinding -A | wc -l
187
[knarra@knarra openshift-client-linux-4.6.0-0.nightly-2021-06-24-012628]$ ./oc get clusterrolebinding -A | wc -l
187

Based on the above moving bug to verified state.

Comment 8 errata-xmlrpc 2021-07-14 07:16:34 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6.38 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:2641

Comment 9 Per da Silva 2022-01-07 02:04:18 UTC

*** Bug 1974401 has been marked as a duplicate of this bug. ***