Bug 1975453

Summary: Uninstalling kube-descheduler clusterkubedescheduleroperator.4.6.0-202106010807.p0.git.5db84c5 removes some clusterrolebindings
Product: OpenShift Container Platform Reporter: Vu Dinh <vdinh>
Component: OLMAssignee: Vu Dinh <vdinh>
OLM sub component: OLM QA Contact: RamaKasturi <knarra>
Status: CLOSED ERRATA Docs Contact:
Severity: urgent    
Priority: urgent    
Version: 4.6   
Target Milestone: ---   
Target Release: 4.7.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-07-06 11:38:38 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On: 1974414    
Bug Blocks: 1975456    

Description Vu Dinh 2021-06-23 17:12:39 UTC
This bug was initially created as a copy of Bug #1974414

I am copying this bug because: 



This bug was initially created as a copy of Bug #1970910

I am copying this bug because: 



Description of problem:
Uninstalling kube-descheduler clusterkubedescheduleroperator.4.6.0-202106010807.p0.git.5db84c5 removes some clusterrolebindings causing the cluster to be unusable.

Version-Release number of selected component (if applicable):
clusterkubedescheduleroperator.4.6.0-202106010807.p0.git.5db84c5

How reproducible:
Always. 

Steps to Reproduce:
1. Create a fresh installation of OCP 4.6
2. oc create -f aio-cluster-kube-descheduler-operator.yaml
3. oc create -f kubedescheduler-cluster.yaml
4. check csv and rolebindings:
oc get clusterrolebinding -A | wc -l
oc get csv
NAME                                                               DISPLAY                     VERSION                             REPLACES   PHASE
clusterkubedescheduleroperator.4.6.0-202106010807.p0.git.5db84c5   Kube Descheduler Operator   4.6.0-202106010807.p0.git.5db84c5              Pending
5. oc delete csv clusterkubedescheduleroperator.4.6.0-202106010807.p0.git.5db84c5
6. Wait for OLM to remove clusterrolebindings
7. oc get clusterrolebinding -A | wc -l

Actual results:
Number of clusterrolebindings reduced severely


Expected results:
Just the clusterrolebindings of the namespace been removed

Additional info:
Adding yaml files mentioned in reproducer steps.

Comment 1 RamaKasturi 2021-06-23 18:19:55 UTC
Verified with 4.7 cluster and did not see any CRB's getting deleted after the descheduler csv which is in pending state is deleted

[knarra@knarra openshift-client-linux-4.7.0-0.nightly-2021-06-23-064238]$ ./oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.7.0-0.nightly-2021-06-23-064238   True        False         19m     Cluster version is 4.7.0-0.nightly-2021-06-23-064238

[knarra@knarra openshift-client-linux-4.7.0-0.nightly-2021-06-23-064238]$ ./oc version -o yaml
clientVersion:
  buildDate: "2021-06-12T02:01:17Z"
  compiler: gc
  gitCommit: 9b9f77a716c8b7ee73f6424e37748ea6b53f495a
  gitTreeState: clean
  gitVersion: 4.7.0-202106120124.p0.git.9b9f77a-9b9f77a
  goVersion: go1.15.7
  major: ""
  minor: ""
  platform: linux/amd64
openshiftVersion: 4.7.0-0.nightly-2021-06-23-064238
releaseClientVersion: 4.7.0-0.nightly-2021-06-23-064238
serverVersion:
  buildDate: "2021-06-19T03:11:54Z"
  compiler: gc
  gitCommit: 87cc9a4ade7ebee64def2489b78c9d8ae5d6fdf9
  gitTreeState: clean
  gitVersion: v1.20.0+87cc9a4
  goVersion: go1.15.7
  major: "1"
  minor: "20"
  platform: linux/amd64


Below are the steps i followed to verify the bug:
================================================
1) Install latest 4.8 cluster
2) create namespace called 'openshift-kube-descheduler-operator'
3) create operatorgroup using the yaml below
[knarra@knarra ~]$ cat /tmp/operatorgroup.yaml 
apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
  name: openshift-kube-descheduler-operator
  namespace: openshift-kube-descheduler-operator
spec:
  targetNamespaces:
    - openshift-kube-descheduler-operator
4) create catalogsource with index image using the yaml below
[knarra@knarra ~]$ cat /tmp/catalogsource.yaml 
apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: qe-app-registry
  namespace: openshift-kube-descheduler-operator
spec:
  sourceType: grpc
  image: docker.io/dinhxuanvu/descheduler-index:v1

5) create subscription using the yaml file below

[knarra@knarra ~]$ cat /tmp/subscription.yaml 
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: cluster-kube-descheduler-operator
  namespace: openshift-kube-descheduler-operator
spec:
  channel: stable
  name: cluster-kube-descheduler-operator
  source: qe-app-registry
  sourceNamespace: openshift-kube-descheduler-operator

Now you can see that csv is in pending state with error "one or more requirements could not be found"

Events:
  Type    Reason               Age                From                        Message
  ----    ------               ----               ----                        -------
  Normal  RequirementsUnknown  14s (x2 over 14s)  operator-lifecycle-manager  requirements not yet checked
  Normal  RequirementsNotMet   14s (x2 over 14s)  operator-lifecycle-manager  one or more requirements couldn't be found

[knarra@knarra openshift-client-linux-4.7.0-0.nightly-2021-06-23-064238]$ oc get clusterrolebinding -A | wc -l
194
[knarra@knarra openshift-client-linux-4.7.0-0.nightly-2021-06-23-064238]$ ./oc get csv
NAME                                                               DISPLAY                     VERSION                             REPLACES   PHASE
clusterkubedescheduleroperator.4.6.0-202106010807.p0.git.5db84c5   Kube Descheduler Operator   4.6.0-202106010807.p0.git.5db84c5              Pending
[knarra@knarra openshift-client-linux-4.7.0-0.nightly-2021-06-23-064238]$ ./oc delete csv clusterkubedescheduleroperator.4.6.0-202106010807.p0.git.5db84c5
clusterserviceversion.operators.coreos.com "clusterkubedescheduleroperator.4.6.0-202106010807.p0.git.5db84c5" deleted
[knarra@knarra openshift-client-linux-4.7.0-0.nightly-2021-06-23-064238]$ ./oc get clusterrolebinding -A | wc -l
194

[knarra@knarra openshift-client-linux-4.7.0-0.nightly-2021-06-23-064238]$ ./oc get clusterrolebinding -A | wc -l
194

Based on the above moving bug to verified state.

Comment 4 errata-xmlrpc 2021-07-06 11:38:38 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.7.19 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:2554