Bug 1879975

Summary: [Descheduler] Descheduler cluster log shows permission issue message for any priorityclass given whether it exists or not.
Product: OpenShift Container Platform Reporter: RamaKasturi <knarra>
Component: kube-schedulerAssignee: Mike Dame <mdame>
Status: CLOSED ERRATA QA Contact: RamaKasturi <knarra>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 4.6CC: aos-bugs, mdame, mfojtik
Target Milestone: ---   
Target Release: 4.6.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-10-27 16:41:54 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description RamaKasturi 2020-09-17 13:37:25 UTC 
Description of problem:
According to the docs here https://github.com/kubernetes-sigs/descheduler#priority-filtering if the priority class does not exist it says descheduler does not get created and throw an error but i do not see that happening. what is happening is
1) Descheduler gets created
2) An Info message is logged in the descheduler logs which reads below
I0917 13:09:55.091198       1 duplicates.go:66] "Failed to get threshold priority from strategy's params" err="priorityclasses.scheduling.k8s.io \"priorityclass1\" is forbidden: User \"system:serviceaccount:openshift-kube-descheduler-operator:openshift-descheduler\" cannot get resource \"priorityclasses\" in API group \"scheduling.k8s.io\" at the cluster scope"
I0917 13:10:55.091325       1 node.go:45] node lister returned empty list, now fetch directly
3) Info message mostly talks about permission but does not talk anything about the class not existing

Version-Release number of selected component (if applicable):
[ramakasturinarra@dhcp35-60 verification-tests]$ oc get csv -n openshift-kube-descheduler-operator
NAME                                                   DISPLAY                     VERSION                 REPLACES   PHASE
clusterkubedescheduleroperator.4.6.0-202009170139.p0   Kube Descheduler Operator   4.6.0-202009170139.p0              Succeeded


How reproducible:
Always

Steps to Reproduce:
1. Install latest 4.6 cluster
2. Add a priorityclass which does not exist on the cluster
3. Now add strategies as below.
spec:
  deschedulingIntervalSeconds: 60
  image: registry.redhat.io/openshift4/ose-descheduler@sha256:f9fa973e56efe2c3ad55c23fe664c16a6432a8282549a2c6c186a024316a22ea
  strategies:
  - name: RemoveDuplicates
    params:
    - name: thresholdPriorityClassName
      value: priorityclass1


Actual results:
1) Descheduler gets created, as per docs it should not
2) Descheduler gives an info message in the cluster pod logs where as it should at least give an error message
3) Message shown in the logs does not indicate anything about non existing class, instead it gives permission error.

Expected results:
1) Either descheduler should not get created and throw error in the first place
2) Display an error message in the cluster pod logs saying class does not exist
3) And discussing in the chat it looks like permission error with descheduler service account / priority class in a different group either of which should be resolved.

Additional info:
Comment 1 RamaKasturi 2020-09-17 13:56:16 UTC 
permission issue is seen in the logs whether it exists or not.
Comment 3 RamaKasturi 2020-09-18 07:42:51 UTC 
Verified with the payload below and i see that permission error is not thrown any more for a priorityclass which exists or does not exist.

[ramakasturinarra@dhcp35-60 cucushift]$ oc get csv
NAME                                                   DISPLAY                     VERSION                 REPLACES   PHASE
clusterkubedescheduleroperator.4.6.0-202009172018.p0   Kube Descheduler Operator   4.6.0-202009172018.p0              Succeeded

[ramakasturinarra@dhcp35-60 cucushift]$ oc version
Client Version: 4.6.0-202009160247.p0-0d989c3
Server Version: 4.6.0-0.nightly-2020-09-18-002612
Kubernetes Version: v1.19.0+b4ffb45

When priorityclass is not present, cluster pod logs shows below error.

[ramakasturinarra@dhcp35-60 cucushift]$ oc logs -f cluster-74db674445-pc6kp
I0918 07:19:46.671949       1 node.go:45] node lister returned empty list, now fetch directly
I0918 07:19:46.763498       1 duplicates.go:66] "Failed to get threshold priority from strategy's params" err="priorityclasses.scheduling.k8s.io \"priorityclass1\" not found"


When specified a default priority class which exists in the system , descheduler does not show any error in the cluster pod logs.

When specified a user created priority class, descheduler does not show any error in the cluster pod logs.

I will discuss with dev and raise new bugs if required for the initial points in the description 1 & 2

Based on the above moving the bug to verified state.
Comment 6 errata-xmlrpc 2020-10-27 16:41:54 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4196