Bug 1989728

Summary:	Descheduler operator should verify config does not conflict with scheduler
Product:	OpenShift Container Platform	Reporter:	Mike Dame <mdame>
Component:	kube-scheduler	Assignee:	Mike Dame <mdame>
Status:	CLOSED ERRATA	QA Contact:	RamaKasturi <knarra>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	4.8	CC:	aos-bugs, knarra, mfojtik
Target Milestone:	---
Target Release:	4.9.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2021-10-18 17:44:27 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Mike Dame 2021-08-03 19:43:19 UTC

There is currently no automatic verification that user configuration for the Descheduler semantically matches what is configured in the Scheduler (and vice-versa).

For example, if a user enables the `HighNodeUtilization` (bin-packing) Profile in the scheduler operator and the `LifecycleAndUtilization` Profile in the Descheduler, evictions will be attempting to balance resource usage in contrary to the scheduler's attempts at bin packing leading to a constant loop of eviction and scheduling.

 

The 2 operators should warn in some way if conflicting settings have been enabled. This could be included as the scheduler's dry-run simulation mode.

Comment 2 RamaKasturi 2021-08-09 16:06:10 UTC

currently blocked on this due to bug https://bugzilla.redhat.com/show_bug.cgi?id=1990603

Comment 3 RamaKasturi 2021-08-14 16:33:14 UTC

Tested using the below procedure & build and i see that it works fine but an improvement is needed and will log another bug for the same.

[knarra@knarra openshift-client-linux-4.9.0-0.nightly-2021-08-14-044521]$ ./oc get csv -n openshift-kube-descheduler-operator
NAME                                                DISPLAY                     VERSION              REPLACES   PHASE
clusterkubedescheduleroperator.4.9.0-202108130204   Kube Descheduler Operator   4.9.0-202108130204              Succeeded

[knarra@knarra openshift-client-linux-4.9.0-0.nightly-2021-08-14-044521]$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.9.0-0.nightly-2021-08-14-044521   True        False         5h22m   Cluster version is 4.9.0-0.nightly-2021-08-14-044521


1) Install latest 4.9 cluster & descheduler operator
2) Enable LifeCycleAndUtilization profile in descheduler
3) Enable HighNodeUtilization profile in kube-scheduler by running the command './oc patch Scheduler cluster --type='json' -p='[{"op": "add", "path": "/spec/profile", "value":"HighNodeUtilization"}]'
4) Descheduler cluster object disappears from cli and in UI it shows status as "Condition: TargetConfigControllerDegraded", but i would expect the same status to be shown via cli as well instead of pod disappearing, will raise  a bug for the same.
5) In the descheduler operator contains below log:

E0814 15:44:31.011369       1 target_config_reconciler.go:124] "Error managing targetConfig" err="enabling Descheduler LowNodeUtilization with Scheduler HighNodeUtilization may cause an eviction/scheduling hot loop"
E0814 15:44:31.035149       1 target_config_reconciler.go:124] "Error managing targetConfig" err="enabling Descheduler LowNodeUtilization with Scheduler HighNodeUtilization may cause an eviction/scheduling hot loop"
E0814 15:51:46.814930       1 target_config_reconciler.go:124] "Error managing targetConfig" err="enabling Descheduler LowNodeUtilization with Scheduler HighNodeUtilization may cause an eviction/scheduling hot loop"
E0814 15:51:46.830883       1 target_config_reconciler.go:124] "Error managing targetConfig" err="enabling Descheduler LowNodeUtilization with Scheduler HighNodeUtilization may cause an eviction/scheduling hot loop"

Based on the above moving the bug to verified state.

Comment 6 errata-xmlrpc 2021-10-18 17:44:27 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.9.0 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:3759