Bug 1729918 - Configure new Scheduler Policy should take effect
Summary: Configure new Scheduler Policy should take effect
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: kube-scheduler
Version: 4.1.z
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: 4.1.z
Assignee: Mike Dame
QA Contact: ge liu
URL:
Whiteboard:
Depends On: 1734486
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-07-15 10:24 UTC by scheng
Modified: 2019-09-25 07:28 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1734486 (view as bug list)
Environment:
Last Closed: 2019-09-25 07:27:53 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift cluster-kube-scheduler-operator pull 158 0 None closed [release-4.1] Bug 1729918: Switch to configobserver pattern 2020-12-18 02:28:36 UTC
Red Hat Product Errata RHBA-2019:2820 0 None None None 2019-09-25 07:28:02 UTC

Comment 1 Sunil Choudhary 2019-07-15 17:27:08 UTC
To add more to first comment. The scheduler policy does gets applied and all 3 openshift-kube-scheduler pods restarts. 
However the predicates and priorities from policy does not get registered after scheduler pod restart.

$ oc version --short
Client Version: v4.1.0-201905191700+7bd2e5b-dirty
Server Version: v1.13.4+6569b4f

$ oc get clusterversion
NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.1.6     True        False         82m     Cluster version is 4.1.6

$ oc describe cm scheduler-policy -n openshift-config
Name:         scheduler-policy
Namespace:    openshift-config
Labels:       <none>
Annotations:  <none>

Data
====
policy.cfg:
----
{
"kind" : "Policy",
"apiVersion" : "v1",
"predicates" : [      
        {"name" : "PodFitsHostPorts"},
        {"name" : "PodFitsResources"},
        {"name" : "NoDiskConflict"},
        {"name" : "NoVolumeZoneConflict"},
        {"name" : "MatchNodeSelector"},
        {"name" : "MaxEBSVolumeCount"},
        {"name" : "MaxAzureDiskVolumeCount"},
        {"name" : "CheckNodeDiskPressure"},
        {"name" : "CheckVolumeBinding"},
        {"name" : "CheckNodeCondition"},
        {"name" : "checkServiceAffinity"},
        {"name" : "PodToleratesNodeNoExecuteTaints"},
        {"name" : "CheckNodeLabelPresence"},
        {"name" : "MaxGCEPDVolumeCount"},
        {"name" : "CheckNodeMemoryPressure"},
        {"name" : "MatchInterPodAffinity"},
        {"name" : "PodToleratesNodeTaints"},
        {"name" : "HostName"}
        ],
"priorities" : [     
        {"name" : "LeastRequestedPriority", "weight" : 1},
        {"name" : "BalancedResourceAllocation", "weight" : 1},
        {"name" : "ServiceSpreadingPriority", "weight" : 1},
        {"name" : "EqualPriority", "weight" : 1}
        ]
}

Events:  <none>

$ oc describe scheduler cluster
Name:         cluster
Namespace:    
Labels:       <none>
Annotations:  release.openshift.io/create-only: true
API Version:  config.openshift.io/v1
Kind:         Scheduler
Metadata:
  Creation Timestamp:  2019-07-15T15:43:46Z
  Generation:          2
  Resource Version:    20724
  Self Link:           /apis/config.openshift.io/v1/schedulers/cluster
  UID:                 54fa9766-a717-11e9-986e-020734821dbc
Spec:
  Policy:
    Name:  scheduler-policy
Events:    <none>

$ oc get pods
NAME                                                                   READY   STATUS      RESTARTS   AGE
openshift-kube-scheduler-ip-10-0-131-170.ap-south-1.compute.internal   1/1     Running     0          26m
openshift-kube-scheduler-ip-10-0-144-254.ap-south-1.compute.internal   1/1     Running     0          27m
openshift-kube-scheduler-ip-10-0-175-14.ap-south-1.compute.internal    1/1     Running     0          28m


$ oc logs openshift-kube-scheduler-ip-10-0-131-170.ap-south-1.compute.internal | grep -i Registering
I0715 16:55:49.311095       1 factory.go:1184] Registering predicate: PodFitsHostPorts
I0715 16:55:49.311115       1 factory.go:1184] Registering predicate: PodFitsResources
I0715 16:55:49.311130       1 factory.go:1184] Registering predicate: NoDiskConflict
I0715 16:55:49.311144       1 factory.go:1184] Registering predicate: NoVolumeZoneConflict
I0715 16:55:49.311154       1 factory.go:1184] Registering predicate: MatchNodeSelector
I0715 16:55:49.311162       1 factory.go:1184] Registering predicate: HostName
I0715 16:55:49.311172       1 factory.go:1199] Registering priority: LeastRequestedPriority
I0715 16:55:49.311187       1 factory.go:1199] Registering priority: BalancedResourceAllocation
I0715 16:55:49.311198       1 factory.go:1199] Registering priority: ServiceSpreadingPriority
I0715 16:55:49.311208       1 factory.go:1199] Registering priority: EqualPriority


oc logs openshift-kube-scheduler-operator-56b48ff967-vbz4b -n openshift-kube-scheduler-operator | grep -i policy
[...]
I0715 16:52:50.736736       1 target_config_reconciler_v311_00.go:118] Error while listing scheduler configmap from openshift-config namespace configmap "scheduler-policy" not found and using default algorithm provider in kubernetes scheduler
I0715 16:52:52.937069       1 target_config_reconciler_v311_00.go:118] Error while listing scheduler configmap from openshift-config namespace configmap "scheduler-policy" not found and using default algorithm provider in kubernetes scheduler
I0715 16:52:55.336806       1 target_config_reconciler_v311_00.go:118] Error while listing scheduler configmap from openshift-config namespace configmap "scheduler-policy" not found and using default algorithm provider in kubernetes scheduler
I0715 16:52:58.336692       1 target_config_reconciler_v311_00.go:118] Error while listing scheduler configmap from openshift-config namespace configmap "scheduler-policy" not found and using default algorithm provider in kubernetes scheduler
I0715 16:53:01.549307       1 target_config_reconciler_v311_00.go:110] Custom policy config map to be used by scheduler is successfully created
I0715 16:53:05.941797       1 event.go:221] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-scheduler-operator", Name:"openshift-kube-scheduler-operator", UID:"4cc23118-a717-11e9-986e-020734821dbc", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ConfigMapCreated' Created ConfigMap/policy-configmap-9 -n openshift-kube-scheduler because it was missing
I0715 16:53:06.944817       1 target_config_reconciler_v311_00.go:110] Custom policy config map to be used by scheduler is successfully created
I0715 16:53:14.744844       1 target_config_reconciler_v311_00.go:110] Custom policy config map to be used by scheduler is successfully created
I0715 16:53:19.544696       1 target_config_reconciler_v311_00.go:110] Custom policy config map to be used by scheduler is successfully created
I0715 16:53:22.545128       1 target_config_reconciler_v311_00.go:110] Custom policy config map to be used by scheduler is successfully created
[...]

Comment 2 Sunil Choudhary 2019-07-15 17:33:14 UTC
I opened below issue assuming this was docs issue.

https://github.com/openshift/openshift-docs/pull/14676

Comment 3 Mike Dame 2019-07-25 16:53:25 UTC
Checking on 4.2.0-0.okd-2019-07-25-125949 this seems to now be fixed (I assume by https://github.com/openshift/cluster-kube-scheduler-operator/pull/136)

@Ravi should we backport that configobserver pattern change to 4.1 (this bug) or just move this bug to 4.2?

Comment 8 errata-xmlrpc 2019-09-25 07:27:53 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2820


Note You need to log in before you can comment on or make changes to this bug.