Description of problem: When upgrading from 4.9 to 4.10 kube-scheduler is in terminated state with errors below tes/enhancements/tree/master/keps/sig-instrumentation/2845-deprecate-klog-specific-flags-in-k8s-components)\nStaticPodsDegraded 01-18 03:03:43.377 [36m - lastTransitionTime: "2022-01-17T19:50:18Z"[0m 01-18 03:03:43.377 [36m message: "StaticPodsDegraded: pod/openshift-kube-scheduler-ip-10-0-151-230.us-east-2.compute.internal[0m 01-18 03:03:43.377 [36m container \"kube-scheduler\" is terminated: Error: tes/enhancements/tree/master/keps/sig-instrumentation/2845-deprecate-klog-specific-flags-in-k8s-components)\nStaticPodsDegraded:[0m 01-18 03:03:43.377 [36m \ --log-file-max-size uint Defines the maximum size a log file[0m 01-18 03:03:43.377 [36m can grow to. Unit is megabytes. If the value is 0, the maximum file size is[0m 01-18 03:03:43.377 [36m unlimited. (default 1800) (DEPRECATED: will be removed in a future release,[0m 01-18 03:03:43.377 [36m see https://github.com/kubernetes/enhancements/tree/master/keps/sig-instrumentation/2845-deprecate-klog-specific-flags-in-k8s-components)\nStaticPodsDegraded:[0m 01-18 03:03:43.377 [36m \ --logtostderr log to standard error instead of files[0m 01-18 03:03:43.377 [36m (default true) (DEPRECATED: will be removed in a future release, see https://github.com/kubernetes/enhancements/tree/master/keps/sig-instrumentation/2845-deprecate-klog-specific-flags-in-k8s-components)\nStaticPodsDegraded:[0m 01-18 03:03:43.377 [36m \ --one-output If true, only write logs to their[0m 01-18 03:03:43.377 [36m native severity level (vs also writing to each lower severity level) (DEPRECATED:[0m 01-18 03:03:43.377 [36m will be removed in a future release, see https://github.com/kubernetes/enhancements/tree/master/keps/sig-instrumentation/2845-deprecate-klog-specific-flags-in-k8s-components)\nStaticPodsDegraded:[0m 01-18 03:03:43.377 [36m \ --skip-headers If true, avoid header prefixes in[0m 01-18 03:03:43.377 [36m the log messages (DEPRECATED: will be removed in a future release, see https://github.com/kubernetes/enhancements/tree/master/keps/sig-instrumentation/2845-deprecate-klog-specific-flags-in-k8s-components)\nStaticPodsDegraded:[0m 01-18 03:03:43.377 [36m \ --skip-log-headers If true, avoid headers when opening[0m 01-18 03:03:43.377 [36m log files (DEPRECATED: will be removed in a future release, see https://github.com/kubernetes/enhancements/tree/master/keps/sig-instrumentation/2845-deprecate-klog-specific-flags-in-k8s-components)\nStaticPodsDegraded:[0m 01-18 03:03:43.377 [36m \ --stderrthreshold severity logs at or above this threshold go[0m 01-18 03:03:43.377 [36m to stderr (default 2) (DEPRECATED: will be removed in a future release, see[0m 01-18 03:03:43.377 [36m https://github.com/kubernetes/enhancements/tree/master/keps/sig-instrumentation/2845-deprecate-klog-specific-flags-in-k8s-components)\nStaticPodsDegraded:[0m 01-18 03:03:43.377 [36m \ --version version[=true] Print version information and quit\nStaticPodsDegraded:[0m 01-18 03:03:43.377 [36m \nStaticPodsDegraded: Error: unknown flag: --policy-configmap\nNodeControllerDegraded:[0m Version-Release number of selected component (if applicable): 4.9.0-0.nightly-2022-01-17-165424 -> 4.10.0-0.nightly-2022-01-17-182202 How reproducible: Always Steps to Reproduce: 1. Install latest 4.9 nightly 2. Create policy.cfg with the contents below create a policy.cfg file with the contents below { "kind" : "Policy", "apiVersion" : "v1", "predicates" : [ {"name" : "MaxGCEPDVolumeCount"}, {"name" : "GeneralPredicates"}, {"name" : "MaxAzureDiskVolumeCount"}, {"name" : "MaxCSIVolumeCountPred"}, {"name" : "CheckVolumeBinding"}, {"name" : "MaxEBSVolumeCount"}, {"name" : "MatchInterPodAffinity"}, {"name" : "CheckNodeUnschedulable"}, {"name" : "NoDiskConflict"}, {"name" : "NoVolumeZoneConflict"}, {"name" : "PodToleratesNodeTaints"} ], "priorities" : [ {"name" : "LeastRequestedPriority", "weight" : 1}, {"name" : "BalancedResourceAllocation", "weight" : 1}, {"name" : "ServiceSpreadingPriority", "weight" : 1}, {"name" : "NodePreferAvoidPodsPriority", "weight" : 1}, {"name" : "NodeAffinityPriority", "weight" : 1}, {"name" : "TaintTolerationPriority", "weight" : 1}, {"name" : "ImageLocalityPriority", "weight" : 1}, {"name" : "SelectorSpreadPriority", "weight" : 1}, {"name" : "InterPodAffinityPriority", "weight" : 1}, {"name" : "EqualPriority", "weight" : 1} ] } 3.oc create configmap -n openshift-config --from-file=policy.cfg scheduler-policy 4. oc patch Scheduler cluster --type='merge' -p '{"spec":{"policy":{"name":"scheduler-policy"}}}' --type=merge 5. Run the upgrade command to perform an upgrade from 4.9 to 4.10 Actual results: kube-scheduler is in terminated state with errors as stated in the description Expected results: kube-scheduler should not be in terminated state and upgrade should work fine. Additional info: I have another upgrade bug (https://bugzilla.redhat.com/show_bug.cgi?id=2037665)with similar steps but errors seen here are different than what we have seen in the other bug.
Link to must-gather log: http://virt-openshift-05.lab.eng.nay.redhat.com/knarra/2041985/must-gather-67902-496827902.tar.gz
*** This bug has been marked as a duplicate of bug 2037665 ***
Closing the as duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=2037665 based on the following error message: ``` StaticPodsDegraded: Error: unknown flag: --policy-configmap ```
I am re-opening this bug because it is blocking QE's 4.10 upgrade tests on ci and we do not have a 4.10 bug to track this.
Closing again because QE's upgrade tests on ci works fine after disabling one of the tests which is related to https://bugzilla.redhat.com/show_bug.cgi?id=2037665