Description of problem: I see warnings related to unrecognized feature gate in kubescheduler logs [knarra@knarra cucushift]$ oc logs -f openshift-kube-scheduler-ip-10-0-134-32.us-east-2.compute.internal -n openshift-kube-scheduler Using deprecated annotation `kubectl.kubernetes.io/default-logs-container` in pod/openshift-kube-scheduler-ip-10-0-134-32.us-east-2.compute.internal. Please use `kubectl.kubernetes.io/default-container` instead W0118 16:18:04.481020 1 feature_gate.go:223] unrecognized feature gate: ServiceNodeExclusion W0118 16:18:04.481090 1 feature_gate.go:223] unrecognized feature gate: LegacyNodeRoleBehavior W0118 16:18:04.481093 1 feature_gate.go:223] unrecognized feature gate: NodeDisruptionExclusion W0118 16:18:04.481098 1 feature_gate.go:223] unrecognized feature gate: SupportPodPidsLimit Version-Release number of selected component (if applicable): [knarra@knarra cucushift]$ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.10.0-0.nightly-2022-01-18-044014 True False 127m Cluster version is 4.10.0-0.nightly-2022-01-18-044014 How reproducible: Always Steps to Reproduce: 1. Install 4.10 cluster 2. Now run the command oc log -f <scheduler_pod> -n openshift-kube-scheduler 3. Actual results: Below warnings about unrecognized feature gates are seen in kubescheduler logs. [knarra@knarra cucushift]$ oc logs -f openshift-kube-scheduler-ip-10-0-134-32.us-east-2.compute.internal -n openshift-kube-scheduler Using deprecated annotation `kubectl.kubernetes.io/default-logs-container` in pod/openshift-kube-scheduler-ip-10-0-134-32.us-east-2.compute.internal. Please use `kubectl.kubernetes.io/default-container` instead W0118 16:18:04.481020 1 feature_gate.go:223] unrecognized feature gate: ServiceNodeExclusion W0118 16:18:04.481090 1 feature_gate.go:223] unrecognized feature gate: LegacyNodeRoleBehavior W0118 16:18:04.481093 1 feature_gate.go:223] unrecognized feature gate: NodeDisruptionExclusion W0118 16:18:04.481098 1 feature_gate.go:223] unrecognized feature gate: SupportPodPidsLimit Expected results: No warnings should be seen Additional info: Same warnins are seen in KCM logs as well [knarra@knarra cucushift]$ oc logs -f kube-controller-manager-ip-10-0-134-32.us-east-2.compute.internal -n openshift-kube-controller-manager + timeout 3m /bin/bash -exuo pipefail -c 'while [ -n "$(ss -Htanop \( sport = 10257 \))" ]; do sleep 1; done' ++ ss -Htanop '(' sport = 10257 ')' Copying system trust bundle + '[' -n '' ']' + '[' -f /etc/kubernetes/static-pod-certs/configmaps/trusted-ca-bundle/ca-bundle.crt ']' + echo 'Copying system trust bundle' + cp -f /etc/kubernetes/static-pod-certs/configmaps/trusted-ca-bundle/ca-bundle.crt /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem + '[' -f /etc/kubernetes/static-pod-resources/configmaps/cloud-config/ca-bundle.pem ']' + exec hyperkube kube-controller-manager --openshift-config=/etc/kubernetes/static-pod-resources/configmaps/config/config.yaml --kubeconfig=/etc/kubernetes/static-pod-resources/configmaps/controller-manager-kubeconfig/kubeconfig --authentication-kubeconfig=/etc/kubernetes/static-pod-resources/configmaps/controller-manager-kubeconfig/kubeconfig --authorization-kubeconfig=/etc/kubernetes/static-pod-resources/configmaps/controller-manager-kubeconfig/kubeconfig --client-ca-file=/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt --requestheader-client-ca-file=/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt -v=2 --tls-cert-file=/etc/kubernetes/static-pod-resources/secrets/serving-cert/tls.crt --tls-private-key-file=/etc/kubernetes/static-pod-resources/secrets/serving-cert/tls.key --allocate-node-cidrs=false --cert-dir=/var/run/kubernetes --cloud-provider=aws --cluster-cidr=10.128.0.0/14 --cluster-name=knarra0118-wbsl9 --cluster-signing-cert-file=/etc/kubernetes/static-pod-certs/secrets/csr-signer/tls.crt --cluster-signing-duration=720h --cluster-signing-key-file=/etc/kubernetes/static-pod-certs/secrets/csr-signer/tls.key --configure-cloud-routes=false '--controllers=*' --controllers=-bootstrapsigner --controllers=-tokencleaner --controllers=-ttl --enable-dynamic-provisioning=true --feature-gates=APIPriorityAndFairness=true --feature-gates=CSIMigrationAWS=false --feature-gates=CSIMigrationAzureDisk=false --feature-gates=CSIMigrationAzureFile=false --feature-gates=CSIMigrationGCE=false --feature-gates=CSIMigrationOpenStack=false --feature-gates=CSIMigrationvSphere=false --feature-gates=DownwardAPIHugePages=true --feature-gates=LegacyNodeRoleBehavior=false --feature-gates=NodeDisruptionExclusion=true --feature-gates=PodSecurity=true --feature-gates=RotateKubeletServerCertificate=true --feature-gates=ServiceNodeExclusion=true --feature-gates=SupportPodPidsLimit=true --flex-volume-plugin-dir=/etc/kubernetes/kubelet-plugins/volume/exec --kube-api-burst=300 --kube-api-qps=150 --leader-elect-resource-lock=configmaps --leader-elect-retry-period=3s --leader-elect=true --pv-recycler-pod-template-filepath-hostpath=/etc/kubernetes/static-pod-resources/configmaps/recycler-config/recycler-pod.yaml --pv-recycler-pod-template-filepath-nfs=/etc/kubernetes/static-pod-resources/configmaps/recycler-config/recycler-pod.yaml --root-ca-file=/etc/kubernetes/static-pod-resources/configmaps/serviceaccount-ca/ca-bundle.crt --secure-port=10257 --service-account-private-key-file=/etc/kubernetes/static-pod-resources/secrets/service-account-private-key/service-account.key --service-cluster-ip-range=172.30.0.0/16 --use-service-account-credentials=true --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 --tls-min-version=VersionTLS12 W0118 16:21:18.036475 1 feature_gate.go:223] unrecognized feature gate: LegacyNodeRoleBehavior W0118 16:21:18.036582 1 feature_gate.go:223] unrecognized feature gate: NodeDisruptionExclusion W0118 16:21:18.036696 1 feature_gate.go:223] unrecognized feature gate: ServiceNodeExclusion W0118 16:21:18.036735 1 feature_gate.go:223] unrecognized feature gate: SupportPodPidsLimit
Based on https://github.com/kubernetes/kubernetes/pull/100776 LegacyNodeRoleBehavior, NodeDisruptionExclusion and ServiceNodeExclusion were GA-ed in 1.22. Based on https://github.com/kubernetes/kubernetes/pull/104163 SupportPodPidsLimit was GA-ed in 1.20.
Verified bug with the payload below and i see that there are no warnings present anymore in the logs. kube-scheduler logs: ====================== [knarra@knarra ~]$ oc logs -f openshift-kube-scheduler-ip-10-0-130-58.ap-northeast-1.compute.internal -n openshift-kube-scheduler Using deprecated annotation `kubectl.kubernetes.io/default-logs-container` in pod/openshift-kube-scheduler-ip-10-0-130-58.ap-northeast-1.compute.internal. Please use `kubectl.kubernetes.io/default-container` instead I0127 06:54:39.651742 1 flags.go:64] FLAG: --add-dir-header="false" I0127 06:54:39.651861 1 flags.go:64] FLAG: --address="127.0.0.1" I0127 06:54:39.651868 1 flags.go:64] FLAG: --allow-metric-labels="[]" I0127 06:54:39.651878 1 flags.go:64] FLAG: --alsologtostderr="false" I0127 06:54:39.651883 1 flags.go:64] FLAG: --authentication-kubeconfig="/etc/kubernetes/static-pod-resources/configmaps/scheduler-kubeconfig/kubeconfig" I0127 06:54:39.651891 1 flags.go:64] FLAG: --authentication-skip-lookup="false" I0127 06:54:39.651905 1 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="10s" I0127 06:54:39.651911 1 flags.go:64] FLAG: --authentication-tolerate-lookup-failure="true" I0127 06:54:39.651915 1 flags.go:64] FLAG: --authorization-always-allow-paths="[/healthz,/readyz,/livez]" I0127 06:54:39.651923 1 flags.go:64] FLAG: --authorization-kubeconfig="/etc/kubernetes/static-pod-resources/configmaps/scheduler-kubeconfig/kubeconfig" I0127 06:54:39.651930 1 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="10s" I0127 06:54:39.651934 1 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="10s" I0127 06:54:39.651939 1 flags.go:64] FLAG: --bind-address="0.0.0.0" I0127 06:54:39.651943 1 flags.go:64] FLAG: --cert-dir="/var/run/kubernetes" I0127 06:54:39.651948 1 flags.go:64] FLAG: --client-ca-file="" I0127 06:54:39.651951 1 flags.go:64] FLAG: --config="/etc/kubernetes/static-pod-resources/configmaps/config/config.yaml" I0127 06:54:39.651957 1 flags.go:64] FLAG: --contention-profiling="true" I0127 06:54:39.651962 1 flags.go:64] FLAG: --disabled-metrics="[]" I0127 06:54:39.651969 1 flags.go:64] FLAG: --experimental-logging-sanitization="false" I0127 06:54:39.651973 1 flags.go:64] FLAG: --feature-gates="APIPriorityAndFairness=true,CSIMigrationAWS=false,CSIMigrationAzureDisk=false,CSIMigrationAzureFile=false,CSIMigrationGCE=false,CSIMigrationOpenStack=false,CSIMigrationvSphere=false,DownwardAPIHugePages=true,PodSecurity=true,RotateKubeletServerCertificate=true" I0127 06:54:39.651999 1 flags.go:64] FLAG: --help="false" kube-controller-manager logs: ==================================== [knarra@knarra ~]$ oc logs -f kube-controller-manager-ip-10-0-130-58.ap-northeast-1.compute.internal -n openshift-kube-controller-manager + timeout 3m /bin/bash -exuo pipefail -c 'while [ -n "$(ss -Htanop \( sport = 10257 \))" ]; do sleep 1; done' ++ ss -Htanop '(' sport = 10257 ')' + '[' -n '' ']' + '[' -f /etc/kubernetes/static-pod-certs/configmaps/trusted-ca-bundle/ca-bundle.crt ']' + echo 'Copying system trust bundle' Copying system trust bundle + cp -f /etc/kubernetes/static-pod-certs/configmaps/trusted-ca-bundle/ca-bundle.crt /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem + '[' -f /etc/kubernetes/static-pod-resources/configmaps/cloud-config/ca-bundle.pem ']' + exec hyperkube kube-controller-manager --openshift-config=/etc/kubernetes/static-pod-resources/configmaps/config/config.yaml --kubeconfig=/etc/kubernetes/static-pod-resources/configmaps/controller-manager-kubeconfig/kubeconfig --authentication-kubeconfig=/etc/kubernetes/static-pod-resources/configmaps/controller-manager-kubeconfig/kubeconfig --authorization-kubeconfig=/etc/kubernetes/static-pod-resources/configmaps/controller-manager-kubeconfig/kubeconfig --client-ca-file=/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt --requestheader-client-ca-file=/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt -v=2 --tls-cert-file=/etc/kubernetes/static-pod-resources/secrets/serving-cert/tls.crt --tls-private-key-file=/etc/kubernetes/static-pod-resources/secrets/serving-cert/tls.key --allocate-node-cidrs=false --cert-dir=/var/run/kubernetes --cloud-provider=aws --cluster-cidr=10.128.0.0/14 --cluster-name=knarra0127-phxxt --cluster-signing-cert-file=/etc/kubernetes/static-pod-certs/secrets/csr-signer/tls.crt --cluster-signing-duration=720h --cluster-signing-key-file=/etc/kubernetes/static-pod-certs/secrets/csr-signer/tls.key --configure-cloud-routes=false '--controllers=*' --controllers=-bootstrapsigner --controllers=-tokencleaner --controllers=-ttl --enable-dynamic-provisioning=true --feature-gates=APIPriorityAndFairness=true --feature-gates=CSIMigrationAWS=false --feature-gates=CSIMigrationAzureDisk=false --feature-gates=CSIMigrationAzureFile=false --feature-gates=CSIMigrationGCE=false --feature-gates=CSIMigrationOpenStack=false --feature-gates=CSIMigrationvSphere=false --feature-gates=DownwardAPIHugePages=true --feature-gates=PodSecurity=true --feature-gates=RotateKubeletServerCertificate=true --flex-volume-plugin-dir=/etc/kubernetes/kubelet-plugins/volume/exec --kube-api-burst=300 --kube-api-qps=150 --leader-elect-resource-lock=leases --leader-elect-retry-period=3s --leader-elect=true --pv-recycler-pod-template-filepath-hostpath=/etc/kubernetes/static-pod-resources/configmaps/recycler-config/recycler-pod.yaml --pv-recycler-pod-template-filepath-nfs=/etc/kubernetes/static-pod-resources/configmaps/recycler-config/recycler-pod.yaml --root-ca-file=/etc/kubernetes/static-pod-resources/configmaps/serviceaccount-ca/ca-bundle.crt --secure-port=10257 --service-account-private-key-file=/etc/kubernetes/static-pod-resources/secrets/service-account-private-key/service-account.key --service-cluster-ip-range=172.30.0.0/16 --use-service-account-credentials=true --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 --tls-min-version=VersionTLS12 I0127 06:58:21.022846 1 flags.go:64] FLAG: --add-dir-header="false" I0127 06:58:21.022935 1 flags.go:64] FLAG: --address="127.0.0.1" I0127 06:58:21.022942 1 flags.go:64] FLAG: --allocate-node-cidrs="false" I0127 06:58:21.022948 1 flags.go:64] FLAG: --allow-metric-labels="[]" I0127 06:58:21.022954 1 flags.go:64] FLAG: --allow-untagged-cloud="false" I0127 06:58:21.022958 1 flags.go:64] FLAG: --alsologtostderr="false" I0127 06:58:21.022963 1 flags.go:64] FLAG: --attach-detach-reconcile-sync-period="1m0s" I0127 06:58:21.022969 1 flags.go:64] FLAG: --authentication-kubeconfig="/etc/kubernetes/static-pod-resources/configmaps/controller-manager-kubeconfig/kubeconfig" I0127 06:58:21.022984 1 flags.go:64] FLAG: --authentication-skip-lookup="false" I0127 06:58:21.022988 1 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="10s" kube-api-server logs: ========================= checked the openshift-kube-api-server logs and i see that warnings are still present. so moving bug back to assigned state. I0127 06:56:55.803934 16 feature_gate.go:246] feature gates: &{map[APIPriorityAndFairness:true]} I0127 06:56:55.804011 16 feature_gate.go:246] feature gates: &{map[APIPriorityAndFairness:true RotateKubeletServerCertificate:true]} W0127 06:56:55.804059 16 feature_gate.go:223] unrecognized feature gate: SupportPodPidsLimit I0127 06:56:55.804065 16 feature_gate.go:246] feature gates: &{map[APIPriorityAndFairness:true RotateKubeletServerCertificate:true]} W0127 06:56:55.804105 16 feature_gate.go:223] unrecognized feature gate: NodeDisruptionExclusion I0127 06:56:55.804111 16 feature_gate.go:246] feature gates: &{map[APIPriorityAndFairness:true RotateKubeletServerCertificate:true]} W0127 06:56:55.804151 16 feature_gate.go:223] unrecognized feature gate: ServiceNodeExclusion I0127 06:56:55.804156 16 feature_gate.go:246] feature gates: &{map[APIPriorityAndFairness:true RotateKubeletServerCertificate:true]} I0127 06:56:55.804196 16 feature_gate.go:246] feature gates: &{map[APIPriorityAndFairness:true DownwardAPIHugePages:true RotateKubeletServerCertificate:true]} I0127 06:56:55.804235 16 feature_gate.go:246] feature gates: &{map[APIPriorityAndFairness:true DownwardAPIHugePages:true PodSecurity:true RotateKubeletServerCertificate:true]} W0127 06:56:55.804299 16 feature_gate.go:223] unrecognized feature gate: LegacyNodeRoleBehavior I0127 06:56:55.804304 16 feature_gate.go:246] feature gates: &{map[APIPriorityAndFairness:true DownwardAPIHugePages:true PodSecurity:true RotateKubeletServerCertificate:true]} I0127 06:56:55.804343 16 feature_gate.go:246] feature gates: &{map[APIPriorityAndFairness:true CSIMigrationAWS:false DownwardAPIHugePages:true PodSecurity:true RotateKubeletServerCertificate:true]} I0127 06:56:55.804393 16 feature_gate.go:246] feature gates: &{map[APIPriorityAndFairness:true CSIMigrationAWS:false CSIMigrationOpenStack:false DownwardAPIHugePages:true PodSecurity:true RotateKubeletServerCertificate:true]} I0127 06:56:55.804434 16 feature_gate.go:246] feature gates: &{map[APIPriorityAndFairness:true CSIMigrationAWS:false CSIMigrationGCE:false CSIMigrationOpenStack:false DownwardAPIHugePages:true PodSecurity:true RotateKubeletServerCertificate:true]} Based on the above moving bug to assigned state.
Verified with the build below and i do not see any warnings related to feature gate present in ks, kcm & kas logs. kube-scheduler logs: ========================= [knarra@knarra cucushift]$ oc logs -f openshift-kube-scheduler-ip-10-0-192-9.us-east-2.compute.internal -n openshift-kube-scheduler I0131 07:28:49.685986 1 flags.go:64] FLAG: --add-dir-header="false" I0131 07:28:49.686132 1 flags.go:64] FLAG: --address="127.0.0.1" I0131 07:28:49.686138 1 flags.go:64] FLAG: --allow-metric-labels="[]" I0131 07:28:49.686152 1 flags.go:64] FLAG: --alsologtostderr="false" I0131 07:28:49.686155 1 flags.go:64] FLAG: --authentication-kubeconfig="/etc/kubernetes/static-pod-resources/configmaps/scheduler-kubeconfig/kubeconfig" I0131 07:28:49.686161 1 flags.go:64] FLAG: --authentication-skip-lookup="false" I0131 07:28:49.686165 1 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="10s" I0131 07:28:49.686169 1 flags.go:64] FLAG: --authentication-tolerate-lookup-failure="true" I0131 07:28:49.686172 1 flags.go:64] FLAG: --authorization-always-allow-paths="[/healthz,/readyz,/livez]" I0131 07:28:49.686179 1 flags.go:64] FLAG: --authorization-kubeconfig="/etc/kubernetes/static-pod-resources/configmaps/scheduler-kubeconfig/kubeconfig" I0131 07:28:49.686183 1 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="10s" I0131 07:28:49.686186 1 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="10s" I0131 07:28:49.686189 1 flags.go:64] FLAG: --bind-address="0.0.0.0" I0131 07:28:49.686192 1 flags.go:64] FLAG: --cert-dir="/var/run/kubernetes" I0131 07:28:49.686195 1 flags.go:64] FLAG: --client-ca-file="" I0131 07:28:49.686198 1 flags.go:64] FLAG: --config="/etc/kubernetes/static-pod-resources/configmaps/config/config.yaml" I0131 07:28:49.686202 1 flags.go:64] FLAG: --contention-profiling="true" I0131 07:28:49.686205 1 flags.go:64] FLAG: --disabled-metrics="[]" I0131 07:28:49.686209 1 flags.go:64] FLAG: --experimental-logging-sanitization="false" I0131 07:28:49.686212 1 flags.go:64] FLAG: --feature-gates="APIPriorityAndFairness=true,CSIMigrationAWS=false,CSIMigrationAzureDisk=false,CSIMigrationAzureFile=false,CSIMigrationGCE=false,CSIMigrationOpenStack=false,CSIMigrationvSphere=false,DownwardAPIHugePages=true,PodSecurity=true,RotateKubeletServerCertificate=true" I0131 07:28:49.686235 1 flags.go:64] FLAG: --help="false" I0131 07:28:49.686238 1 flags.go:64] FLAG: --http2-max-streams-per-connection="0" I0131 07:28:49.686245 1 flags.go:64] FLAG: --kube-api-burst="100" I0131 07:28:49.686250 1 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" I0131 07:28:49.686254 1 flags.go:64] FLAG: --kube-api-qps="50" I0131 07:28:49.686259 1 flags.go:64] FLAG: --kubeconfig="" I0131 07:28:49.686262 1 flags.go:64] FLAG: --leader-elect="true" I0131 07:28:49.686265 1 flags.go:64] FLAG: --leader-elect-lease-duration="15s" I0131 07:28:49.686269 1 flags.go:64] FLAG: --leader-elect-renew-deadline="10s" I0131 07:28:49.686272 1 flags.go:64] FLAG: --leader-elect-resource-lock="leases" I0131 07:28:49.686276 1 flags.go:64] FLAG: --leader-elect-resource-name="kube-scheduler" I0131 07:28:49.686279 1 flags.go:64] FLAG: --leader-elect-resource-namespace="kube-system" I0131 07:28:49.686282 1 flags.go:64] FLAG: --leader-elect-retry-period="2s" I0131 07:28:49.686286 1 flags.go:64] FLAG: --lock-object-name="kube-scheduler" I0131 07:28:49.686290 1 flags.go:64] FLAG: --lock-object-namespace="kube-system" I0131 07:28:49.686293 1 flags.go:64] FLAG: --log-backtrace-at=":0" I0131 07:28:49.686296 1 flags.go:64] FLAG: --log-dir="" I0131 07:28:49.686300 1 flags.go:64] FLAG: --log-file="" I0131 07:28:49.686303 1 flags.go:64] FLAG: --log-file-max-size="1800" I0131 07:28:49.686307 1 flags.go:64] FLAG: --log-flush-frequency="5s" I0131 07:28:49.686310 1 flags.go:64] FLAG: --log-json-info-buffer-size="0" I0131 07:28:49.686324 1 flags.go:64] FLAG: --log-json-split-stream="false" I0131 07:28:49.686327 1 flags.go:64] FLAG: --logging-format="text" I0131 07:28:49.686331 1 flags.go:64] FLAG: --logtostderr="true" I0131 07:28:49.686334 1 flags.go:64] FLAG: --master="" I0131 07:28:49.686337 1 flags.go:64] FLAG: --one-output="false" I0131 07:28:49.686342 1 flags.go:64] FLAG: --permit-address-sharing="false" I0131 07:28:49.686345 1 flags.go:64] FLAG: --permit-port-sharing="false" I0131 07:28:49.686348 1 flags.go:64] FLAG: --port="0" I0131 07:28:49.686351 1 flags.go:64] FLAG: --profiling="true" I0131 07:28:49.686354 1 flags.go:64] FLAG: --requestheader-allowed-names="[]" I0131 07:28:49.686358 1 flags.go:64] FLAG: --requestheader-client-ca-file="" I0131 07:28:49.686362 1 flags.go:64] FLAG: --requestheader-extra-headers-prefix="[x-remote-extra-]" I0131 07:28:49.686368 1 flags.go:64] FLAG: --requestheader-group-headers="[x-remote-group]" I0131 07:28:49.686373 1 flags.go:64] FLAG: --requestheader-username-headers="[x-remote-user]" I0131 07:28:49.686378 1 flags.go:64] FLAG: --secure-port="10259" I0131 07:28:49.686381 1 flags.go:64] FLAG: --show-hidden-metrics-for-version="" I0131 07:28:49.686384 1 flags.go:64] FLAG: --skip-headers="false" I0131 07:28:49.686387 1 flags.go:64] FLAG: --skip-log-headers="false" I0131 07:28:49.686390 1 flags.go:64] FLAG: --stderrthreshold="2" I0131 07:28:49.686394 1 flags.go:64] FLAG: --tls-cert-file="/etc/kubernetes/static-pod-resources/secrets/serving-cert/tls.crt" I0131 07:28:49.686398 1 flags.go:64] FLAG: --tls-cipher-suites="[TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256]" I0131 07:28:49.686412 1 flags.go:64] FLAG: --tls-min-version="VersionTLS12" I0131 07:28:49.686417 1 flags.go:64] FLAG: --tls-private-key-file="/etc/kubernetes/static-pod-resources/secrets/serving-cert/tls.key" I0131 07:28:49.686421 1 flags.go:64] FLAG: --tls-sni-cert-key="[]" I0131 07:28:49.686426 1 flags.go:64] FLAG: --unsupported-kube-api-over-localhost="false" I0131 07:28:49.686429 1 flags.go:64] FLAG: --v="2" I0131 07:28:49.686435 1 flags.go:64] FLAG: --version="false" I0131 07:28:49.686440 1 flags.go:64] FLAG: --vmodule="" I0131 07:28:49.686445 1 flags.go:64] FLAG: --write-config-to="" I0131 07:28:49.687568 1 dynamic_serving_content.go:112] "Loaded a new cert/key pair" name="serving-cert::/etc/kubernetes/static-pod-resources/secrets/serving-cert/tls.crt::/etc/kubernetes/static-pod-resources/secrets/serving-cert/tls.key" I0131 07:28:49.967606 1 requestheader_controller.go:244] Loaded a new request header values for RequestHeaderAuthRequestController [knarra@knarra cucushift]$ oc logs openshift-kube-scheduler-ip-10-0-192-9.us-east-2.compute.internal -n openshift-kube-scheduler | grep "unrecognized feature gate" [knarra@knarra cucushift]$ [knarra@knarra cucushift]$ oc logs openshift-kube-scheduler-ip-10-0-192-9.us-east-2.compute.internal -n openshift-kube-scheduler | grep "unrecognized" [knarra@knarra cucushift]$ kube-controller-manager logs: ================================ [knarra@knarra cucushift]$ oc logs -f kube-controller-manager-ip-10-0-192-9.us-east-2.compute.internal -n openshift-kube-controller-manager + timeout 3m /bin/bash -exuo pipefail -c 'while [ -n "$(ss -Htanop \( sport = 10257 \))" ]; do sleep 1; done' ++ ss -Htanop '(' sport = 10257 ')' + '[' -n '' ']' + '[' -f /etc/kubernetes/static-pod-certs/configmaps/trusted-ca-bundle/ca-bundle.crt ']' Copying system trust bundle + echo 'Copying system trust bundle' + cp -f /etc/kubernetes/static-pod-certs/configmaps/trusted-ca-bundle/ca-bundle.crt /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem + '[' -f /etc/kubernetes/static-pod-resources/configmaps/cloud-config/ca-bundle.pem ']' + exec hyperkube kube-controller-manager --openshift-config=/etc/kubernetes/static-pod-resources/configmaps/config/config.yaml --kubeconfig=/etc/kubernetes/static-pod-resources/configmaps/controller-manager-kubeconfig/kubeconfig --authentication-kubeconfig=/etc/kubernetes/static-pod-resources/configmaps/controller-manager-kubeconfig/kubeconfig --authorization-kubeconfig=/etc/kubernetes/static-pod-resources/configmaps/controller-manager-kubeconfig/kubeconfig --client-ca-file=/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt --requestheader-client-ca-file=/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt -v=2 --tls-cert-file=/etc/kubernetes/static-pod-resources/secrets/serving-cert/tls.crt --tls-private-key-file=/etc/kubernetes/static-pod-resources/secrets/serving-cert/tls.key --allocate-node-cidrs=false --cert-dir=/var/run/kubernetes --cloud-provider=aws --cluster-cidr=10.128.0.0/14 --cluster-name=knarra0131-zbwqr --cluster-signing-cert-file=/etc/kubernetes/static-pod-certs/secrets/csr-signer/tls.crt --cluster-signing-duration=720h --cluster-signing-key-file=/etc/kubernetes/static-pod-certs/secrets/csr-signer/tls.key --configure-cloud-routes=false '--controllers=*' --controllers=-bootstrapsigner --controllers=-tokencleaner --controllers=-ttl --enable-dynamic-provisioning=true --feature-gates=APIPriorityAndFairness=true --feature-gates=CSIMigrationAWS=false --feature-gates=CSIMigrationAzureDisk=false --feature-gates=CSIMigrationAzureFile=false --feature-gates=CSIMigrationGCE=false --feature-gates=CSIMigrationOpenStack=false --feature-gates=CSIMigrationvSphere=false --feature-gates=DownwardAPIHugePages=true --feature-gates=PodSecurity=true --feature-gates=RotateKubeletServerCertificate=true --flex-volume-plugin-dir=/etc/kubernetes/kubelet-plugins/volume/exec --kube-api-burst=300 --kube-api-qps=150 --leader-elect-resource-lock=leases --leader-elect-retry-period=3s --leader-elect=true --pv-recycler-pod-template-filepath-hostpath=/etc/kubernetes/static-pod-resources/configmaps/recycler-config/recycler-pod.yaml --pv-recycler-pod-template-filepath-nfs=/etc/kubernetes/static-pod-resources/configmaps/recycler-config/recycler-pod.yaml --root-ca-file=/etc/kubernetes/static-pod-resources/configmaps/serviceaccount-ca/ca-bundle.crt --secure-port=10257 --service-account-private-key-file=/etc/kubernetes/static-pod-resources/secrets/service-account-private-key/service-account.key --service-cluster-ip-range=172.30.0.0/16 --use-service-account-credentials=true --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 --tls-min-version=VersionTLS12 I0131 07:32:09.399916 1 flags.go:64] FLAG: --add-dir-header="false" I0131 07:32:09.400009 1 flags.go:64] FLAG: --address="127.0.0.1" I0131 07:32:09.400016 1 flags.go:64] FLAG: --allocate-node-cidrs="false" I0131 07:32:09.400020 1 flags.go:64] FLAG: --allow-metric-labels="[]" I0131 07:32:09.400025 1 flags.go:64] FLAG: --allow-untagged-cloud="false" I0131 07:32:09.400028 1 flags.go:64] FLAG: --alsologtostderr="false" I0131 07:32:09.400031 1 flags.go:64] FLAG: --attach-detach-reconcile-sync-period="1m0s" I0131 07:32:09.400040 1 flags.go:64] FLAG: --authentication-kubeconfig="/etc/kubernetes/static-pod-resources/configmaps/controller-manager-kubeconfig/kubeconfig" I0131 07:32:09.400049 1 flags.go:64] FLAG: --authentication-skip-lookup="false" I0131 07:32:09.400052 1 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="10s" I0131 07:32:09.400055 1 flags.go:64] FLAG: --authentication-tolerate-lookup-failure="false" I0131 07:32:09.400058 1 flags.go:64] FLAG: --authorization-always-allow-paths="[/healthz,/readyz,/livez]" I0131 07:32:09.400067 1 flags.go:64] FLAG: --authorization-kubeconfig="/etc/kubernetes/static-pod-resources/configmaps/controller-manager-kubeconfig/kubeconfig" I0131 07:32:09.400072 1 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="10s" I0131 07:32:09.400076 1 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="10s" I0131 07:32:09.400078 1 flags.go:64] FLAG: --bind-address="0.0.0.0" I0131 07:32:09.400081 1 flags.go:64] FLAG: --cert-dir="/var/run/kubernetes" I0131 07:32:09.400085 1 flags.go:64] FLAG: --cidr-allocator-type="RangeAllocator" I0131 07:32:09.400088 1 flags.go:64] FLAG: --client-ca-file="/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt" I0131 07:32:09.400092 1 flags.go:64] FLAG: --cloud-config="" I0131 07:32:09.400095 1 flags.go:64] FLAG: --cloud-provider="aws" I0131 07:32:09.400098 1 flags.go:64] FLAG: --cloud-provider-gce-lb-src-cidrs="130.211.0.0/22,209.85.152.0/22,209.85.204.0/22,35.191.0.0/16" I0131 07:32:09.400105 1 flags.go:64] FLAG: --cluster-cidr="10.128.0.0/14" I0131 07:32:09.400108 1 flags.go:64] FLAG: --cluster-name="knarra0131-zbwqr" I0131 07:32:09.400112 1 flags.go:64] FLAG: --cluster-signing-cert-file="/etc/kubernetes/static-pod-certs/secrets/csr-signer/tls.crt" I0131 07:32:09.400117 1 flags.go:64] FLAG: --cluster-signing-duration="720h0m0s" I0131 07:32:09.400121 1 flags.go:64] FLAG: --cluster-signing-key-file="/etc/kubernetes/static-pod-certs/secrets/csr-signer/tls.key" I0131 07:32:09.400125 1 flags.go:64] FLAG: --cluster-signing-kube-apiserver-client-cert-file="" I0131 07:32:09.400128 1 flags.go:64] FLAG: --cluster-signing-kube-apiserver-client-key-file="" I0131 07:32:09.400131 1 flags.go:64] FLAG: --cluster-signing-kubelet-client-cert-file="" I0131 07:32:09.400134 1 flags.go:64] FLAG: --cluster-signing-kubelet-client-key-file="" I0131 07:32:09.400137 1 flags.go:64] FLAG: --cluster-signing-kubelet-serving-cert-file="" I0131 07:32:09.400140 1 flags.go:64] FLAG: --cluster-signing-kubelet-serving-key-file="" I0131 07:32:09.400143 1 flags.go:64] FLAG: --cluster-signing-legacy-unknown-cert-file="" I0131 07:32:09.400146 1 flags.go:64] FLAG: --cluster-signing-legacy-unknown-key-file="" I0131 07:32:09.400149 1 flags.go:64] FLAG: --concurrent-deployment-syncs="5" I0131 07:32:09.400165 1 flags.go:64] FLAG: --concurrent-endpoint-syncs="5" I0131 07:32:09.400172 1 flags.go:64] FLAG: --concurrent-ephemeralvolume-syncs="5" I0131 07:32:09.400175 1 flags.go:64] FLAG: --concurrent-gc-syncs="20" I0131 07:32:09.400178 1 flags.go:64] FLAG: --concurrent-namespace-syncs="10" I0131 07:32:09.400181 1 flags.go:64] FLAG: --concurrent-rc-syncs="5" I0131 07:32:09.400185 1 flags.go:64] FLAG: --concurrent-replicaset-syncs="5" I0131 07:32:09.400188 1 flags.go:64] FLAG: --concurrent-resource-quota-syncs="5" I0131 07:32:09.400191 1 flags.go:64] FLAG: --concurrent-service-endpoint-syncs="5" I0131 07:32:09.400194 1 flags.go:64] FLAG: --concurrent-service-syncs="1" I0131 07:32:09.400197 1 flags.go:64] FLAG: --concurrent-serviceaccount-token-syncs="5" I0131 07:32:09.400200 1 flags.go:64] FLAG: --concurrent-statefulset-syncs="5" I0131 07:32:09.400203 1 flags.go:64] FLAG: --concurrent-ttl-after-finished-syncs="5" I0131 07:32:09.400206 1 flags.go:64] FLAG: --configure-cloud-routes="false" I0131 07:32:09.400209 1 flags.go:64] FLAG: --contention-profiling="false" I0131 07:32:09.400213 1 flags.go:64] FLAG: --controller-start-interval="0s" I0131 07:32:09.400216 1 flags.go:64] FLAG: --controllers="[*,-bootstrapsigner,-tokencleaner,-ttl]" I0131 07:32:09.400221 1 flags.go:64] FLAG: --deleting-pods-burst="0" I0131 07:32:09.400224 1 flags.go:64] FLAG: --deleting-pods-qps="0.1" I0131 07:32:09.400231 1 flags.go:64] FLAG: --deployment-controller-sync-period="30s" I0131 07:32:09.400235 1 flags.go:64] FLAG: --disable-attach-detach-reconcile-sync="false" I0131 07:32:09.400238 1 flags.go:64] FLAG: --disabled-metrics="[]" I0131 07:32:09.400244 1 flags.go:64] FLAG: --enable-dynamic-provisioning="true" I0131 07:32:09.400247 1 flags.go:64] FLAG: --enable-garbage-collector="true" I0131 07:32:09.400251 1 flags.go:64] FLAG: --enable-hostpath-provisioner="false" I0131 07:32:09.400254 1 flags.go:64] FLAG: --enable-leader-migration="false" I0131 07:32:09.400257 1 flags.go:64] FLAG: --enable-taint-manager="true" I0131 07:32:09.400260 1 flags.go:64] FLAG: --endpoint-updates-batch-period="0s" I0131 07:32:09.400264 1 flags.go:64] FLAG: --endpointslice-updates-batch-period="0s" I0131 07:32:09.400267 1 flags.go:64] FLAG: --experimental-cluster-signing-duration="720h0m0s" I0131 07:32:09.400270 1 flags.go:64] FLAG: --experimental-logging-sanitization="false" I0131 07:32:09.400273 1 flags.go:64] FLAG: --external-cloud-volume-plugin="" I0131 07:32:09.400276 1 flags.go:64] FLAG: --feature-gates="APIPriorityAndFairness=true,CSIMigrationAWS=false,CSIMigrationAzureDisk=false,CSIMigrationAzureFile=false,CSIMigrationGCE=false,CSIMigrationOpenStack=false,CSIMigrationvSphere=false,DownwardAPIHugePages=true,PodSecurity=true,RotateKubeletServerCertificate=true" I0131 07:32:09.400298 1 flags.go:64] FLAG: --flex-volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" I0131 07:32:09.400304 1 flags.go:64] FLAG: --help="false" I0131 07:32:09.400307 1 flags.go:64] FLAG: --horizontal-pod-autoscaler-cpu-initialization-period="5m0s" I0131 07:32:09.400310 1 flags.go:64] FLAG: --horizontal-pod-autoscaler-downscale-delay="5m0s" I0131 07:32:09.400314 1 flags.go:64] FLAG: --horizontal-pod-autoscaler-downscale-stabilization="5m0s" I0131 07:32:09.400317 1 flags.go:64] FLAG: --horizontal-pod-autoscaler-initial-readiness-delay="30s" I0131 07:32:09.400320 1 flags.go:64] FLAG: --horizontal-pod-autoscaler-sync-period="15s" I0131 07:32:09.400323 1 flags.go:64] FLAG: --horizontal-pod-autoscaler-tolerance="0.1" I0131 07:32:09.400328 1 flags.go:64] FLAG: --horizontal-pod-autoscaler-upscale-delay="3m0s" I0131 07:32:09.400331 1 flags.go:64] FLAG: --http2-max-streams-per-connection="0" I0131 07:32:09.400336 1 flags.go:64] FLAG: --insecure-experimental-approve-all-kubelet-csrs-for-group="" I0131 07:32:09.400339 1 flags.go:64] FLAG: --kube-api-burst="300" I0131 07:32:09.400342 1 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" I0131 07:32:09.400347 1 flags.go:64] FLAG: --kube-api-qps="150" I0131 07:32:09.400351 1 flags.go:64] FLAG: --kubeconfig="/etc/kubernetes/static-pod-resources/configmaps/controller-manager-kubeconfig/kubeconfig" I0131 07:32:09.400356 1 flags.go:64] FLAG: --large-cluster-size-threshold="50" I0131 07:32:09.400360 1 flags.go:64] FLAG: --leader-elect="true" I0131 07:32:09.400363 1 flags.go:64] FLAG: --leader-elect-lease-duration="15s" I0131 07:32:09.400367 1 flags.go:64] FLAG: --leader-elect-renew-deadline="10s" I0131 07:32:09.400371 1 flags.go:64] FLAG: --leader-elect-resource-lock="leases" I0131 07:32:09.400374 1 flags.go:64] FLAG: --leader-elect-resource-name="kube-controller-manager" I0131 07:32:09.400378 1 flags.go:64] FLAG: --leader-elect-resource-namespace="kube-system" I0131 07:32:09.400382 1 flags.go:64] FLAG: --leader-elect-retry-period="3s" I0131 07:32:09.400386 1 flags.go:64] FLAG: --leader-migration-config="" I0131 07:32:09.400389 1 flags.go:64] FLAG: --log-backtrace-at=":0" I0131 07:32:09.400394 1 flags.go:64] FLAG: --log-dir="" I0131 07:32:09.400398 1 flags.go:64] FLAG: --log-file="" I0131 07:32:09.400401 1 flags.go:64] FLAG: --log-file-max-size="1800" I0131 07:32:09.400405 1 flags.go:64] FLAG: --log-flush-frequency="5s" I0131 07:32:09.400409 1 flags.go:64] FLAG: --log-json-info-buffer-size="0" I0131 07:32:09.400416 1 flags.go:64] FLAG: --log-json-split-stream="false" I0131 07:32:09.400420 1 flags.go:64] FLAG: --logging-format="text" I0131 07:32:09.400423 1 flags.go:64] FLAG: --logtostderr="true" I0131 07:32:09.400427 1 flags.go:64] FLAG: --master="" I0131 07:32:09.400430 1 flags.go:64] FLAG: --max-endpoints-per-slice="100" I0131 07:32:09.400433 1 flags.go:64] FLAG: --min-resync-period="12h0m0s" I0131 07:32:09.400436 1 flags.go:64] FLAG: --mirroring-concurrent-service-endpoint-syncs="5" I0131 07:32:09.400439 1 flags.go:64] FLAG: --mirroring-endpointslice-updates-batch-period="0s" I0131 07:32:09.400442 1 flags.go:64] FLAG: --mirroring-max-endpoints-per-subset="1000" I0131 07:32:09.400447 1 flags.go:64] FLAG: --namespace-sync-period="5m0s" I0131 07:32:09.400451 1 flags.go:64] FLAG: --node-cidr-mask-size="0" I0131 07:32:09.400454 1 flags.go:64] FLAG: --node-cidr-mask-size-ipv4="0" I0131 07:32:09.400457 1 flags.go:64] FLAG: --node-cidr-mask-size-ipv6="0" I0131 07:32:09.400460 1 flags.go:64] FLAG: --node-eviction-rate="0.1" I0131 07:32:09.400463 1 flags.go:64] FLAG: --node-monitor-grace-period="40s" I0131 07:32:09.400467 1 flags.go:64] FLAG: --node-monitor-period="5s" I0131 07:32:09.400470 1 flags.go:64] FLAG: --node-startup-grace-period="1m0s" I0131 07:32:09.400473 1 flags.go:64] FLAG: --node-sync-period="0s" I0131 07:32:09.400476 1 flags.go:64] FLAG: --one-output="false" I0131 07:32:09.400479 1 flags.go:64] FLAG: --openshift-config="/etc/kubernetes/static-pod-resources/configmaps/config/config.yaml" I0131 07:32:09.400484 1 flags.go:64] FLAG: --permit-address-sharing="false" I0131 07:32:09.400487 1 flags.go:64] FLAG: --permit-port-sharing="false" I0131 07:32:09.400490 1 flags.go:64] FLAG: --pod-eviction-timeout="5m0s" I0131 07:32:09.400521 1 flags.go:64] FLAG: --port="0" I0131 07:32:09.400528 1 flags.go:64] FLAG: --profiling="true" I0131 07:32:09.400531 1 flags.go:64] FLAG: --pv-recycler-increment-timeout-nfs="30" I0131 07:32:09.400534 1 flags.go:64] FLAG: --pv-recycler-minimum-timeout-hostpath="60" I0131 07:32:09.400537 1 flags.go:64] FLAG: --pv-recycler-minimum-timeout-nfs="300" I0131 07:32:09.400554 1 flags.go:64] FLAG: --pv-recycler-pod-template-filepath-hostpath="/etc/kubernetes/static-pod-resources/configmaps/recycler-config/recycler-pod.yaml" I0131 07:32:09.400559 1 flags.go:64] FLAG: --pv-recycler-pod-template-filepath-nfs="/etc/kubernetes/static-pod-resources/configmaps/recycler-config/recycler-pod.yaml" I0131 07:32:09.400573 1 flags.go:64] FLAG: --pv-recycler-timeout-increment-hostpath="30" I0131 07:32:09.400580 1 flags.go:64] FLAG: --pvclaimbinder-sync-period="15s" I0131 07:32:09.400583 1 flags.go:64] FLAG: --register-retry-count="10" I0131 07:32:09.400587 1 flags.go:64] FLAG: --requestheader-allowed-names="[]" I0131 07:32:09.400591 1 flags.go:64] FLAG: --requestheader-client-ca-file="/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt" I0131 07:32:09.400595 1 flags.go:64] FLAG: --requestheader-extra-headers-prefix="[x-remote-extra-]" I0131 07:32:09.400601 1 flags.go:64] FLAG: --requestheader-group-headers="[x-remote-group]" I0131 07:32:09.400606 1 flags.go:64] FLAG: --requestheader-username-headers="[x-remote-user]" I0131 07:32:09.400613 1 flags.go:64] FLAG: --resource-quota-sync-period="5m0s" I0131 07:32:09.400616 1 flags.go:64] FLAG: --root-ca-file="/etc/kubernetes/static-pod-resources/configmaps/serviceaccount-ca/ca-bundle.crt" I0131 07:32:09.400621 1 flags.go:64] FLAG: --route-reconciliation-period="10s" I0131 07:32:09.400624 1 flags.go:64] FLAG: --secondary-node-eviction-rate="0.01" I0131 07:32:09.400629 1 flags.go:64] FLAG: --secure-port="10257" I0131 07:32:09.400632 1 flags.go:64] FLAG: --service-account-private-key-file="/etc/kubernetes/static-pod-resources/secrets/service-account-private-key/service-account.key" I0131 07:32:09.400637 1 flags.go:64] FLAG: --service-cluster-ip-range="172.30.0.0/16" I0131 07:32:09.400640 1 flags.go:64] FLAG: --show-hidden-metrics-for-version="" I0131 07:32:09.400643 1 flags.go:64] FLAG: --skip-headers="false" I0131 07:32:09.400647 1 flags.go:64] FLAG: --skip-log-headers="false" I0131 07:32:09.400652 1 flags.go:64] FLAG: --stderrthreshold="2" I0131 07:32:09.400655 1 flags.go:64] FLAG: --terminated-pod-gc-threshold="12500" I0131 07:32:09.400658 1 flags.go:64] FLAG: --tls-cert-file="/etc/kubernetes/static-pod-resources/secrets/serving-cert/tls.crt" I0131 07:32:09.400663 1 flags.go:64] FLAG: --tls-cipher-suites="[TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256]" I0131 07:32:09.400678 1 flags.go:64] FLAG: --tls-min-version="VersionTLS12" I0131 07:32:09.400683 1 flags.go:64] FLAG: --tls-private-key-file="/etc/kubernetes/static-pod-resources/secrets/serving-cert/tls.key" I0131 07:32:09.400687 1 flags.go:64] FLAG: --tls-sni-cert-key="[]" I0131 07:32:09.400693 1 flags.go:64] FLAG: --unhealthy-zone-threshold="0.55" I0131 07:32:09.400696 1 flags.go:64] FLAG: --unsupported-kube-api-over-localhost="false" I0131 07:32:09.400699 1 flags.go:64] FLAG: --use-service-account-credentials="true" I0131 07:32:09.400703 1 flags.go:64] FLAG: --v="2" I0131 07:32:09.400707 1 flags.go:64] FLAG: --version="false" I0131 07:32:09.400718 1 flags.go:64] FLAG: --vmodule="" I0131 07:32:09.400722 1 flags.go:64] FLAG: --volume-host-allow-local-loopback="true" I0131 07:32:09.400725 1 flags.go:64] FLAG: --volume-host-cidr-denylist="[]" I0131 07:32:09.402141 1 dynamic_serving_content.go:112] "Loaded a new cert/key pair" name="serving-cert::/et [knarra@knarra cucushift]$ oc logs kube-controller-manager-ip-10-0-192-9.us-east-2.compute.internal -n openshift-kube-controller-manager | grep "unrecognized feature gate" [knarra@knarra cucushift]$ [knarra@knarra cucushift]$ oc logs kube-controller-manager-ip-10-0-192-9.us-east-2.compute.internal -n openshift-kube-controller-manager | grep "unrecognized" [knarra@knarra cucushift]$ kube-apiserver logs: ==================== I0131 07:27:42.172438 16 feature_gate.go:246] feature gates: &{map[APIPriorityAndFairness:true]} I0131 07:27:42.172504 16 feature_gate.go:246] feature gates: &{map[APIPriorityAndFairness:true RotateKubeletServerCertificate:true]} I0131 07:27:42.172543 16 feature_gate.go:246] feature gates: &{map[APIPriorityAndFairness:true DownwardAPIHugePages:true RotateKubeletServerCertificate:true]} I0131 07:27:42.172599 16 feature_gate.go:246] feature gates: &{map[APIPriorityAndFairness:true DownwardAPIHugePages:true PodSecurity:true RotateKubeletServerCertificate:true]} I0131 07:27:42.172640 16 feature_gate.go:246] feature gates: &{map[APIPriorityAndFairness:true CSIMigrationAWS:false DownwardAPIHugePages:true PodSecurity:true RotateKubeletServerCertificate:true]} I0131 07:27:42.172675 16 feature_gate.go:246] feature gates: &{map[APIPriorityAndFairness:true CSIMigrationAWS:false CSIMigrationOpenStack:false DownwardAPIHugePages:true PodSecurity:true RotateKubeletServe rCertificate:true]} I0131 07:27:42.172714 16 feature_gate.go:246] feature gates: &{map[APIPriorityAndFairness:true CSIMigrationAWS:false CSIMigrationGCE:false CSIMigrationOpenStack:false DownwardAPIHugePages:true PodSecurity:t rue RotateKubeletServerCertificate:true]} I0131 07:27:42.172751 16 feature_gate.go:246] feature gates: &{map[APIPriorityAndFairness:true CSIMigrationAWS:false CSIMigrationAzureDisk:false CSIMigrationGCE:false CSIMigrationOpenStack:false DownwardAPI HugePages:true PodSecurity:true RotateKubeletServerCertificate:true]} I0131 07:27:42.172847 16 feature_gate.go:246] feature gates: &{map[APIPriorityAndFairness:true CSIMigrationAWS:false CSIMigrationAzureDisk:false CSIMigrationAzureFile:false CSIMigrationGCE:false CSIMigratio nOpenStack:false DownwardAPIHugePages:true PodSecurity:true RotateKubeletServerCertificate:true]} I0131 07:27:42.172888 16 feature_gate.go:246] feature gates: &{map[APIPriorityAndFairness:true CSIMigrationAWS:false CSIMigrationAzureDisk:false CSIMigrationAzureFile:false CSIMigrationGCE:false CSIMigratio nOpenStack:false CSIMigrationvSphere:false DownwardAPIHugePages:true PodSecurity:true RotateKubeletServerCertificate:true]} [knarra@knarra cucushift]$ oc logs kube-apiserver-ip-10-0-178-236.us-east-2.compute.internal -n openshift-kube-apiserver | grep "unrecognized feature gate" [knarra@knarra cucushift]$ [knarra@knarra cucushift]$ oc logs kube-apiserver-ip-10-0-178-236.us-east-2.compute.internal -n openshift-kube-apiserver | grep "unrecognized" [knarra@knarra cucushift]$ Based on the above moving bug to verified state.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:0056