Bug 2035986 - Some pods under kube-scheduler/kube-controller-manager are using the deprecated annotation
Summary: Some pods under kube-scheduler/kube-controller-manager are using the deprecat...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: kube-scheduler
Version: 4.10
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 4.10.0
Assignee: Ross Peoples
QA Contact: RamaKasturi
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-12-29 05:59 UTC by RamaKasturi
Modified: 2022-03-10 16:37 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-03-10 16:36:47 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift cluster-kube-apiserver-operator pull 1288 0 None open Bug 2035986: Replace deprecated annotation 2022-01-05 15:22:52 UTC
Github openshift cluster-kube-controller-manager-operator pull 586 0 None open Bug 2035986: Replace deprecated annotation 2022-01-05 14:54:16 UTC
Github openshift cluster-kube-descheduler-operator pull 235 0 None open Bug 2035986: Add default-container annotation 2022-01-05 15:41:25 UTC
Github openshift cluster-kube-scheduler-operator pull 393 0 None open Bug 2035986: Replace deprecated annotation 2022-01-05 15:28:12 UTC
Red Hat Product Errata RHSA-2022:0056 0 None None None 2022-03-10 16:36:59 UTC

Description RamaKasturi 2021-12-29 05:59:23 UTC
Description of problem:
Some pods under openshift-kube-scheduler/openshift-kube-controller-manager are using the deprecated annotation ‘kubectl.kubernetes.io/default-logs-container’ instead of ‘kubectl.kubernetes.io/default-container’

# oc -n openshift-kube-scheduler logs openshift-kube-scheduler-ip-10-0-134-157.us-east-2.compute.internal | head
Using deprecated annotation `kubectl.kubernetes.io/default-logs-container` in pod/openshift-kube-scheduler-ip-10-0-134-157.us-east-2.compute.internal. Please use `kubectl.kubernetes.io/default-container` instead
...

# oc -n openshift-kube-scheduler get po openshift-kube-scheduler-ip-10-0-134-157.us-east-2.compute.internal -oyaml | grep "kubectl.kubernetes.io/default-logs-container"
    kubectl.kubernetes.io/default-logs-container: kube-scheduler

# [knarra@knarra ~]$ oc -n openshift-kube-controller-manager logs kube-controller-manager-knarra28153925-p6jh7-master-0 | head
Using deprecated annotation `kubectl.kubernetes.io/default-logs-container` in pod/kube-controller-manager-knarra28153925-p6jh7-master-0. Please use `kubectl.kubernetes.io/default-container` instead

# [knarra@knarra ~]$ oc get pod kube-controller-manager-knarra28153925-p6jh7-master-0 -oyaml -n openshift-kube-controller-manager | grep "kubectl.kubernetes.io/default-logs-container"
    kubectl.kubernetes.io/default-logs-container: kube-controller-manager


Version-Release number of selected component (if applicable):
4.10.0-0.nightly-2021-12-20-231053

How reproducible:
Always

Steps to Reproduce:
1. Install 4.10/ 4.9 cluster
2. Run the command “oc -n openshift-kube-scheduler/openshift-kube-controller-manager logs  <scheduler_pod/kube_controller_manager_pod>” | head
3. Run the command “oc get pod <kube-controller-manager-pod/kube-scheduler-pod> -oyaml -n openshift-kube-controller-manager/openshift-kube-scheduler | grep “kubectl.kubernetes.io/default-logs-container”

Actual results:
Openshift-kube-scheduler & openshift-kube-controller-manager used the deprecated annotation kubectl.kubernetes.io/default-logs-container

Expected results:
Openshift-kube-scheduler & openshift-kube-controller-manager should use the right annotation kubectl.kubernetes.io/default-container


Additional info:
similar issue is seen in 4.9 as well

Comment 5 RamaKasturi 2022-01-31 09:10:20 UTC
Verified bug with the build below and i see that ks, kcm, kas are no longer using the deprecated annotation ‘kubectl.kubernetes.io/default-logs-container’  instead they are using `kubectl.kubernetes.io/default-container`

[knarra@knarra cucushift]$ oc get clusterversion
NAME      VERSION                              AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.10.0-0.nightly-2022-01-31-012936   True        False         95m     Cluster version is 4.10.0-0.nightly-2022-01-31-012936

kube-scheduler:
==========================
[knarra@knarra cucushift]$ oc -n openshift-kube-scheduler logs openshift-kube-scheduler-ip-10-0-140-122.us-east-2.compute.internal | head
I0131 07:27:32.571846       1 flags.go:64] FLAG: --add-dir-header="false"
I0131 07:27:32.571941       1 flags.go:64] FLAG: --address="127.0.0.1"
I0131 07:27:32.571947       1 flags.go:64] FLAG: --allow-metric-labels="[]"
I0131 07:27:32.571957       1 flags.go:64] FLAG: --alsologtostderr="false"
I0131 07:27:32.571960       1 flags.go:64] FLAG: --authentication-kubeconfig="/etc/kubernetes/static-pod-resources/configmaps/scheduler-kubeconfig/kubeconfig"
I0131 07:27:32.571967       1 flags.go:64] FLAG: --authentication-skip-lookup="false"
I0131 07:27:32.571972       1 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="10s"
I0131 07:27:32.571976       1 flags.go:64] FLAG: --authentication-tolerate-lookup-failure="true"
I0131 07:27:32.572004       1 flags.go:64] FLAG: --authorization-always-allow-paths="[/healthz,/readyz,/livez]"
I0131 07:27:32.572014       1 flags.go:64] FLAG: --authorization-kubeconfig="/etc/kubernetes/static-pod-resources/configmaps/scheduler-kubeconfig/kubeconfig"
[knarra@knarra cucushift]$ oc -n openshift-kube-scheduler get po openshift-kube-scheduler-ip-10-0-140-122.us-east-2.compute.internal -o yaml | grep "kubectl.kubernetes.io/default-logs-container"
[knarra@knarra cucushift]$ oc -n openshift-kube-scheduler get po openshift-kube-scheduler-ip-10-0-140-122.us-east-2.compute.internal -o yaml | grep "kubectl.kubernetes.io/default-container"
    kubectl.kubernetes.io/default-container: kube-scheduler


kube-controller-manager:
============================
[knarra@knarra cucushift]$ oc -n openshift-kube-controller-manager logs kube-controller-manager-ip-10-0-192-9.us-east-2.compute.internal | head
+ timeout 3m /bin/bash -exuo pipefail -c 'while [ -n "$(ss -Htanop \( sport = 10257 \))" ]; do sleep 1; done'
++ ss -Htanop '(' sport = 10257 ')'
+ '[' -n '' ']'
+ '[' -f /etc/kubernetes/static-pod-certs/configmaps/trusted-ca-bundle/ca-bundle.crt ']'
Copying system trust bundle
+ echo 'Copying system trust bundle'
+ cp -f /etc/kubernetes/static-pod-certs/configmaps/trusted-ca-bundle/ca-bundle.crt /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
+ '[' -f /etc/kubernetes/static-pod-resources/configmaps/cloud-config/ca-bundle.pem ']'
+ exec hyperkube kube-controller-manager --openshift-config=/etc/kubernetes/static-pod-resources/configmaps/config/config.yaml --kubeconfig=/etc/kubernetes/static-pod-resources/configmaps/controller-manager-kubeconfig/kubeconfig --authentication-kubeconfig=/etc/kubernetes/static-pod-resources/configmaps/controller-manager-kubeconfig/kubeconfig --authorization-kubeconfig=/etc/kubernetes/static-pod-resources/configmaps/controller-manager-kubeconfig/kubeconfig --client-ca-file=/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt --requestheader-client-ca-file=/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt -v=2 --tls-cert-file=/etc/kubernetes/static-pod-resources/secrets/serving-cert/tls.crt --tls-private-key-file=/etc/kubernetes/static-pod-resources/secrets/serving-cert/tls.key --allocate-node-cidrs=false --cert-dir=/var/run/kubernetes --cloud-provider=aws --cluster-cidr=10.128.0.0/14 --cluster-name=knarra0131-zbwqr --cluster-signing-cert-file=/etc/kubernetes/static-pod-certs/secrets/csr-signer/tls.crt --cluster-signing-duration=720h --cluster-signing-key-file=/etc/kubernetes/static-pod-certs/secrets/csr-signer/tls.key --configure-cloud-routes=false '--controllers=*' --controllers=-bootstrapsigner --controllers=-tokencleaner --controllers=-ttl --enable-dynamic-provisioning=true --feature-gates=APIPriorityAndFairness=true --feature-gates=CSIMigrationAWS=false --feature-gates=CSIMigrationAzureDisk=false --feature-gates=CSIMigrationAzureFile=false --feature-gates=CSIMigrationGCE=false --feature-gates=CSIMigrationOpenStack=false --feature-gates=CSIMigrationvSphere=false --feature-gates=DownwardAPIHugePages=true --feature-gates=PodSecurity=true --feature-gates=RotateKubeletServerCertificate=true --flex-volume-plugin-dir=/etc/kubernetes/kubelet-plugins/volume/exec --kube-api-burst=300 --kube-api-qps=150 --leader-elect-resource-lock=leases --leader-elect-retry-period=3s --leader-elect=true --pv-recycler-pod-template-filepath-hostpath=/etc/kubernetes/static-pod-resources/configmaps/recycler-config/recycler-pod.yaml --pv-recycler-pod-template-filepath-nfs=/etc/kubernetes/static-pod-resources/configmaps/recycler-config/recycler-pod.yaml --root-ca-file=/etc/kubernetes/static-pod-resources/configmaps/serviceaccount-ca/ca-bundle.crt --secure-port=10257 --service-account-private-key-file=/etc/kubernetes/static-pod-resources/secrets/service-account-private-key/service-account.key --service-cluster-ip-range=172.30.0.0/16 --use-service-account-credentials=true --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 --tls-min-version=VersionTLS12
I0131 07:32:09.399916       1 flags.go:64] FLAG: --add-dir-header="false"
[knarra@knarra cucushift]$ oc -n openshift-kube-controller-manager get pod kube-controller-manager-ip-10-0-192-9.us-east-2.compute.internal -o yaml | grep "kubectl.kubernetes.io/default-logs-container"
[knarra@knarra cucushift]$ oc -n openshift-kube-controller-manager get pod kube-controller-manager-ip-10-0-192-9.us-east-2.compute.internal -o yaml | grep "kubectl.kubernetes.io/default-container"
    kubectl.kubernetes.io/default-container: kube-controller-manager

kube-apiserver:
============================
[knarra@knarra cucushift]$ oc -n openshift-kube-apiserver logs kube-apiserver-ip-10-0-178-236.us-east-2.compute.internal | head
flock: getting lock took 0.000005 seconds
Copying system trust bundle ...
I0131 07:27:42.092198       1 loader.go:372] Config loaded from file:  /etc/kubernetes/static-pod-resources/configmaps/kube-apiserver-cert-syncer-kubeconfig/kubeconfig
Copying termination logs to "/var/log/kube-apiserver/termination.log"
I0131 07:27:42.092863       1 main.go:161] Touching termination lock file "/var/log/kube-apiserver/.terminating"
I0131 07:27:42.093167       1 main.go:219] Launching sub-process "/usr/bin/hyperkube kube-apiserver --openshift-config=/etc/kubernetes/static-pod-resources/configmaps/config/config.yaml --advertise-address=10.0.178.236 -v=2 --permit-address-sharing"
Flag --openshift-config has been deprecated, to be removed
I0131 07:27:42.168774      16 flags.go:64] FLAG: --add-dir-header="false"
I0131 07:27:42.168852      16 flags.go:64] FLAG: --address="127.0.0.1"
I0131 07:27:42.168874      16 flags.go:64] FLAG: --admission-control="[]"
[knarra@knarra cucushift]$ oc -n openshift-kube-apiserver get pod kube-apiserver-ip-10-0-178-236.us-east-2.compute.internal -o yaml | grep "kubectl.kubernetes.io/default-logs-container"
[knarra@knarra cucushift]$ oc -n openshift-kube-apiserver get pod kube-apiserver-ip-10-0-178-236.us-east-2.compute.internal -o yaml | grep "kubectl.kubernetes.io/default-container"
    kubectl.kubernetes.io/default-container: kube-apiserver


Based on the above moving bug to verified state.

Comment 8 errata-xmlrpc 2022-03-10 16:36:47 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:0056


Note You need to log in before you can comment on or make changes to this bug.