Bug 1846357 - Machine Config Daemon Daemon Set does not set universal Toleration (and therefore gets booted if taints are set on a node)
Summary: Machine Config Daemon Daemon Set does not set universal Toleration (and there...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Machine Config Operator
Version: 4.2.z
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.4.z
Assignee: Antonio Murdaca
QA Contact: Michael Nguyen
URL:
Whiteboard:
Depends On: 1846354
Blocks: 1846358
TreeView+ depends on / blocked
 
Reported: 2020-06-11 12:38 UTC by OpenShift BugZilla Robot
Modified: 2024-03-25 16:02 UTC (History)
26 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-07-06 20:47:17 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift machine-config-operator pull 1815 0 None closed [release-4.4] Bug 1846357: MCD: tolerate all taints 2021-02-03 12:45:56 UTC
Red Hat Product Errata RHBA-2020:2786 0 None None None 2020-07-06 20:47:39 UTC

Comment 3 Michael Nguyen 2020-06-29 13:44:09 UTC 
Verified on 4.4.0-0.nightly-2020-06-29-071755.  Tolerations set to 'Operator=exists'.  Setting taints does not affect the machine config daemons running.

$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.4.0-0.nightly-2020-06-29-071755   True        False         108s    Cluster version is 4.4.0-0.nightly-2020-06-29-071755
$ export KUBECONFIG=/home/mnguyen/openshift/4.4/testcluster/auth/kubeconfig
$ oc -n openshift-machine-config-operator get ds machine-config-daemon -o yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
  annotations:
    deprecated.daemonset.template.generation: "1"
  creationTimestamp: "2020-06-29T13:15:31Z"
  generation: 1
  name: machine-config-daemon
  namespace: openshift-machine-config-operator
  resourceVersion: "18101"
  selfLink: /apis/apps/v1/namespaces/openshift-machine-config-operator/daemonsets/machine-config-daemon
  uid: a208a515-015a-47db-8877-07282513d11c
spec:
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      k8s-app: machine-config-daemon
  template:
    metadata:
      creationTimestamp: null
      labels:
        k8s-app: machine-config-daemon
      name: machine-config-daemon
    spec:
      containers:
      - args:
        - start
        command:
        - /usr/bin/machine-config-daemon
        env:
        - name: NODE_NAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: spec.nodeName
        image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3e92e2fada5baf7bf6336d366a4817394cefa0f43bf3de881c0bcb6f5c555a38
        imagePullPolicy: IfNotPresent
        name: machine-config-daemon
        resources:
          requests:
            cpu: 20m
            memory: 50Mi
        securityContext:
          privileged: true
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: FallbackToLogsOnError
        volumeMounts:
        - mountPath: /rootfs
          name: rootfs
      - args:
        - --https-address=:9001
        - --provider=openshift
        - --openshift-service-account=machine-config-daemon
        - --upstream=http://127.0.0.1:8797
        - --tls-cert=/etc/tls/private/tls.crt
        - --tls-key=/etc/tls/private/tls.key
        - --cookie-secret-file=/etc/tls/cookie-secret/cookie-secret
        - '--openshift-sar={"resource": "namespaces", "verb": "get"}'
        - '--openshift-delegate-urls={"/": {"resource": "namespaces", "verb": "get"}}'
        image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b1de153bc8773f9857efbb092f0e2756a6d253c0cb5189310641c9eda0ef8c61
        imagePullPolicy: IfNotPresent
        name: oauth-proxy
        ports:
        - containerPort: 9001
          hostPort: 9001
          name: metrics
          protocol: TCP
        resources:
          requests:
            cpu: 20m
            memory: 50Mi
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /etc/tls/private
          name: proxy-tls
        - mountPath: /etc/tls/cookie-secret
          name: cookie-secret
      dnsPolicy: ClusterFirst
      hostNetwork: true
      hostPID: true
      nodeSelector:
        kubernetes.io/os: linux
      priorityClassName: system-node-critical
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: machine-config-daemon
      serviceAccountName: machine-config-daemon
      terminationGracePeriodSeconds: 600
      tolerations:
      - operator: Exists
      volumes:
      - hostPath:
          path: /
          type: ""
        name: rootfs
      - name: proxy-tls
        secret:
          defaultMode: 420
          secretName: proxy-tls
      - name: cookie-secret
        secret:
          defaultMode: 420
          secretName: cookie-secret
  updateStrategy:
    rollingUpdate:
      maxUnavailable: 1
    type: RollingUpdate
status:
  currentNumberScheduled: 6
  desiredNumberScheduled: 6
  numberAvailable: 6
  numberMisscheduled: 0
  numberReady: 6
  observedGeneration: 1
  updatedNumberScheduled: 6
$ oc get nodes
NAME                                         STATUS   ROLES    AGE   VERSION
ip-10-0-142-91.us-west-2.compute.internal    Ready    master   26m   v1.17.1+1aa1c48
ip-10-0-156-158.us-west-2.compute.internal   Ready    worker   12m   v1.17.1+1aa1c48
ip-10-0-187-39.us-west-2.compute.internal    Ready    master   26m   v1.17.1+1aa1c48
ip-10-0-189-140.us-west-2.compute.internal   Ready    worker   12m   v1.17.1+1aa1c48
ip-10-0-209-25.us-west-2.compute.internal    Ready    master   26m   v1.17.1+1aa1c48
ip-10-0-216-105.us-west-2.compute.internal   Ready    worker   12m   v1.17.1+1aa1c48
$ oc adm taint node ip-10-0-216-105.us-west-2.compute.internal worker=reserved:NoExecute; oc adm taint node ip-10-0-216-105.us-west-2.compute.internal worker=reserved:NoSchedule
node/ip-10-0-216-105.us-west-2.compute.internal tainted
node/ip-10-0-216-105.us-west-2.compute.internal tainted
$ oc get node ip-10-0-216-105.us-west-2.compute.internal -o yaml
apiVersion: v1
kind: Node
metadata:
  annotations:
    machine.openshift.io/machine: openshift-machine-api/mnguyen44-9zf88-worker-us-west-2c-9wvn2
    machineconfiguration.openshift.io/currentConfig: rendered-worker-c659c17246ab98f74d109fb679f15a2c
    machineconfiguration.openshift.io/desiredConfig: rendered-worker-c659c17246ab98f74d109fb679f15a2c
    machineconfiguration.openshift.io/reason: ""
    machineconfiguration.openshift.io/state: Done
    volumes.kubernetes.io/controller-managed-attach-detach: "true"
  creationTimestamp: "2020-06-29T13:27:31Z"
  labels:
    beta.kubernetes.io/arch: amd64
    beta.kubernetes.io/instance-type: m5.large
    beta.kubernetes.io/os: linux
    failure-domain.beta.kubernetes.io/region: us-west-2
    failure-domain.beta.kubernetes.io/zone: us-west-2c
    kubernetes.io/arch: amd64
    kubernetes.io/hostname: ip-10-0-216-105
    kubernetes.io/os: linux
    node-role.kubernetes.io/worker: ""
    node.kubernetes.io/instance-type: m5.large
    node.openshift.io/os_id: rhcos
    topology.kubernetes.io/region: us-west-2
    topology.kubernetes.io/zone: us-west-2c
  name: ip-10-0-216-105.us-west-2.compute.internal
  resourceVersion: "23888"
  selfLink: /api/v1/nodes/ip-10-0-216-105.us-west-2.compute.internal
  uid: 251f052e-cc76-456d-80b9-776323c76140
spec:
  providerID: aws:///us-west-2c/i-0810d4543d01d299c
  taints:
  - effect: NoSchedule
    key: worker
    value: reserved
  - effect: NoExecute
    key: worker
    value: reserved
status:
  addresses:
  - address: 10.0.216.105
    type: InternalIP
  - address: ip-10-0-216-105.us-west-2.compute.internal
    type: Hostname
  - address: ip-10-0-216-105.us-west-2.compute.internal
    type: InternalDNS
  allocatable:
    attachable-volumes-aws-ebs: "25"
    cpu: 1500m
    ephemeral-storage: "114381692328"
    hugepages-1Gi: "0"
    hugepages-2Mi: "0"
    memory: 6710204Ki
    pods: "250"
  capacity:
    attachable-volumes-aws-ebs: "25"
    cpu: "2"
    ephemeral-storage: 125277164Ki
    hugepages-1Gi: "0"
    hugepages-2Mi: "0"
    memory: 7861180Ki
    pods: "250"
  conditions:
  - lastHeartbeatTime: "2020-06-29T13:40:01Z"
    lastTransitionTime: "2020-06-29T13:27:31Z"
    message: kubelet has sufficient memory available
    reason: KubeletHasSufficientMemory
    status: "False"
    type: MemoryPressure
  - lastHeartbeatTime: "2020-06-29T13:40:01Z"
    lastTransitionTime: "2020-06-29T13:27:31Z"
    message: kubelet has no disk pressure
    reason: KubeletHasNoDiskPressure
    status: "False"
    type: DiskPressure
  - lastHeartbeatTime: "2020-06-29T13:40:01Z"
    lastTransitionTime: "2020-06-29T13:27:31Z"
    message: kubelet has sufficient PID available
    reason: KubeletHasSufficientPID
    status: "False"
    type: PIDPressure
  - lastHeartbeatTime: "2020-06-29T13:40:01Z"
    lastTransitionTime: "2020-06-29T13:29:01Z"
    message: kubelet is posting ready status
    reason: KubeletReady
    status: "True"
    type: Ready
  daemonEndpoints:
    kubeletEndpoint:
      Port: 10250
  images:
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:87f5f1f1566ea8ec07e9b15ff46d1767ee49148205f26a6de6aa168d8c853d43
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:<none>
    sizeBytes: 872290769
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f0a47fb26cd52ba22dd7fc963e74c105d5b6e091ab7922a4766efb4e63f52bb6
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:<none>
    sizeBytes: 467326588
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3e92e2fada5baf7bf6336d366a4817394cefa0f43bf3de881c0bcb6f5c555a38
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:<none>
    sizeBytes: 429897331
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c28a4cf03f78374c4a18b91d14d4e886aee4aeb782301c32436889895ef88e96
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:<none>
    sizeBytes: 428823988
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:963fe01272ee5fe6deeafa453087f981733c3277944f6d103fd246ea6e21e1a6
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:<none>
    sizeBytes: 367543661
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d478d1a212bd1a2a5bd14e36e0fd740607c4097c7498d7afea028b8eec699755
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:<none>
    sizeBytes: 364474354
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b6ecb98c9a40eb5c0f69fbb66ea5c77e451e6d2c5ffc2c529781490ff9da3223
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:<none>
    sizeBytes: 342626046
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cbf4b2d6fa1a405387bea55db2a293fb196e3808d61a269b1bf5b00dfe054997
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:<none>
    sizeBytes: 338257579
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b159beb6abe48be63ae129575fcf9d552bd08f44208bcbf66d851cbf058c0f4a
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:<none>
    sizeBytes: 334914435
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7cb23c271c3b40a1733f7ae366167cdb91050a449c263811c066f582b772054c
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:<none>
    sizeBytes: 325673898
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:be8b79f4e4a00c54194d278133fbf1abcfbcd77025984039251bd582a84b4070
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:<none>
    sizeBytes: 318699317
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:569fc5846c8b59b0488823a99a17b2354bf7d532418958c06aadd177186fed15
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:<none>
    sizeBytes: 311016402
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1cfcdb3c2406c10e980eabd454ef2640877b15d6576e7dfae2beaf129ec94f03
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:<none>
    sizeBytes: 305583473
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cab7d794cde59b44fa9a3de07445961d4440f801fd854bf530852fdb5a20a914
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:<none>
    sizeBytes: 289298460
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6ccce1c6c856a113d219a284ad9bc3f89c1f49f124a42c02a9c087f1d23fe8ba
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:<none>
    sizeBytes: 283435012
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:229d2c63f30a59b446c229b37d810e23c972970e5cd496c9c62b066f8cf4c1a7
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:<none>
    sizeBytes: 276837709
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b1de153bc8773f9857efbb092f0e2756a6d253c0cb5189310641c9eda0ef8c61
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:<none>
    sizeBytes: 267611081
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72b80cd480bb49dfa28ce72e082d69d62b23594db2c0c9e6a0a35ddbf477e4de
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:<none>
    sizeBytes: 258193954
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0d8469257c72f4d4deb8f2d72802488645360182a186f43c21746202b09f41ad
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:<none>
    sizeBytes: 257198663
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:40bb48291f2937c1a0bf84eee5f26fc98d400e5242ee75afc4e6a3c5b75e7a03
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:<none>
    sizeBytes: 255814539
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cec70cc0697d893d2b5c83b11e0888488a6c7556a945da11b59015ad69f98a62
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:<none>
    sizeBytes: 251028388
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:47b150b2d79c5eebbf6e9bd189daa4c5d7a412c926ba258898b0af071c33f6e9
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:<none>
    sizeBytes: 243281399
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:52686d7a2b9ffbc8619ba2a4835dd91bab71f52a4a959144ec7bde5a3ed04004
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:<none>
    sizeBytes: 237302018
  nodeInfo:
    architecture: amd64
    bootID: dd1cf7b0-c56a-4821-ba0e-f0ff97893398
    containerRuntimeVersion: cri-o://1.17.4-18.dev.rhaos4.4.gitfb8131a.el8
    kernelVersion: 4.18.0-147.20.1.el8_1.x86_64
    kubeProxyVersion: v1.17.1+1aa1c48
    kubeletVersion: v1.17.1+1aa1c48
    machineID: ec24db13f9f65cdd340e1fda9c7fad61
    operatingSystem: linux
    osImage: Red Hat Enterprise Linux CoreOS 44.81.202006271430-0 (Ootpa)
    systemUUID: ec24db13-f9f6-5cdd-340e-1fda9c7fad61
$ oc -n openshift-machine-config-operator get ds machine-config-daemon 
NAME                    DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
machine-config-daemon   6         6         6       6            6           kubernetes.io/os=linux   26m
Comment 6 errata-xmlrpc 2020-07-06 20:47:17 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2786
Note You need to log in before you can comment on or make changes to this bug.