Description of problem: When attempting to create a LocalVolume for logging ElasticSearch pod storage, the LocalVolume is not being created on the nodes labeled as 'infra' using toleration or taints. Version-Release number of selected component (if applicable): 4.2.x How reproducible: Always Steps to Reproduce: 1. Follow this guide: https://docs.openshift.com/container-platform/4.2/storage/persistent-storage/persistent-storage-local.html#create-local-pvc_persistent-storage-local apiVersion: local.storage.openshift.io/v1 kind: LocalVolume metadata: name: local-disks namespace: local-storage spec: tolerations: - key: "localstorage" operator: "Exists" storageClassDevices: - storageClassName: local volumeMode: Block devicePaths: - /dev/xvdf 2. Follow this guide: https://docs.openshift.com/container-platform/4.2/logging/config/cluster-logging-tolerations.html#cluster-logging-elasticsearch-tolerations_cluster-logging-tolerations 4. Attempt to create volume Actual results: Fails Expected results: Succeeds Additional info:
We currently don't propagate Tolerations defined in the LocalVolume CR to the created diskmaker or provisioner DaemonSets. Due to this, it's not possible to have a Managed LocalVolume resource that respects Node taints. I've submitted a PR [1] that includes Tolerations in the LocalVolumeSpec, and allows this behavior to function as the user expects. [1] https://github.com/openshift/local-storage-operator/pull/59
Verified with local-storage-operator.4.4.0-201912170523 on 4.4.0-0.nightly-2019-12-18-064553 $ oc get node --selector=node-role.kubernetes.io/worker= NAME STATUS ROLES AGE VERSION lxia19-48fdq-worker-2pj5w Ready worker 4h9m v1.16.2 lxia19-48fdq-worker-6fgxw Ready worker 4h7m v1.16.2 lxia19-48fdq-worker-dsf5s Ready worker 4h12m v1.16.2 $ oc adm taint nodes lxia19-48fdq-worker-2pj5w disks=localvolume:NoExecute node/lxia19-48fdq-worker-2pj5w tainted $ oc adm taint nodes lxia19-48fdq-worker-6fgxw disks=dynamic:NoExecute node/lxia19-48fdq-worker-6fgxw tainted $ oc get all NAME READY STATUS RESTARTS AGE pod/local-storage-operator-89f4b74-x22hc 1/1 Running 0 81m NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/local-storage-operator ClusterIP 172.30.127.222 <none> 60000/TCP 80m NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/local-storage-operator 1/1 1 1 86m NAME DESIRED CURRENT READY AGE replicaset.apps/local-storage-operator-684bcf594c 0 0 0 86m replicaset.apps/local-storage-operator-89f4b74 1 1 1 81m $ oc get sc,pv NAME PROVISIONER AGE storageclass.storage.k8s.io/local-sc-taints kubernetes.io/no-provisioner 115s storageclass.storage.k8s.io/standard (default) kubernetes.io/cinder 4h25m NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE persistentvolume/local-pv-365ad695 2Gi RWO Delete Available local-sc-taints 98s persistentvolume/local-pv-58433e2d 1Gi RWO Delete Available local-sc-taints 97s
Will there be backport to 4.3.z?
(In reply to Dan Yocum from comment #11) > Will there be backport to 4.3.z? It's in 4.3.0, https://bugzilla.redhat.com/show_bug.cgi?id=1779189
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:0581