Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1774278

Summary:	Taints and Tolerations are not working with Local Storage Operator
Product:	OpenShift Container Platform	Reporter:	Dan Yocum <dyocum>
Component:	Storage	Assignee:	Christian Huffman <chuffman>
Status:	CLOSED ERRATA	QA Contact:	Liang Xia <lxia>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	4.2.z	CC:	aos-bugs, chuffman, jmalde, jsafrane, lxia, nthomas, tsmetana
Target Milestone:	---
Target Release:	4.4.0
Hardware:	All
OS:	All
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:
Clones:	1779189 (view as bug list)		Environment:
Last Closed:	2020-05-13 21:53:01 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1783863
Bug Blocks:	1779189

Description Dan Yocum 2019-11-19 23:36:39 UTC

Description of problem:

When attempting to create a LocalVolume for logging ElasticSearch pod storage, the LocalVolume is not being created on the nodes labeled as 'infra' using toleration or taints.

Version-Release number of selected component (if applicable):

4.2.x

How reproducible:

Always

Steps to Reproduce:
1. Follow this guide: 

https://docs.openshift.com/container-platform/4.2/storage/persistent-storage/persistent-storage-local.html#create-local-pvc_persistent-storage-local

apiVersion: local.storage.openshift.io/v1
kind: LocalVolume
metadata:
  name: local-disks
  namespace: local-storage
spec:
  tolerations:
  - key: "localstorage"
    operator: "Exists"
  storageClassDevices:
    - storageClassName: local
      volumeMode: Block
      devicePaths:
        - /dev/xvdf

2. Follow this guide: 

https://docs.openshift.com/container-platform/4.2/logging/config/cluster-logging-tolerations.html#cluster-logging-elasticsearch-tolerations_cluster-logging-tolerations

4. Attempt to create volume

Actual results:

Fails

Expected results:

Succeeds
Additional info:

Comment 7 Christian Huffman 2019-12-02 17:49:37 UTC

We currently don't propagate Tolerations defined in the LocalVolume CR to the created diskmaker or provisioner DaemonSets. Due to this, it's not possible to have a Managed LocalVolume resource that respects Node taints.

I've submitted a PR [1] that includes Tolerations in the LocalVolumeSpec, and allows this behavior to function as the user expects.

[1] https://github.com/openshift/local-storage-operator/pull/59

Comment 10 Liang Xia 2019-12-19 07:23:00 UTC

Verified with local-storage-operator.4.4.0-201912170523 on 4.4.0-0.nightly-2019-12-18-064553


$ oc get node --selector=node-role.kubernetes.io/worker=
NAME                        STATUS   ROLES    AGE     VERSION
lxia19-48fdq-worker-2pj5w   Ready    worker   4h9m    v1.16.2
lxia19-48fdq-worker-6fgxw   Ready    worker   4h7m    v1.16.2
lxia19-48fdq-worker-dsf5s   Ready    worker   4h12m   v1.16.2



$ oc adm taint nodes lxia19-48fdq-worker-2pj5w disks=localvolume:NoExecute
node/lxia19-48fdq-worker-2pj5w tainted

$ oc adm taint nodes lxia19-48fdq-worker-6fgxw disks=dynamic:NoExecute
node/lxia19-48fdq-worker-6fgxw tainted



$ oc get all
NAME                                       READY   STATUS    RESTARTS   AGE
pod/local-storage-operator-89f4b74-x22hc   1/1     Running   0          81m

NAME                             TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)     AGE
service/local-storage-operator   ClusterIP   172.30.127.222   <none>        60000/TCP   80m

NAME                                     READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/local-storage-operator   1/1     1            1           86m

NAME                                                DESIRED   CURRENT   READY   AGE
replicaset.apps/local-storage-operator-684bcf594c   0         0         0       86m
replicaset.apps/local-storage-operator-89f4b74      1         1         1       81m



$ oc get sc,pv
NAME                                             PROVISIONER                    AGE
storageclass.storage.k8s.io/local-sc-taints      kubernetes.io/no-provisioner   115s
storageclass.storage.k8s.io/standard (default)   kubernetes.io/cinder           4h25m

NAME                                 CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM   STORAGECLASS      REASON   AGE
persistentvolume/local-pv-365ad695   2Gi        RWO            Delete           Available           local-sc-taints            98s
persistentvolume/local-pv-58433e2d   1Gi        RWO            Delete           Available           local-sc-taints            97s

Comment 11 Dan Yocum 2020-01-17 20:33:05 UTC

Will there be backport to 4.3.z?

Comment 12 Liang Xia 2020-01-19 02:02:19 UTC

(In reply to Dan Yocum from comment #11)
> Will there be backport to 4.3.z?

It's in 4.3.0, https://bugzilla.redhat.com/show_bug.cgi?id=1779189

Comment 14 errata-xmlrpc 2020-05-13 21:53:01 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0581