Hide Forgot
Description of problem: attachable-volumes-csi-ebs.csi.aws.com=39 does not work for csi ebs driver, user can still attach 52 volumes to one node Version-Release number of selected component (if applicable): 4.2.0-0.nightly-2019-08-29-170426 How reproducible: Always Steps to Reproduce: 1.Creat sc foo-encrypted ebs.csi.aws.com 89m 2.Only make one worker can be scheduled 3.Create 54 pods with above sc apiVersion: apps/v1 kind: StatefulSet metadata: name: web0 spec: selector: matchLabels: app: nginx # has to match .spec.template.metadata.labels serviceName: "nginx" replicas: 54 # by default is 1 template: metadata: labels: app: nginx # has to match .spec.selector.matchLabels spec: terminationGracePeriodSeconds: 10 containers: - name: nginx image: aosqe/hello-openshift ports: - containerPort: 80 name: web volumeMounts: - name: ww0 mountPath: /usr/share/nginx/html volumeClaimTemplates: - metadata: name: ww0 spec: accessModes: [ "ReadWriteOnce" ] resources: requests: storage: 1Gi storageClassName: 'foo-encrypted' 4.Node info status: addresses: - address: 10.0.134.8 type: InternalIP - address: ip-10-0-134-8.us-east-2.compute.internal type: InternalDNS - address: ip-10-0-134-8.us-east-2.compute.internal type: Hostname allocatable: attachable-volumes-aws-ebs: "39" attachable-volumes-csi-ebs.csi.aws.com: "39" cpu: 1500m hugepages-1Gi: "0" hugepages-2Mi: "0" memory: 7548496Ki pods: "250" capacity: attachable-volumes-aws-ebs: "39" attachable-volumes-csi-ebs.csi.aws.com: "39" cpu: "2" hugepages-1Gi: "0" hugepages-2Mi: "0" memory: 8162896Ki pods: "250" Actual results: 52 pods attached to one node Expected results: Should be 39 pods attach to one node Master Log: Node Log (of failed PODs): PV Dump: PVC Dump: StorageClass Dump (if StorageClass used by PV/PVC): Additional info:
> Expected results: > Should be 39 pods attach to one node It's possible that 52 pods are scheduled in a single node, as long as they don't use more than 39 volumes. In this case there's only 1 volume shared among 52 pods, which is OK.
This StatefulSet will create 52 volumes and attach to node. But it should be 39 volumes according to the set in the node.status.capacity.attachable-volumes-csi-ebs.csi.aws.com: "39"
It is failed on 4.2.0-0.nightly-2019-09-08-180038 We still can attach 40 volumes to one node.
The last OpenShift rebase [0] reverted the patch that fixed this issue due to a potential bug in the publishing bot. I re-submitted the patch at [1]. [0] https://github.com/openshift/origin/pull/23674 [1] https://github.com/openshift/origin/pull/23756
Tested with 4.2.0-0.okd-2019-09-11-080531. Volume limit was enforced correctly: $ oc describe pod/web0-39 (...) Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning FailedScheduling 1s (x2 over 66s) default-scheduler 0/6 nodes are available: 1 node(s) exceed max volume count, 2 node(s) were unschedulable, 3 node(s) had taints that the pod didn't tolerate.
It is passed on 4.2.0-0.nightly-2019-09-15-052022 Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning FailedScheduling 88s (x9 over 13m) default-scheduler 0/5 nodes are available: 1 node(s) exceed max volume count, 1 node(s) were unschedulable, 3 node(s) had taints that the pod didn't tolerate.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:2922