Description of problem: Thanks to the work included on https://bugzilla.redhat.com/show_bug.cgi?id=2065597, now it is possible to configure cinder-csi by editing cloud-provider-config. One of the parameters included is 'node-volume-attach-limit' [1]. Changing the param to 10 and running 11 pods connected to 11 different PVCs in the same worker is accepted: $ oc get cm -n openshift-cluster-csi-drivers cloud-conf -o yaml apiVersion: v1 data: cloud.conf: |+ [Global] region = regionOne ca-file = /etc/kubernetes/static-pod-resources/configmaps/cloud-config/ca-bundle.pem use-clouds = true clouds-file = /etc/kubernetes/secret/clouds.yaml cloud = openstack [LoadBalancer] use-octavia = True [BlockStorage] ignore-volume-az = yes node-volume-attach-limit = 10 [Metadata] search-order = metadataService $ oc get pods NAME READY STATUS RESTARTS AGE demo-0-8466fc476-mbll5 1/1 Running 0 2m17s demo-1-649948d68-dnhsv 1/1 Running 0 2m14s demo-10-676c554976-52bwt 1/1 Running 0 103s demo-2-6947d8d46d-mtxn6 1/1 Running 0 2m11s demo-3-79db5fd4d8-qxgbl 1/1 Running 0 2m7s demo-4-65678d5484-qlngk 1/1 Running 0 2m4s demo-5-f75648d76-wzcmw 1/1 Running 0 2m demo-6-856dc89dd8-s9bp4 1/1 Running 0 117s demo-7-64f5944f8d-shq4d 1/1 Running 0 114s demo-8-7b677bb644-4bhnn 1/1 Running 0 111s demo-9-9b88b4c56-m7swp 1/1 Running 0 107s $ oc get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE pvc-0 Bound pvc-92aaeab8-6c60-4f32-9ab6-a703761ad3cd 1Gi RWO topology-aware-0 2m19s pvc-1 Bound pvc-336915af-8154-4e67-b37d-2fa93f2db645 1Gi RWO topology-aware-1 2m16s pvc-10 Bound pvc-a7f58825-e465-4e2b-b9ec-36362589d1d2 1Gi RWO topology-aware-10 105s pvc-2 Bound pvc-4ded3dab-170d-4573-ba29-8bb10b3e857b 1Gi RWO topology-aware-2 2m13s pvc-3 Bound pvc-b3d7ec16-5b7e-4964-9a0a-29480dee31a1 1Gi RWO topology-aware-3 2m9s pvc-4 Bound pvc-a6ae7415-00b5-4849-8065-7fe7acace165 1Gi RWO topology-aware-4 2m6s pvc-5 Bound pvc-246421b3-2716-41c8-88e9-d6253c49acdd 1Gi RWO topology-aware-5 2m2s pvc-6 Bound pvc-61ec111a-d5ae-47d2-83be-1a4a344afea7 1Gi RWO topology-aware-6 119s pvc-7 Bound pvc-b65cd362-977a-4613-84a3-bb75b1adf971 1Gi RWO topology-aware-7 116s pvc-8 Bound pvc-a79d7bb1-9697-4c1b-8ad3-696a0c2edafd 1Gi RWO topology-aware-8 113s pvc-9 Bound pvc-48e37bc7-ff50-4517-8cac-a57fd6f0f47b 1Gi RWO topology-aware-9 109s $ openstack volume list -c 'Attached to' | grep ostest | awk -F' ' '{print $4}' | sort | uniq -c 11 ostest-zh8g7-worker-0-7gl4r $ openstack volume list +--------------------------------------+------------------------------------------+--------+------+------------------------------------------------------+ | ID | Name | Status | Size | Attached to | +--------------------------------------+------------------------------------------+--------+------+------------------------------------------------------+ | ce816f7d-6873-457a-acfa-327cf9357c4a | pvc-a7f58825-e465-4e2b-b9ec-36362589d1d2 | in-use | 1 | Attached to ostest-zh8g7-worker-0-7gl4r on /dev/vdl | | dd774cc3-488c-4fa7-bf5d-f84c3d67f99b | pvc-48e37bc7-ff50-4517-8cac-a57fd6f0f47b | in-use | 1 | Attached to ostest-zh8g7-worker-0-7gl4r on /dev/vdk | | 1fc32fa8-feeb-4f06-a712-5b9da301e187 | pvc-a79d7bb1-9697-4c1b-8ad3-696a0c2edafd | in-use | 1 | Attached to ostest-zh8g7-worker-0-7gl4r on /dev/vdj | | 0ced70af-0a85-4cd2-bfab-f7299b4bd128 | pvc-b65cd362-977a-4613-84a3-bb75b1adf971 | in-use | 1 | Attached to ostest-zh8g7-worker-0-7gl4r on /dev/vdi | | dc413629-382b-492b-9e41-ae9163ff542f | pvc-61ec111a-d5ae-47d2-83be-1a4a344afea7 | in-use | 1 | Attached to ostest-zh8g7-worker-0-7gl4r on /dev/vdh | | 7aab9069-dc48-4413-b4b9-f7c31bdaeaa6 | pvc-246421b3-2716-41c8-88e9-d6253c49acdd | in-use | 1 | Attached to ostest-zh8g7-worker-0-7gl4r on /dev/vdg | | c1656f81-3d55-4f15-aab8-d84b37ceae9b | pvc-a6ae7415-00b5-4849-8065-7fe7acace165 | in-use | 1 | Attached to ostest-zh8g7-worker-0-7gl4r on /dev/vdf | | d8608274-5a75-4daf-adb5-575160b296e3 | pvc-b3d7ec16-5b7e-4964-9a0a-29480dee31a1 | in-use | 1 | Attached to ostest-zh8g7-worker-0-7gl4r on /dev/vde | | c59bc638-f410-4d3e-a3b9-d31288a0c4a0 | pvc-4ded3dab-170d-4573-ba29-8bb10b3e857b | in-use | 1 | Attached to ostest-zh8g7-worker-0-7gl4r on /dev/vdd | | 7f779063-61f1-462b-bb9d-f77429dc9485 | pvc-336915af-8154-4e67-b37d-2fa93f2db645 | in-use | 1 | Attached to ostest-zh8g7-worker-0-7gl4r on /dev/vdc | | 2c17fa64-1b3a-44e1-8ad0-8fb5c4e3ffc1 | pvc-92aaeab8-6c60-4f32-9ab6-a703761ad3cd | in-use | 1 | Attached to ostest-zh8g7-worker-0-7gl4r on /dev/vdb | +--------------------------------------+------------------------------------------+--------+------+------------------------------------------------------+ [1] https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/cinder-csi-plugin/using-cinder-csi-plugin.md#driver-config Version-Release number of selected component (if applicable): 4.11.0-0.nightly-2022-06-06-025509 RHOS-16.2-RHEL-8-20220311.n.1 How reproducible: Always Steps to Reproduce: 1. Change cloud cm to node-volume-attach-limit = 10 2. Wait until cluster re-configuration is completed (around 30 minutes) 3. Create 11 pods using cinder PVCs with NodeSelector set to use always the same worker. Actual results: The node-volume-attach-limit is ignored. Expected results: The PVC number 11 should not becreated.
Okay, a simple workaround to the problem is to delete CSINode objects and restart kubelet on every of the nodes. Then the node-volume-attach-limit is correctly put there and everything will work find. We don't understand the root cause yet, but clearly it's an issue with the fact you're updating the configuration after the installation.
OpenShift has moved to Jira for its defect tracking! This bug can now be found in the OCPBUGS project in Jira. https://issues.redhat.com/browse/OCPBUGS-9308