Description of problem: Following tests fail in 4.11 CI: External Storage [Driver: disk.csi.azure.com] [Testpattern: Dynamic PV (default fs)] provisioning should provision storage with pvc data source in parallel [Slow] External Storage [Driver: disk.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] provisioning should provision storage with pvc data source in parallel [Slow] Events in the test namespace suggest that there might be something wrong with the CSI idempotency - first few provisioning times out and then the driver reports the volume already exists. ProvisioningFailed: failed to provision volume with StorageClass "e2e-provisioning-5808-e2e-sc8rlfx": rpc error: code = DeadlineExceeded desc = context deadline exceeded ProvisioningFailed: failed to provision volume with StorageClass "e2e-provisioning-5808-e2e-sc8rlfx": rpc error: code = DeadlineExceeded desc = context deadline exceeded ProvisioningFailed: failed to provision volume with StorageClass "e2e-provisioning-5808-e2e-sc8rlfx": rpc error: code = DeadlineExceeded desc = context deadline exceeded ProvisioningFailed: failed to provision volume with StorageClass "e2e-provisioning-5808-e2e-sc8rlfx": rpc error: code = DeadlineExceeded desc = context deadline exceeded ProvisioningFailed: failed to provision volume with StorageClass "e2e-provisioning-5808-e2e-sc8rlfx": rpc error: code = DeadlineExceeded desc = context deadline exceeded ProvisioningFailed: failed to provision volume with StorageClass "e2e-provisioning-5808-e2e-sc8rlfx": rpc error: code = Aborted desc = An operation with the given Volume ID pvc-7036e240-36f6-4a1b-b4bf-827c8e0d2d5a already exists ProvisioningFailed: failed to provision volume with StorageClass "e2e-provisioning-5808-e2e-sc8rlfx": rpc error: code = Aborted desc = An operation with the given Volume ID pvc-022fd7e2-377f-4bb8-891a-ac1d29bcae6b already exists ProvisioningFailed: failed to provision volume with StorageClass "e2e-provisioning-5808-e2e-sc8rlfx": rpc error: code = Aborted desc = An operation with the given Volume ID pvc-ae2482ad-ff83-43db-a7ec-002b083a415f already exists ProvisioningFailed: failed to provision volume with StorageClass "e2e-provisioning-5808-e2e-sc8rlfx": rpc error: code = Aborted desc = An operation with the given Volume ID pvc-7ed1f60f-fd98-434a-af94-7bf0d0b95ac7 already exists ProvisioningFailed: failed to provision volume with StorageClass "e2e-provisioning-5808-e2e-sc8rlfx": rpc error: code = Aborted desc = An operation with the given Volume ID pvc-c8aa9145-352f-45fd-9435-09b7baac3df7 already exists
Sample CI runs: https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_azure-disk-csi-driver-operator/44/pull-ci-openshift-azure-disk-csi-driver-operator-master-e2e-azurestack-csi/1501427081093844992 https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-nightly-4.11-e2e-azure-csi/1500919765546307584
I checked the sources, the driver is idempotent, it only takes > 5 minutes to provision few volumes in parallel.
I'm keeping the BZ open to skip the "parallel" tests.
The PR aboved seemed to have fixed the issue in jobs 4.11 CSI jobs (as intended), but I still see failures in Azure StackHub jobs: https://search.ci.openshift.org/?search=provisioning+should+provision+storage+with+pvc+data+source+in+parallel&maxAge=48h&context=1&type=bug%2Bissue%2Bjunit&name=azure&excludeName=&maxMatches=5&maxBytes=20971520&groupBy=job
Checked two cases are disabled in azurestack csi CI for multi releases. Mark it as Verified.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.11.3 packages and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:6287