Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1730339

Summary:	Potential volume leak in CSI external provisioner
Product:	OpenShift Container Platform	Reporter:	Jan Safranek <jsafrane>
Component:	Storage	Assignee:	Jan Safranek <jsafrane>
Status:	CLOSED ERRATA	QA Contact:	Chao Yang <chaoyang>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	4.2.0	CC:	aos-bugs, aos-storage-staff, srangana
Target Milestone:	---
Target Release:	4.2.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-10-16 06:29:44 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Jan Safranek 2019-07-16 13:48:43 UTC

CSI external provisioner may leak volumes after timeouts from a CSI driver. See https://github.com/kubernetes-csi/external-provisioner/issues/311 for details.

It's complicated to reproduce with a testing mock driver and it's almost impossible with a real driver (unless the driver takes really long time to provision a volume).

1. Set up a CSI driver.
2. Create a PVC to dynamically provision a PV.
3. The CSI driver must time out while provisioning a PV (!)
4. While the volume is being provisioned, delete the PVC.

Actual result:
The provisioner stops provisioning of the volume when PVC is deleted. The driver finishes provisioning, but no PV is created for the volume -> the volume is orphaned in the storage backend.

Expected result:
external-provisioner continues provisioning the volume even after PVC is deleted. Eventually, a PV is created for the volume and it's immediately deleted. As result, there is no orphan volume in the storage backend.

Fix: https://github.com/kubernetes-csi/external-provisioner/pull/312
We need the fix in 4.2

Comment 1 Jan Safranek 2019-07-16 14:25:30 UTC

OCP 4.2 / OKD PR: https://github.com/openshift/csi-external-provisioner/pull/13

Comment 3 Chao Yang 2019-08-13 08:17:42 UTC

It is passed on the 4.2.0-0.nightly-2019-08-08-002434apiVersion: storage.k8s.io/v1

Dynamic create a 10000Gi csi ebs volume, and delete the pvc.

kind: StorageClass
metadata:
  name: test
parameters:
  fsType: ext4
  csi.storage.k8s.io/provisioner-secret-name: aws-creds
  csi.storage.k8s.io/provisioner-secret-namespace: kube-system
provisioner: ebs.csi.aws.com
reclaimPolicy: Delete

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: test
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 10000Gi
  storageClassName: 'test'


oc create -f pvc-foo.yaml;oc delete pvc test;oc get pv -w
persistentvolumeclaim/test created
persistentvolumeclaim "test" deleted
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS    CLAIM          STORAGECLASS   REASON   AGE
pvc-d19f75e9-bda1-11e9-b187-0abf16ed4052   10000Gi    RWO            Delete           Pending   default/test   test                    1s
pvc-d19f75e9-bda1-11e9-b187-0abf16ed4052   10000Gi    RWO            Delete           Released   default/test   test                    1s
pvc-d19f75e9-bda1-11e9-b187-0abf16ed4052   10000Gi    RWO            Delete           Terminating   default/test   test                    1s
pvc-d19f75e9-bda1-11e9-b187-0abf16ed4052   10000Gi    RWO            Delete           Terminating   default/test   test                    1s

Comment 4 errata-xmlrpc 2019-10-16 06:29:44 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2922