Bug 1876935 - AWS volume snapshot is not deleted after the cluster is destroyed
Summary: AWS volume snapshot is not deleted after the cluster is destroyed
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Storage
Version: 4.6
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: 4.7.0
Assignee: Fabio Bertinatto
QA Contact: Wei Duan
Depends On:
TreeView+ depends on / blocked
Reported: 2020-09-08 14:39 UTC by Qin Ping
Modified: 2021-02-24 15:17 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Last Closed: 2021-02-24 15:17:26 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Github openshift aws-ebs-csi-driver pull 174 0 None closed Bug 1876935: UPSTREAM: 568: Apply extra volume tags to EBS snapshots 2021-01-13 15:06:17 UTC
Red Hat Product Errata RHSA-2020:5633 0 None None None 2021-02-24 15:17:46 UTC

Description Qin Ping 2020-09-08 14:39:18 UTC
Description of problem:
volume snapshot is not deleted after the cluster is destroyed

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. Launched a 4.6 OCP cluster on AWS
2. Created a volumesnapshot for a PVC
3. Didn't delete the volumesnapshotcontent
4. Destroy cluster

Actual results:
Checked from the AWS UI, the volume snapshot is not deleted.

Expected results:
Deleted the volume snapshot when cluster is destroyed.

Master Log:

Node Log (of failed PODs):

PV Dump:

PVC Dump:

StorageClass Dump (if StorageClass used by PV/PVC):

Additional info:

Comment 4 Jan Safranek 2020-09-09 13:40:57 UTC
We need to take some holistic approach what to do with volumes / snapshots when a cluster is destroyed. Volumes of some in-tree volume plugins are destroyed (AWS), some are not (vSphere?).

Comment 5 Jan Safranek 2020-09-17 15:53:57 UTC
Result: 'openshift-install destroy cluster' should delete volumes & snapshots provisioned in the cluster.

Therefore we need:
1. The CSI driver to label snapshots accordingly
2. openshift-install to list & delete them (this may be already implemented).

Comment 6 Fabio Bertinatto 2020-10-01 09:41:07 UTC
CSI Driver patch (in review): https://github.com/kubernetes-sigs/aws-ebs-csi-driver/pull/568

The library-go patch (in review): https://github.com/openshift/library-go/pull/909

Once those are merged we need to merge the operator patch here: https://github.com/openshift/aws-ebs-csi-driver-operator/pull/83

Comment 9 Wei Duan 2020-10-21 04:38:48 UTC
Please ignore my previous comments (https://bugzilla.redhat.com/show_bug.cgi?id=1876935#c8), I mixed with another bug.
Verified pass on 4.7.0-0.nightly-2020-10-17-034503, after deleteing the cluster, the previously created snapshot on aws UI are removed accordingly, changed the status to VERIFIED.

Comment 12 errata-xmlrpc 2021-02-24 15:17:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.