Bug 1965468 - [OSP] Delete volume snapshots based on cluster ID in their metadata
Summary: [OSP] Delete volume snapshots based on cluster ID in their metadata
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 4.8
Hardware: Unspecified
OS: Unspecified
low
low
Target Milestone: ---
: 4.11.0
Assignee: Stephen Finucane
QA Contact: Jon Uriarte
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-05-27 18:33 UTC by Mike Fedosin
Modified: 2022-08-10 10:36 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-08-10 10:36:25 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift installer pull 5923 0 None open Bug 1965468: Revert "Bug 1909136: OpenStack: delete volume snapshots" 2022-05-17 16:58:40 UTC
Red Hat Product Errata RHSA-2022:5069 0 None None None 2022-08-10 10:36:35 UTC

Description Mike Fedosin 2021-05-27 18:33:47 UTC
Because Cinder CSI driver currently doesn't support adding cluster ID to volume snapshot metadata we had to introduce a workaround PR to delete snapshots: https://github.com/openshift/installer/pull/4962

It should be reverted when the upstream fix is merged: https://github.com/kubernetes/cloud-provider-openstack/pull/1544

Comment 1 ShiftStack Bugwatcher 2021-11-25 16:11:50 UTC
Removing the Triaged keyword because:
* the target release value is missing

* the QE automation assessment (flag qe_test_coverage) is missing

Comment 5 Jon Uriarte 2022-06-21 11:42:19 UTC
Verified in 4.11.0-0.nightly-2022-06-15-222801 on top of OSP 16.1.6.

Cluster destroy deletes now the volumes and snapshots in OSP created by the cluster.
Verification steps:

> Create a PVC:

$ cat <<EOF | oc apply -f -
---
apiVersion: project.openshift.io/v1
kind: Project
metadata:
  name: cinderpvc
  labels:
    kubernetes.io/metadata.name: cinderpvc
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: "pvc-1"
  namespace: "cinderpvc"
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
  #storageClassName: (default)
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo-1
  namespace: "cinderpvc"
spec:
  replicas: 1
  selector:
    matchLabels:
      app: demo-1
  template:
    metadata:
      labels:
        app: demo-1
    spec:
      containers:
      - name: demo
        image: quay.io/kuryr/demo
        ports:
        - containerPort: 80
          protocol: TCP
        volumeMounts:
          - mountPath: /var/lib/www/data
            name: mydata
      volumes:
        - name: mydata
          persistentVolumeClaim:
            claimName: pvc-1
            readOnly: false
EOF

$ oc -n cinderpvc get pod -o wide
NAME                     READY   STATUS    RESTARTS   AGE   IP            NODE                          NOMINATED NODE   READINESS GATES
demo-1-d9b4c79b6-f58lm   1/1     Running   0          53s   10.131.0.18   ostest-b5mzn-worker-0-2hhl5   <none>           <none>

$ oc -n cinderpvc get pvc
NAME    STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
pvc-1   Bound    pvc-5c1fc842-ae70-4397-a4f9-7a0130739444   1Gi        RWO            standard-csi   36s

$ openstack volume list
+--------------------------------------+------------------------------------------+--------+------+------------------------------------------------------+
| ID                                   | Name                                     | Status | Size | Attached to                                          |
+--------------------------------------+------------------------------------------+--------+------+------------------------------------------------------+
| 2134abf0-9b56-4c81-9b4c-8fb75e037b81 | pvc-5c1fc842-ae70-4397-a4f9-7a0130739444 | in-use |    1 | Attached to ostest-b5mzn-worker-0-2hhl5 on /dev/vdb  |
+--------------------------------------+------------------------------------------+--------+------+------------------------------------------------------+

> Write some data in the volume:
$ oc -n cinderpvc rsh demo-1-d9b4c79b6-f58lm /bin/sh
/home/kuryr $ echo “hello from pvc-5c1fc842-ae70-4397-a4f9-7a0130739444 volume” > /var/lib/www/data/test.txt
/home/kuryr $ cat /var/lib/www/data/test.txt
“hello from pvc-5c1fc842-ae70-4397-a4f9-7a0130739444 volume”
/home/kuryr $

> Create a VolumeSnapshotClass:
$ cat <<EOF | oc apply -f -
---
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshotClass
metadata:
  name: cinderpvc-snapclass
  namespace: cinderpvc
driver: cinder.csi.openstack.org
deletionPolicy: Delete
parameters:
  force-create: "true"
EOF

$ oc get VolumeSnapshotClass
NAME                  DRIVER                     DELETIONPOLICY   AGE
cinderpvc-snapclass   cinder.csi.openstack.org   Delete           28s
standard-csi          cinder.csi.openstack.org   Delete           127m

> Create a VolumeSnapshot:
$ cat <<EOF | oc apply -f -
---
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
  name: pvc-1-snap
  namespace: cinderpvc
spec:
  volumeSnapshotClassName: cinderpvc-snapclass
  source:
    persistentVolumeClaimName: pvc-1
EOF

$ oc -n cinderpvc get volumesnapshot
NAME         READYTOUSE   SOURCEPVC   SOURCESNAPSHOTCONTENT   RESTORESIZE   SNAPSHOTCLASS         SNAPSHOTCONTENT                                    CREATIONTIME   AGE
pvc-1-snap   true         pvc-1                               1Gi           cinderpvc-snapclass   snapcontent-77dc9462-3910-4a76-b004-1c45f628d140   111s           114s

$ oc -n cinderpvc get volumesnapshotcontent
NAME                                               READYTOUSE   RESTORESIZE   DELETIONPOLICY   DRIVER                     VOLUMESNAPSHOTCLASS   VOLUMESNAPSHOT   VOLUMESNAPSHOTNAMESPACE   AGE
snapcontent-77dc9462-3910-4a76-b004-1c45f628d140   true         1073741824    Delete           cinder.csi.openstack.org   cinderpvc-snapclass   pvc-1-snap       cinderpvc                 2m12s

$ oc -n cinderpvc describe volumesnapshot pvc-1-snap
ame:         pvc-1-snap
Namespace:    cinderpvc
Labels:       <none>
Annotations:  <none>
API Version:  snapshot.storage.k8s.io/v1
Kind:         VolumeSnapshot
Metadata:
  Creation Timestamp:  2022-06-21T11:04:39Z
  Finalizers:
    snapshot.storage.kubernetes.io/volumesnapshot-as-source-protection
    snapshot.storage.kubernetes.io/volumesnapshot-bound-protection
  Generation:  1
  Managed Fields:
    API Version:  snapshot.storage.k8s.io/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:                                                                                                                                                                                                                       
          .:                                                                                                                                                                                                                                 
          f:kubectl.kubernetes.io/last-applied-configuration:
      f:spec:
        .:
        f:source:
          .:
          f:persistentVolumeClaimName:
        f:volumeSnapshotClassName:
    Manager:      kubectl-client-side-apply
    Operation:    Update
    Time:         2022-06-21T11:04:39Z
    API Version:  snapshot.storage.k8s.io/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:finalizers:
          .:
          v:"snapshot.storage.kubernetes.io/volumesnapshot-as-source-protection":
          v:"snapshot.storage.kubernetes.io/volumesnapshot-bound-protection":
    Manager:      snapshot-controller
    Operation:    Update
    Time:         2022-06-21T11:04:39Z
    API Version:  snapshot.storage.k8s.io/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        .:
        f:boundVolumeSnapshotContentName:
        f:creationTime:
        f:readyToUse:
        f:restoreSize:
    Manager:         snapshot-controller
    Operation:       Update
    Subresource:     status
    Time:            2022-06-21T11:04:44Z
  Resource Version:  70668
  UID:               77dc9462-3910-4a76-b004-1c45f628d140
Spec:
  Source:
    Persistent Volume Claim Name:  pvc-1
  Volume Snapshot Class Name:      cinderpvc-snapclass
Status:
  Bound Volume Snapshot Content Name:  snapcontent-77dc9462-3910-4a76-b004-1c45f628d140
  Creation Time:                       2022-06-21T11:04:42Z
  Ready To Use:                        true
  Restore Size:                        1Gi
Events:
  Type    Reason            Age    From                 Message
  ----    ------            ----   ----                 -------
  Normal  CreatingSnapshot  2m29s  snapshot-controller  Waiting for a snapshot cinderpvc/pvc-1-snap to be created by the CSI driver.
  Normal  SnapshotCreated   2m24s  snapshot-controller  Snapshot cinderpvc/pvc-1-snap was successfully created by the CSI driver.
  Normal  SnapshotReady     2m24s  snapshot-controller  Snapshot cinderpvc/pvc-1-snap is ready to use.


> Add data to the original pvc-1 volume:
$ oc -n cinderpvc rsh demo-1-d9b4c79b6-f58lm /bin/sh
/home/kuryr $ cat /var/lib/www/data/test.txt
“hello from pvc-5c1fc842-ae70-4397-a4f9-7a0130739444 volume”
/home/kuryr $ echo “new data after pvc-1-snap creation” >> /var/lib/www/data/test.txt
/home/kuryr $ cat /var/lib/www/data/test.txt
“hello from pvc-5c1fc842-ae70-4397-a4f9-7a0130739444 volume”
“new data after pvc-1-snap creation”
/home/kuryr $
 
> Create a new pvc from pvc-1-snap snapshot:
$ cat <<EOF | oc apply -f -
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-1-restore
  namespace: cinderpvc
spec:
  #storageClassName: (default)
  dataSource:
    name: pvc-1-snap 
    kind: VolumeSnapshot 
    apiGroup: snapshot.storage.k8s.io 
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
EOF

$ oc -n cinderpvc get pvc
NAME            STATUS    VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
pvc-1           Bound     pvc-5c1fc842-ae70-4397-a4f9-7a0130739444   1Gi        RWO            standard-csi   13m
pvc-1-restore   Pending                                                                        standard-csi   16s

$ openstack volume list
+--------------------------------------+------------------------------------------+--------+------+------------------------------------------------------+                                                                                   
| ID                                   | Name                                     | Status | Size | Attached to                                          |                                                                                   
+--------------------------------------+------------------------------------------+--------+------+------------------------------------------------------+                                                                                   
| 2134abf0-9b56-4c81-9b4c-8fb75e037b81 | pvc-5c1fc842-ae70-4397-a4f9-7a0130739444 | in-use |    1 | Attached to ostest-b5mzn-worker-0-2hhl5 on /dev/vdb  |                                                                                   
+--------------------------------------+------------------------------------------+--------+------+------------------------------------------------------+ 

> Attach to new pvc pvc-1-restore to the deployment, and dettach the previous one pvc-1:
$ oc -n cinderpvc edit deployment.apps/demo-1

Replace:
      volumes:
      - name: mydata
        persistentVolumeClaim:
          claimName: pvc-1

by:
      volumes:
      - name: mydata
        persistentVolumeClaim:
          claimName: pvc-1-restore

$ oc -n cinderpvc get pod -o wide
NAME                     READY   STATUS    RESTARTS   AGE     IP            NODE                          NOMINATED NODE   READINESS GATES
demo-1-d5bfbf74f-8xlwd   1/1     Running   0          2m14s   10.128.2.29   ostest-b5mzn-worker-0-gpcz7   <none>           <none>


$ oc -n cinderpvc get pvc
NAME            STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
pvc-1           Bound    pvc-5c1fc842-ae70-4397-a4f9-7a0130739444   1Gi        RWO            standard-csi   17m
pvc-1-restore   Bound    pvc-376c25ed-c9f5-4655-801a-ef1cc155e945   1Gi        RWO            standard-csi   4m20s


$ openstack volume list
+--------------------------------------+------------------------------------------+-----------+------+------------------------------------------------------+                                                                                
| ID                                   | Name                                     | Status    | Size | Attached to                                          |                                                                                
+--------------------------------------+------------------------------------------+-----------+------+------------------------------------------------------+                                                                                
| eeda7488-1906-47a8-85f4-8b9211cb3d2e | pvc-376c25ed-c9f5-4655-801a-ef1cc155e945 | in-use    |    1 | Attached to ostest-b5mzn-worker-0-gpcz7 on /dev/vdb  |                                                                                
| 2134abf0-9b56-4c81-9b4c-8fb75e037b81 | pvc-5c1fc842-ae70-4397-a4f9-7a0130739444 | available |    1 |                                                      |                                                                                
+--------------------------------------+------------------------------------------+-----------+------+------------------------------------------------------+


> Check the content (original content from pvc-1 volume):
$ oc -n cinderpvc rsh demo-1-d5bfbf74f-8xlwd /bin/sh
/home/kuryr $ cat /var/lib/www/data/test.txt
“hello from pvc-5c1fc842-ae70-4397-a4f9-7a0130739444 volume”


> Additional data:
$ openstack volume snapshot list
+--------------------------------------+-----------------------------------------------+----------------------------------------+-----------+------+
| ID                                   | Name                                          | Description                            | Status    | Size |
+--------------------------------------+-----------------------------------------------+----------------------------------------+-----------+------+
| bb87eb75-66f0-4390-836b-6f298639825f | snapshot-77dc9462-3910-4a76-b004-1c45f628d140 | Created by OpenStack Cinder CSI driver | available |    1 |
+--------------------------------------+-----------------------------------------------+----------------------------------------+-----------+------+

$ openstack volume snapshot show snapshot-77dc9462-3910-4a76-b004-1c45f628d140
+--------------------------------------------+-------------------------------------------------+
| Field                                      | Value                                           |
+--------------------------------------------+-------------------------------------------------+
| created_at                                 | 2022-06-21T11:04:42.000000                      |
| description                                | Created by OpenStack Cinder CSI driver          |
| id                                         | bb87eb75-66f0-4390-836b-6f298639825f            |
| name                                       | snapshot-77dc9462-3910-4a76-b004-1c45f628d140   |
| os-extended-snapshot-attributes:progress   | 100%                                            |
| os-extended-snapshot-attributes:project_id | 42a001f17da442e9a0e894a1a7052603                |
| properties                                 | cinder.csi.openstack.org/cluster='ostest-b5mzn' |
| size                                       | 1                                               |
| status                                     | available                                       |
| updated_at                                 | 2022-06-21T11:04:43.000000                      |
| volume_id                                  | 2134abf0-9b56-4c81-9b4c-8fb75e037b81            |
+--------------------------------------------+-------------------------------------------------+

$ openstack volume show pvc-376c25ed-c9f5-4655-801a-ef1cc155e945 -f value -c properties
{'cinder.csi.openstack.org/cluster': 'ostest-b5mzn'}
$ openstack volume show pvc-5c1fc842-ae70-4397-a4f9-7a0130739444 -f value -c properties
{'cinder.csi.openstack.org/cluster': 'ostest-b5mzn'}

$ openstack volume show eeda7488-1906-47a8-85f4-8b9211cb3d2e
+------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field                        | Value                                                                                                                                                                                                                                                                                                                         |
+------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| attachments                  | [{'id': 'eeda7488-1906-47a8-85f4-8b9211cb3d2e', 'attachment_id': '12a4f496-a9c6-4ed6-aa5e-7116d92db0de', 'volume_id': 'eeda7488-1906-47a8-85f4-8b9211cb3d2e', 'server_id': '4c51eda8-c9a3-4c75-bc8a-c8b02e5410f6', 'host_name': 'compute-1.redhat.local', 'device': '/dev/vdb', 'attached_at': '2022-06-21T11:15:23.000000'}] |
| availability_zone            | nova                                                                                                                                                                                                                                                                                                                          |
| bootable                     | false                                                                                                                                                                                                                                                                                                                         |
| consistencygroup_id          | None                                                                                                                                                                                                                                                                                                                          |
| created_at                   | 2022-06-21T11:15:17.000000                                                                                                                                                                                                                                                                                                    |
| description                  | Created by OpenStack Cinder CSI driver                                                                                                                                                                                                                                                                                        |
| encrypted                    | False                                                                                                                                                                                                                                                                                                                         |
| id                           | eeda7488-1906-47a8-85f4-8b9211cb3d2e                                                                                                                                                                                                                                                                                          |
| multiattach                  | False                                                                                                                                                                                                                                                                                                                         |
| name                         | pvc-376c25ed-c9f5-4655-801a-ef1cc155e945                                                                                                                                                                                                                                                                                      |
| os-vol-tenant-attr:tenant_id | 42a001f17da442e9a0e894a1a7052603                                                                                                                                                                                                                                                                                              |
| properties                   | cinder.csi.openstack.org/cluster='ostest-b5mzn'                                                                                                                                                                                                                                                                               |
| replication_status           | None                                                                                                                                                                                                                                                                                                                          |
| size                         | 1                                                                                                                                                                                                                                                                                                                             |
| snapshot_id                  | bb87eb75-66f0-4390-836b-6f298639825f                                                                                                                                                                                                                                                                                          |
| source_volid                 | None                                                                                                                                                                                                                                                                                                                          |
| status                       | in-use                                                                                                                                                                                                                                                                                                                        |
| type                         | tripleo                                                                                                                                                                                                                                                                                                                       |
| updated_at                   | 2022-06-21T11:15:24.000000                                                                                                                                                                                                                                                                                                    |
| user_id                      | 550df7bf64db4a20850d477c9f889a43                                                                                                                                                                                                                                                                                              |
+------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

$ openstack volume show 2134abf0-9b56-4c81-9b4c-8fb75e037b81
+------------------------------+-------------------------------------------------+
| Field                        | Value                                           |
+------------------------------+-------------------------------------------------+
| attachments                  | []                                              |
| availability_zone            | nova                                            |
| bootable                     | false                                           |
| consistencygroup_id          | None                                            |
| created_at                   | 2022-06-21T10:57:31.000000                      |
| description                  | Created by OpenStack Cinder CSI driver          |
| encrypted                    | False                                           |
| id                           | 2134abf0-9b56-4c81-9b4c-8fb75e037b81            |
| multiattach                  | False                                           |
| name                         | pvc-5c1fc842-ae70-4397-a4f9-7a0130739444        |
| os-vol-tenant-attr:tenant_id | 42a001f17da442e9a0e894a1a7052603                |
| properties                   | cinder.csi.openstack.org/cluster='ostest-b5mzn' |
| replication_status           | None                                            |
| size                         | 1                                               |
| snapshot_id                  | None                                            |
| source_volid                 | None                                            |
| status                       | available                                       |
| type                         | tripleo                                         |
| updated_at                   | 2022-06-21T11:15:40.000000                      |
| user_id                      | 550df7bf64db4a20850d477c9f889a43                |
+------------------------------+-------------------------------------------------+

> Destroy the cluster and check the volumes and snapshot were deleted:

[...]
time="2022-06-21T07:32:20-04:00" level=debug msg="Deleting volume snapshot \"bb87eb75-66f0-4390-836b-6f298639825f\""
time="2022-06-21T07:32:21-04:00" level=debug msg="Deleting volume \"eeda7488-1906-47a8-85f4-8b9211cb3d2e\""
time="2022-06-21T07:32:21-04:00" level=debug msg="Deleting volume \"2134abf0-9b56-4c81-9b4c-8fb75e037b81\""
[...]

$ openstack volume snapshot list
$
$ openstack volume list
$

Comment 6 errata-xmlrpc 2022-08-10 10:36:25 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: OpenShift Container Platform 4.11.0 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:5069


Note You need to log in before you can comment on or make changes to this bug.