Description of problem: It is not possible to remove a volume e.g. test-pvc if that volume was used to * create a snapshot e.g. test-pvc-snap (that was removed later) * create a volume from that snapshot e.g. test-pvc-from-snap Until you remove the volume from that snapshot (test-pvc-from-snap). openstack client returns 0 but cinder-volume logs returns: 2021-08-03 13:09:28.616 135 DEBUG cinder.volume.drivers.rbd [req-d3dabd8a-4dff-4b0f-8145-10fa303e7980 ebcbf8b5fdde4763ba119de20b84d4f7 d8efce877a93437d8e15d11ec19e3d0e - default default] volume has no backup snaps _delete_backup_snaps /usr/lib/python3.6/site-packages/cinder/volume/drivers/rbd.py:1038 2021-08-03 13:09:28.621 135 ERROR cinder.volume.manager [req-d3dabd8a-4dff-4b0f-8145-10fa303e7980 ebcbf8b5fdde4763ba119de20b84d4f7 d8efce877a93437d8e15d11ec19e3d0e - default default] Unable to delete busy volume.: cinder.exception.VolumeIsBusy: deleting volume volume-22a31673-e685-46ef-9656-382c8dca8357 that has snapshot Version-Release number of selected component (if applicable): * OSP 16.2-beta * tested with and without rbd_flatten_volume_from_snapshot enabled (no difference was observed) * Ceph cluster configured with 'rbd default clone format = 2' * ceph osd set-require-min-compat-client mimic Ceph + Cinder containers restarted. Steps to Reproduce: Create a volume: openstack volume create --size 1 test-pvc Create a snapshot: openstack volume snapshot create --volume test-pvc test-snapshot-pvc Create a volume from the snapshot: openstack volume create --snapshot test-snapshot-pvc test-pvc-clone Try deleting the initial volume, it fails as expected: openstack volume delete test-pvc Delete the snapshot: openstack volume snapshot delete test-snapshot-pvc Now, retry to delete the initial volume: it doesn't report any error: openstack volume delete test-pvc But the volume is still here and available: openstack volume list cinder-volume logs: 2021-08-03 13:09:28.616 135 DEBUG cinder.volume.drivers.rbd [req-d3dabd8a-4dff-4b0f-8145-10fa303e7980 ebcbf8b5fdde4763ba119de20b84d4f7 d8efce877a93437d8e15d11ec19e3d0e - default default] volume has no backup snaps _delete_backup_snaps /usr/lib/python3.6/site-packages/cinder/volume/drivers/rbd.py:1038 2021-08-03 13:09:28.621 135 ERROR cinder.volume.manager [req-d3dabd8a-4dff-4b0f-8145-10fa303e7980 ebcbf8b5fdde4763ba119de20b84d4f7 d8efce877a93437d8e15d11ec19e3d0e - default default] Unable to delete busy volume.: cinder.exception.VolumeIsBusy: deleting volume volume-22a31673-e685-46ef-9656-382c8dca8357 that has snapshot Now, delete the volume clone (created from a snapshot): openstack volume delete test-pvc-clone And retry to delete the initial volume: openstack volume delete test-pvc It worked, the volume was actually removed: openstack volume list Actual results: openstackclient returns 0 but the volume isn't removed, an ERROR is seen in the logs. We can see in the logs that Clone v2 is used: "Using v2 Clone API" Expected results: The volume should be removed. Additional info: This was supposed to be fixed by https://bugzilla.redhat.com/show_bug.cgi?id=1764324 but we found the issue while investigating https://bugzilla.redhat.com/show_bug.cgi?id=1917710
The V2 CLONE API makes it impossible to delete the snapshot without removing the original volume test-pvc. However, in order to delete the cloned volume, cinder expects an extra configuration: rbd_flatten_volume_from_snapshot [1]. This config is set to False by default. To solve this issue enable it on cinder.conf. [1] https://opendev.org/openstack/cinder/src/branch/master/cinder/volume/drivers/rbd.py#L1073
*** Bug 2050163 has been marked as a duplicate of this bug. ***
For Openshift on Openstack, this bug is provoking the K8s being hung on PVC deletion. Rising the severity.
Hello Currently there is a Cinder upstream patch in progress. We need changes in Glance similar to Cinder to allow deletion of parent images that have dependent volumes. Eric lead the discussion at Bobcat PTG https://wiki.openstack.org/wiki/CinderBobcatPTGSummary#RBD_deletion_issues Cheers,