Bug 1923882 - Deleting volume backup does not clear the corresponding directories when cinder-backp backend is NFS. [NEEDINFO]
Summary: Deleting volume backup does not clear the corresponding directories when cind...
Keywords:
Status: ON_DEV
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-cinder
Version: 13.0 (Queens)
Hardware: All
OS: All
low
medium
Target Milestone: z2
: 17.1
Assignee: Rajat Dhasmana
QA Contact: Evelina Shames
Andy Stillman
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-02-02 06:51 UTC by Rohini Diwakar
Modified: 2023-08-03 15:46 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Target Upstream Version:
Embargoed:
ifrangs: needinfo? (rdhasman)


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 804603 0 None NEW NFS Backup: Remove leftover dirs 2023-06-05 13:37:01 UTC
Red Hat Issue Tracker OSP-1542 0 None None None 2021-11-18 14:20:55 UTC

Description Rohini Diwakar 2021-02-02 06:51:55 UTC
Description of problem:
Using cinder backup with NFS backend, everything works fine, except that when we delete a backup the actual "content" of the backup gets deleted but not the directory structure on the NFS share.

(overcloud) [stack@undercloud-0 ~]$ openstack volume backup list --long --all-projects -c Container -c ID
+--------------------------------------+--------------------------------------------+
| ID                                   | Container                                  |
+--------------------------------------+--------------------------------------------+
| 17aed1d1-33ba-466e-a51f-d6af156d120e | 17/ae/17aed1d1-33ba-466e-a51f-d6af156d120e |
+--------------------------------------+--------------------------------------------+

[backups]# ll 
total 0
drwxr-xr-x. 3 root root 16 Feb  2 06:28 02
drwxr-xr-x. 3 root root 16 Feb  2 06:33 17
drwxr-xr-x. 3 root root 16 Feb  2 06:19 5a
drwxr-xr-x. 3 root root 16 Feb  2 06:20 e4
drwxr-xr-x. 3 root root 16 Feb  2 06:32 e6

The only available volume backup is 17aed1d1-33ba-466e-a51f-d6af156d120e and the rest of the backups were deleted yet their corresponding directories are still present in the backend. There is no content in the deleted volume backup directories. 

Version-Release number of selected component (if applicable):
RHOSP13

How reproducible:


Steps to Reproduce:
1. Deploy RHOSP13 with cinder backend as NFS and configure cinder-backup service.
2.
3.

Actual results:
Directories for deleted volume backup are not deleted. 

Expected results:
Directories for deleted volume backup should also be deleted. 

Additional info:

Comment 2 Sofia Enriquez 2021-02-08 15:48:42 UTC
Greetings Rohini Diwakar,

This is the expected behaviour.

This happens because the backup service might reuse some sub-folders. Backup data and metadata will
be written to the repository at unique paths that are a function of the backup ID. That path will
be stored in the backup record service_metatdata so that it can be used to navigate to the backup
data and metadata for restore and delete operations (thanks Tzach for the detailed explanation).

Comment 5 Luigi Toscano 2021-02-08 16:55:08 UTC
Just another point in order to assess this fix: apart from the wasted inodes, does having the leftover empty directories around lead to any other issue?

Comment 7 Tzach Shefi 2021-02-10 09:29:06 UTC
Luigi, 

Even if not on this specific case, left unchecked these empty folders may potentially cause problems, 
depending on NFS server/storage system's limits, it could affect other deployments/releases too.
Reaching limits may cause: slow to no access, failure to backup/restore just when you need it most.

Agreed low priority for sure, as we need a whole lot of empty folders before things may fail.
And yep 13 EOL-ing soon means it might not be a good place to start fixing this.


Note You need to log in before you can comment on or make changes to this bug.