Bug 1695399 - With parallel-readdir enabled, deleting a directory containing stale linkto files fails with "Directory not empty"
Summary: With parallel-readdir enabled, deleting a directory containing stale linkto f...
Keywords:
Status: CLOSED NEXTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: distribute
Version: 5
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Nithya Balachandran
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-04-03 03:55 UTC by Nithya Balachandran
Modified: 2019-05-08 14:06 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-05-08 14:06:25 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Gluster.org Gerrit 22485 None Merged cluster/dht: Request linkto xattrs in dht_rmdir opendir 2019-05-08 14:06:23 UTC

Description Nithya Balachandran 2019-04-03 03:55:41 UTC
This bug was initially created as a copy of Bug #1672851

I am copying this bug because: 



Description of problem:

If parallel-readdir is enabled on a volume, rm -rf <dir> fails with "Directory not empty" if <dir> contains stale linkto files.


Version-Release number of selected component (if applicable):


How reproducible:
Consistently

Steps to Reproduce:
1. Create a 3 brick distribute volume
2. Enable parallel-readdir and readdir-ahead on the volume
3. Fuse mount the volume and mkdir dir0
4. Create some files inside dir0 and rename them so linkto files are created on the bricks
5. Check the bricks to see which files have linkto files. Delete the data files directly on the bricks, leaving the linkto files behind. These are now stale linkto files.
6. Remount the volume
7. rm -rf dir0

Actual results:
[root@rhgs313-6 fuse1]# rm -rf dir0/
rm: cannot remove ‘dir0/’: Directory not empty


Expected results:
dir0 should be deleted without errors

Additional info:

Comment 1 Nithya Balachandran 2019-04-03 03:57:00 UTC
RCA:

rm -rf <dir> works by first listing and unlinking all entries in <dir> and then calling an rmdir <dir>.
As DHT readdirp does not return linkto files in the listing, they are not unlinked as part of the rm -rf itself. dht_rmdir handles this by performing a readdirp internally on <dir> and deleting all stale linkto files before proceeding with the actual rmdir operation.

When parallel-readdir is enabled, the rda xlator is loaded below dht in the graph and proactively lists and caches entries when an opendir is performed. Entries are returned from this cache for any subsequent readdirp calls on the directory that was opened.
DHT uses the presence of the trusted.glusterfs.dht.linkto xattr to determine whether a file is a linkto file. As this call to opendir does not set trusted.glusterfs.dht.linkto in the list of requested xattrs for the opendir call, the cached entries do not contain this xattr value.  As none of the entries returned will have the xattr, DHT believes they are all data files and fails the rmdir with ENOTEMPTY.

Turning off parallel-readdir allows the rm -rf to succeed.


Upstream master:  https://review.gluster.org/22160

Comment 2 Worker Ant 2019-04-03 04:07:33 UTC
REVIEW: https://review.gluster.org/22485 (cluster/dht: Request linkto xattrs in dht_rmdir opendir) posted (#1) for review on release-5 by N Balachandran

Comment 3 Worker Ant 2019-05-08 14:06:25 UTC
REVIEW: https://review.gluster.org/22485 (cluster/dht: Request linkto xattrs in dht_rmdir opendir) merged (#3) on release-5 by N Balachandran


Note You need to log in before you can comment on or make changes to this bug.