Red Hat Bugzilla – Bug 1255639
[libgfapi]: do an explicit lookup on the inodes linked in readdirp
Last modified: 2016-09-17 10:42:06 EDT
+++ This bug was initially created as a clone of Bug #1236009 +++
Description of problem:
Like fuse, its better to do an explicit lookup on the inodes linked via readdirp. Otherwise, post linkage directly fops will come and below xlators such as AFR, DHT will not be able to do their house keeping activities for the inode, which they would have done in lookup.
So do an explicit lookup on those inodes linked via readdirp, before proceeding with the first fop on those inodes.
Version-Release number of selected component (if applicable):
Steps to Reproduce:
--- Additional comment from Anand Avati on 2015-07-02 05:16:18 EDT ---
COMMIT: http://review.gluster.org/11236 committed in master by Niels de Vos (firstname.lastname@example.org)
Author: Raghavendra Bhat <email@example.com>
Date: Fri Jun 12 15:12:05 2015 +0530
libgfapi: send explicit lookups on inodes linked in readdirp
If the inode is linked via readdirp, then the consuners of gfapi which are using
handles (got either in lookup or readdirp) might not send an explicit lookup on
that object again (ex: NFS, samba, USS). If there is a replicate volume where
the replicas of the object are not in sync, then readdirp followed by fops might
lead data being served from the subvolume which is not in sync with latest
data. And since lookup is needed to trigger self-heal on that object the
consumers might keep getting wrong data until an explicit lookup is not done.
Fuse handles this situation by sending an explicit lookup by itself (fuse
xlator) on those inodes which are linked via readdirp, whenever a fop comes on
The same procedure is done in gfapi as well to address this situation.
Thanks to shyam(firstname.lastname@example.org) for valuable inputs
Signed-off-by: Raghavendra Bhat <email@example.com>
Tested-by: NetBSD Build System <firstname.lastname@example.org>
Reviewed-by: Shyamsundar Ranganathan <email@example.com>
Tested-by: Gluster Build System <firstname.lastname@example.org>
Reviewed-by: Niels de Vos <email@example.com>
A way to do this testing is to have distribute setup (either plain distribute or dist-rep etc) having gfapi - samba/nfs-ganesha. Following are the steps
1. This test needs to have a pre-populated data before we start nfs ganesha/smb server. The point is to have the inode linked during readdirp and not to have them linked during mkdir. So we can create data, stop the volume and restart the volume.
2. Lets say readdirp returns a directory - dir
3. try to create files/directories within "dir". If lookup was not done on "dir" layout wouldn't be set on "dir" and hence create/mkdir within dir will fail (mostly with EINVAL/EIO). Note that dht doesn't populate layout during readdirp and hence this can be a good test case.
tested the scenarios on 3.1.3 setup with the steps mentioned in comment 5. Volume is disp-rep.
1.Created data on mount. mkdir -p /mnt/cifs/xyz
2.Stopped the volume.
3.Start the volume.
4.ls, mkdir & touch operations are working fine.
Did not faced any issues regarding the same.
Verified with ganesha setup on 3.1.3 build with below steps:
>>> Mounted the ganesha vol with v4
[root@dhcp37-206 ~]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/rhel_dhcp37--206-root 27740944 1309384 26431560 5% /
devtmpfs 3995072 0 3995072 0% /dev
tmpfs 4005468 0 4005468 0% /dev/shm
tmpfs 4005468 16684 3988784 1% /run
tmpfs 4005468 0 4005468 0% /sys/fs/cgroup
/dev/vda1 508588 126928 381660 25% /boot
tmpfs 801096 0 801096 0% /run/user/0
10.70.36.217:/testvolume 187676672 222208 187454464 1% /mnt/nfs1
>>> Created a dir on the mount point:
[root@dhcp37-206 nfs1]# mkdir dir
>>> Stop and start the volume:
[root@dhcp37-44 ~]# gluster vol stop testvolume
Stopping volume will make its data inaccessible. Do you want to continue? (y/n) y
volume stop: testvolume: success
[root@dhcp37-44 ~]# gluster vol start testvolume
volume start: testvolume: success
>>> Do a ls and create a file inside the previously created dir
[root@dhcp37-206 nfs1]# ls
[root@dhcp37-206 nfs1]# touch dir/file
[root@dhcp37-206 nfs1]# cd dir/
[root@dhcp37-206 dir]# ls
[root@dhcp37-206 dir]# ls -ltr
-rw-r--r--. 1 root root 0 Jun 7 04:41 file
>>> As mentioned in comment 5: "try to create files/directories within "dir". If lookup was not done on "dir" layout wouldn't be set on "dir" and hence create/mkdir within dir will fail "
the file creation inside the dir is successful when lookup was not done on the dir.
So based on the verification with samba and nfs-ganesha and with the steps mentioned in comment 5 and 6, marking this bug as Verified for 3.1.3
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.