Bug 1255639

Summary: [libgfapi]: do an explicit lookup on the inodes linked in readdirp
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Raghavendra Bhat <rabhat>
Component: coreAssignee: Bug Updates Notification Mailing List <rhs-bugs>
Status: CLOSED ERRATA QA Contact: Rahul Hinduja <rhinduja>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rhgs-3.1CC: asrivast, ndevos, pkarampu, rcyriac, rhinduja, rhs-bugs, sankarshan, sdharane, sraj, storage-qa-internal, vdas
Target Milestone: ---Keywords: Triaged, ZStream
Target Release: RHGS 3.1.3   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: glusterfs-3.7.9-2 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1236009 Environment:
Last Closed: 2016-06-23 04:54:50 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1236009    
Bug Blocks: 1240190, 1299184    

Description Raghavendra Bhat 2015-08-21 09:00:33 UTC 
+++ This bug was initially created as a clone of Bug #1236009 +++

Description of problem:

Like fuse, its better to do an explicit lookup on the inodes linked via readdirp. Otherwise, post linkage directly fops will come and below xlators such as AFR, DHT will not be able to do their house keeping activities for the inode, which they would have done in lookup.

So do an explicit lookup on those inodes linked via readdirp, before proceeding with the first fop on those inodes.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

--- Additional comment from Anand Avati on 2015-07-02 05:16:18 EDT ---

COMMIT: http://review.gluster.org/11236 committed in master by Niels de Vos (ndevos) 
------
commit c1a8ae20bd055227921d199b952582eca4104f67
Author: Raghavendra Bhat <raghavendra>
Date:   Fri Jun 12 15:12:05 2015 +0530

    libgfapi: send explicit lookups on inodes linked in readdirp
    
    If the inode is linked via readdirp, then the consuners of gfapi which are using
    handles (got either in lookup or readdirp) might not send an explicit lookup on
    that object again (ex: NFS, samba, USS). If there is a replicate volume where
    the replicas of the object are not in sync, then readdirp followed by fops might
    lead data being served from the subvolume which is not in sync with latest
    data. And since lookup is needed to trigger self-heal on that object the
    consumers might keep getting wrong data until an explicit lookup is not done.
    
    Fuse handles this situation by sending an explicit lookup by itself (fuse
    xlator) on those inodes which are linked via readdirp, whenever a fop comes on
    that inode.
    
    The same procedure is done in gfapi as well to address this situation.
    
    Thanks to shyam(srangana) for valuable inputs
    
    Change-Id: I64f0591495dddc1dea7f8dc319f2558a7e342871
    BUG: 1236009
    Signed-off-by: Raghavendra Bhat <raghavendra>
    Reviewed-on: http://review.gluster.org/11236
    Tested-by: NetBSD Build System <jenkins.org>
    Reviewed-by: Shyamsundar Ranganathan <srangana>
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Niels de Vos <ndevos>
Comment 5 Pranith Kumar K 2016-06-06 12:00:58 UTC 
A way to do this testing is to have distribute setup (either plain distribute or dist-rep etc) having gfapi - samba/nfs-ganesha. Following are the steps

1. This test needs to have a pre-populated data before we start nfs ganesha/smb server. The point is to have the inode linked during readdirp and not to have them linked during mkdir. So we can create data, stop the volume and restart the volume.
2. Lets say readdirp returns a directory - dir
3. try to create files/directories within "dir". If lookup was not done on "dir" layout wouldn't be set on "dir" and hence create/mkdir within dir will fail (mostly with EINVAL/EIO). Note that dht doesn't populate layout during readdirp and hence this can be a good test case.

regards,
Raghavendra
Comment 6 Vivek Das 2016-06-07 10:04:18 UTC 
tested the scenarios on 3.1.3 setup with the steps mentioned in comment 5. Volume is disp-rep.
samba4.4.3-7
gluster3.7.9-8

1.Created data on mount. mkdir -p /mnt/cifs/xyz
2.Stopped the volume.
3.Start the volume.
4.ls, mkdir & touch operations are working fine.

Did not faced any issues regarding the same.
Comment 7 Shashank Raj 2016-06-07 10:15:31 UTC 
Verified with ganesha setup on 3.1.3 build with below steps:

>>> Mounted the ganesha vol with v4

[root@dhcp37-206 ~]# df
Filesystem                        1K-blocks    Used Available Use% Mounted on
/dev/mapper/rhel_dhcp37--206-root  27740944 1309384  26431560   5% /
devtmpfs                            3995072       0   3995072   0% /dev
tmpfs                               4005468       0   4005468   0% /dev/shm
tmpfs                               4005468   16684   3988784   1% /run
tmpfs                               4005468       0   4005468   0% /sys/fs/cgroup
/dev/vda1                            508588  126928    381660  25% /boot
tmpfs                                801096       0    801096   0% /run/user/0
10.70.36.217:/testvolume          187676672  222208 187454464   1% /mnt/nfs1

>>> Created a dir on the mount point:

[root@dhcp37-206 nfs1]# mkdir dir
[root@dhcp37-206 nfs1]#

>>> Stop and start the volume:

[root@dhcp37-44 ~]# gluster vol stop testvolume
Stopping volume will make its data inaccessible. Do you want to continue? (y/n) y
volume stop: testvolume: success
[root@dhcp37-44 ~]# gluster vol start testvolume
volume start: testvolume: success

>>> Do a ls and create a file inside the previously created dir

[root@dhcp37-206 nfs1]# ls
dir
[root@dhcp37-206 nfs1]# touch dir/file
[root@dhcp37-206 nfs1]# cd dir/
[root@dhcp37-206 dir]# ls
file
[root@dhcp37-206 dir]# ls -ltr
total 0
-rw-r--r--. 1 root root 0 Jun  7 04:41 file
[root@dhcp37-206 dir]#

>>> As mentioned in comment 5: "try to create files/directories within "dir". If lookup was not done on "dir" layout wouldn't be set on "dir" and hence create/mkdir within dir will fail "

the file creation inside the dir is successful when lookup was not done on the dir.

So based on the verification with samba and nfs-ganesha and with the steps mentioned in comment 5 and 6, marking this bug as Verified for 3.1.3
Comment 9 errata-xmlrpc 2016-06-23 04:54:50 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1240