+++ This bug was initially created as a clone of Bug #1101438 +++ Description of problem: the problem is seen till now only in one scenario, where root-squash is enabled on volume, after a umount and a subsequent mount of the same volume throws ls only for a single file, not for others. workaround, follow this sequence 1. umount the volume 2. disable root-squash on the volume 3. mount the volume 4. enable the root-squash on the same volume, the ls does not throw any error. Version-Release number of selected component (if applicable): glusterfs-3.6.0.7-1.el6rhs.x86_64 How reproducible: always Steps to Reproduce: 1. create a volume of 6x2 type 2. mount the volume over nfs 3. create directory say "dir3" 4. create a file say "3" and chmod 770 <mount-point>/dir3/3 5. enable root-squash 6. umount <mount-point> 7. mount <mount-point> 8. ls -lR <mount-point> Actual results: [root@rhsauto054 ~]# ls -lR /mnt/nfs-test/ /mnt/nfs-test/: total 0 drwxrwxrwx. 2 root root 76 May 27 12:05 dir1 drwxr-xr-x. 2 root root 44 May 27 11:48 dir2 drwxr-xr-x. 2 root root 44 May 27 11:48 dir3 drwxr-xr-x. 2 root root 36 May 27 11:48 dir4 /mnt/nfs-test/dir1: total 1 -rwxrwxrwx. 1 root root 6 May 27 11:49 1 -rw-r--r--. 1 root root 0 May 27 01:17 newfile -rw-r--r--. 1 nfsnobody nfsnobody 17 May 27 01:27 nobody-file /mnt/nfs-test/dir2: total 1 -rwxrwxrwx. 1 root root 6 May 27 11:49 2 /mnt/nfs-test/dir3: ls: /mnt/nfs-test/dir3/3: Permission denied total 0 -rwxrwx---. 1 root root 0 May 27 01:06 3 /mnt/nfs-test/dir4: total 0 check the permission denied error for dir3/3 nfs.log glimpse, [2014-05-26 20:59:03.738805] W [client-rpc-fops.c:1088:client3_3_getxattr_cbk] 0-dist-rep-client-4: remote operation failed: Permission denied. Path: <gfid:c3a3e8c5-0654-4de9-acdf-1a3562d4e1fd> (c3a3e8c5-0654-4de9-acdf-1a3562d4e1fd). Key: (null) [2014-05-26 20:59:03.739852] W [client-rpc-fops.c:1088:client3_3_getxattr_cbk] 0-dist-rep-client-5: remote operation failed: Permission denied. Path: <gfid:c3a3e8c5-0654-4de9-acdf-1a3562d4e1fd> (c3a3e8c5-0654-4de9-acdf-1a3562d4e1fd). Key: (null) [2014-05-26 20:59:08.975778] W [client-rpc-fops.c:1088:client3_3_getxattr_cbk] 0-dist-rep-client-4: remote operation failed: Permission denied. Path: <gfid:c3a3e8c5-0654-4de9-acdf-1a3562d4e1fd> (c3a3e8c5-0654-4de9-acdf-1a3562d4e1fd). Key: (null) [2014-05-26 20:59:08.976858] W [client-rpc-fops.c:1088:client3_3_getxattr_cbk] 0-dist-rep-client-5: remote operation failed: Permission denied. Path: <gfid:c3a3e8c5-0654-4de9-acdf-1a3562d4e1fd> (c3a3e8c5-0654-4de9-acdf-1a3562d4e1fd). Key: (null) Expected results: no permission denied expected Additional info: --- Additional comment from Saurabh on 2014-05-27 04:03:09 EDT --- --- Additional comment from RHEL Product and Program Management on 2014-05-27 04:13:31 EDT --- Since this issue was entered in bugzilla, the release flag has been set to ? to ensure that it is properly evaluated for this release. --- Additional comment from John Skeoch on 2014-09-21 18:55:53 EDT --- User spradhan's account has been closed --- Additional comment from Niels de Vos on 2014-10-23 08:34:02 EDT --- I'm not actively working on this yet, moving back to NEW so that others can take it if they want to. --- Additional comment from Meghana on 2014-11-04 07:15:47 EST --- The behaviour is seen on all the files that are created following the steps that are listed in the "Steps to reproduce" section. Packet trace shows expected ACCESS reply, allowing read and lookup permissions for the file. There are errors seen in server_getxattr_cbk.Yet to root cause the issue. --- Additional comment from Niels de Vos on 2014-11-05 10:34:00 EST --- Capturing what we discussed earlier today. Permission denied comes from xlators/system/posix-acl/src/posix-acl.c:acl_permits(): ... 212 for (i = 0; i < acl->count; i++) { 213 switch (ace->tag) { ... 245 case POSIX_ACL_OTHER: 246 perm = (ctx->perm & S_IRWXO); 247 if (!found) 248 goto perm_check; ... 269 perm_check: 270 if ((perm & want) == want) { 271 goto green; 272 } else { 273 goto red; 274 } ... acl_permits() lands in the switch statement at POSIX_ACL_OTHER. 'found' is false, so it goes to 'perm_check', where the if-statement results in the 'else' branch. I do not remember what the 'want' was though... Please attach a tcpdump and output from rpcdebug captured on the NFS-client so that I can review it. Thanks! --- Additional comment from Meghana on 2014-11-05 12:25:33 EST --- The value of want was 4 in this case. --- Additional comment from Meghana on 2014-11-05 12:54:55 EST --- --- Additional comment from Meghana on 2014-11-06 03:38:17 EST --- Attaching rpcdebug output on the NFS client.
REVIEW: http://review.gluster.org/9064 (When OTHERS are not given any permissions, ACLs are not displayed. Even when there are no permissions to any of the users/groups/others, ACLs and attrs have to be disaplyed. Fixed this by removing unnecessary checks. Signed-off-by: Meghana Madhusudhan <mmadhusu>) posted (#1) for review on master by Meghana M (mmadhusu)
REVIEW: http://review.gluster.org/9085 (When root-squash is enabled or when no permissions are given to a file, NFS threw permission errors. According to the kernel-nfs behaviour, no permissions were required to read ACLs and other extended attributes.) posted (#1) for review on master by Meghana M (mmadhusu)
Created attachment 955752 [details] kernel-nfs.logs I have attached kernel-nfs logs here. I performed numerous tests on kernel-nfs and observed a few things. 1. It doesn't require read permissions for getfacl, ls -l and getfattr commands. 2. Using getfattr commands on an NFS mount, we can only see the nfs_acl attrs. 3. I set extended attributes on the file on a fuse mount. On an NFS mount it is not visible. 4. Extended attrs could not be set on the file on an NFS mount. setfattr commands are not supported on NFS. 5. nfs4_acl xattrs were visible to the client, without any permissions.
REVIEW: http://review.gluster.org/9085 (gNFS: Allow reading ACLs even without read permissions on the file.) posted (#2) for review on master by Meghana M (mmadhusu)
REVIEW: http://review.gluster.org/9085 (gNFS: Allow reading ACLs even without read permissions on the file.) posted (#3) for review on master by Meghana M (mmadhusu)
REVIEW: http://review.gluster.org/9085 (gNFS: Allow reading ACLs even without read permissions on the file.) posted (#4) for review on master by Niels de Vos (ndevos)
REVIEW: http://review.gluster.org/9085 (gNFS: Allow reading ACLs even without read permissions on the file.) posted (#5) for review on master by Meghana M (mmadhusu)
REVIEW: http://review.gluster.org/9085 (gNFS: Allow reading ACLs even without read permissions on the file.) posted (#6) for review on master by Meghana M (mmadhusu)
REVIEW: http://review.gluster.org/9085 (gNFS: Allow reading ACLs even without read permissions on the file.) posted (#7) for review on master by Niels de Vos (ndevos)
COMMIT: http://review.gluster.org/9085 committed in master by Niels de Vos (ndevos) ------ commit 878b30420891c8b00043391cdce90930eaf18795 Author: Meghana Madhusudhan <mmadhusu> Date: Mon Nov 10 15:20:51 2014 +0530 gNFS: Allow reading ACLs even without read permissions on the file. When root-squash is enabled or when no permissions are given to a file, NFS threw permission errors. According to the kernel-nfs behaviour, no permissions are required to read ACLs. When no ACLs are set, the system call sys_lgetxattr fails and returns a ENODATA error. This translates to ESERVERFAULT error in NFS. Fuse makes an exception to this error and returns a success case. Similar changes are made here to achieve the expected behaviour. Change-Id: I46b8f5911114eb087a3f8ca4e921b6b41e83f3b3 BUG: 1161092 Signed-off-by: Meghana Madhusudhan <mmadhusu> Signed-off-by: Niels de Vos <ndevos> Reviewed-on: http://review.gluster.org/9085 Tested-by: Gluster Build System <jenkins.com>
Thanks a lot to Niels and Soumya for their help and timely reviews!
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.0, please open a new bug report. glusterfs-3.7.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/10939 [2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user