Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1161092

Summary: nfs: ls shows "Permission denied" with root-squash
Product: [Community] GlusterFS Reporter: Meghana <mmadhusu>
Component: nfsAssignee: Meghana <mmadhusu>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: medium Docs Contact:
Priority: medium    
Version: mainlineCC: amainkar, bugs, gluster-bugs, mmadhusu, ndevos, saujain, vagarwal
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: glusterfs-3.7.0 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1101438
: 1177899 (view as bug list) Environment:
Last Closed: 2015-05-14 17:28:20 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1101438, 1177899    
Attachments:
Description Flags
kernel-nfs.logs none

Description Meghana 2014-11-06 11:18:01 UTC 
+++ This bug was initially created as a clone of Bug #1101438 +++

Description of problem:
the problem is seen till now only in one scenario, where root-squash is enabled on volume, after a umount and a subsequent mount of the same volume throws ls only for a single file, not for others.

workaround, follow this sequence 
1. umount the volume
2. disable root-squash on the volume
3. mount the volume
4. enable the root-squash on the same volume, 

the ls does not throw any error.

Version-Release number of selected component (if applicable):
glusterfs-3.6.0.7-1.el6rhs.x86_64

How reproducible:
always

Steps to Reproduce:
1. create a volume of 6x2 type
2. mount the volume over nfs
3. create directory say "dir3"
4. create a file say "3" and chmod 770 <mount-point>/dir3/3
5. enable root-squash
6. umount <mount-point>
7. mount <mount-point>
8. ls -lR <mount-point>

Actual results:
[root@rhsauto054 ~]# ls -lR /mnt/nfs-test/
/mnt/nfs-test/:
total 0
drwxrwxrwx. 2 root root 76 May 27 12:05 dir1
drwxr-xr-x. 2 root root 44 May 27 11:48 dir2
drwxr-xr-x. 2 root root 44 May 27 11:48 dir3
drwxr-xr-x. 2 root root 36 May 27 11:48 dir4

/mnt/nfs-test/dir1:
total 1
-rwxrwxrwx. 1 root      root       6 May 27 11:49 1
-rw-r--r--. 1 root      root       0 May 27 01:17 newfile
-rw-r--r--. 1 nfsnobody nfsnobody 17 May 27 01:27 nobody-file

/mnt/nfs-test/dir2:
total 1
-rwxrwxrwx. 1 root root 6 May 27 11:49 2

/mnt/nfs-test/dir3:
ls: /mnt/nfs-test/dir3/3: Permission denied
total 0
-rwxrwx---. 1 root root 0 May 27 01:06 3

/mnt/nfs-test/dir4:
total 0


check the permission denied error for dir3/3



nfs.log glimpse,
[2014-05-26 20:59:03.738805] W [client-rpc-fops.c:1088:client3_3_getxattr_cbk] 0-dist-rep-client-4: remote operation failed: Permission denied. Path: <gfid:c3a3e8c5-0654-4de9-acdf-1a3562d4e1fd> (c3a3e8c5-0654-4de9-acdf-1a3562d4e1fd). Key: (null)
[2014-05-26 20:59:03.739852] W [client-rpc-fops.c:1088:client3_3_getxattr_cbk] 0-dist-rep-client-5: remote operation failed: Permission denied. Path: <gfid:c3a3e8c5-0654-4de9-acdf-1a3562d4e1fd> (c3a3e8c5-0654-4de9-acdf-1a3562d4e1fd). Key: (null)
[2014-05-26 20:59:08.975778] W [client-rpc-fops.c:1088:client3_3_getxattr_cbk] 0-dist-rep-client-4: remote operation failed: Permission denied. Path: <gfid:c3a3e8c5-0654-4de9-acdf-1a3562d4e1fd> (c3a3e8c5-0654-4de9-acdf-1a3562d4e1fd). Key: (null)
[2014-05-26 20:59:08.976858] W [client-rpc-fops.c:1088:client3_3_getxattr_cbk] 0-dist-rep-client-5: remote operation failed: Permission denied. Path: <gfid:c3a3e8c5-0654-4de9-acdf-1a3562d4e1fd> (c3a3e8c5-0654-4de9-acdf-1a3562d4e1fd). Key: (null)


Expected results:
no permission denied expected

Additional info:

--- Additional comment from Saurabh on 2014-05-27 04:03:09 EDT ---



--- Additional comment from RHEL Product and Program Management on 2014-05-27 04:13:31 EDT ---

Since this issue was entered in bugzilla, the release flag has been
set to ? to ensure that it is properly evaluated for this release.

--- Additional comment from John Skeoch on 2014-09-21 18:55:53 EDT ---

User spradhan's account has been closed

--- Additional comment from Niels de Vos on 2014-10-23 08:34:02 EDT ---

I'm not actively working on this yet, moving back to NEW so that others can take it if they want to.

--- Additional comment from Meghana on 2014-11-04 07:15:47 EST ---

The behaviour is seen on all the files that are created following the steps that are listed in the "Steps to reproduce" section. Packet trace shows expected ACCESS reply, allowing read and lookup permissions for the file. There are errors seen in server_getxattr_cbk.Yet to root cause the issue.

--- Additional comment from Niels de Vos on 2014-11-05 10:34:00 EST ---

Capturing what we discussed earlier today. Permission denied comes from xlators/system/posix-acl/src/posix-acl.c:acl_permits():

 ...
 212         for (i = 0; i < acl->count; i++) {
 213                 switch (ace->tag) {
 ...
 245                 case POSIX_ACL_OTHER:
 246                         perm = (ctx->perm & S_IRWXO);
 247                         if (!found)
 248                                 goto perm_check;
 ...
 269 perm_check:
 270         if ((perm & want) == want) {
 271                 goto green;
 272         } else {
 273                 goto red;
 274         }
 ...


acl_permits() lands in the switch statement at POSIX_ACL_OTHER. 'found' is false, so it goes to 'perm_check', where the if-statement results in the 'else' branch.

I do not remember what the 'want' was though... Please attach a tcpdump and output from rpcdebug captured on the NFS-client so that I can review it.

Thanks!

--- Additional comment from Meghana on 2014-11-05 12:25:33 EST ---

The value of want was 4 in this case.

--- Additional comment from Meghana on 2014-11-05 12:54:55 EST ---



--- Additional comment from Meghana on 2014-11-06 03:38:17 EST ---

Attaching rpcdebug output on the NFS client.
Comment 1 Anand Avati 2014-11-06 17:32:31 UTC 
REVIEW: http://review.gluster.org/9064 (When OTHERS are not given any permissions, ACLs are not displayed. Even when there are no permissions to any of the users/groups/others, ACLs and attrs have to be disaplyed. Fixed this by removing unnecessary checks. Signed-off-by: Meghana Madhusudhan <mmadhusu>) posted (#1) for review on master by Meghana M (mmadhusu)
Comment 2 Anand Avati 2014-11-10 10:17:52 UTC 
REVIEW: http://review.gluster.org/9085 (When root-squash is enabled or when no permissions are given to a file, NFS threw permission errors. According to the kernel-nfs behaviour, no permissions were required to read ACLs and other extended attributes.) posted (#1) for review on master by Meghana M (mmadhusu)
Comment 3 Meghana 2014-11-10 10:19:31 UTC 
Created attachment 955752 [details]
kernel-nfs.logs

I have attached kernel-nfs logs here. I performed numerous tests on kernel-nfs and observed a few things.
1. It doesn't require read permissions  for getfacl, ls -l and getfattr commands.
2. Using getfattr commands on an NFS mount, we can only see the nfs_acl attrs.
3. I set extended attributes on the file on a fuse mount. On an NFS mount it is not visible.
4. Extended attrs could not be set on the file on an NFS mount. setfattr commands
are not supported on NFS.
5. nfs4_acl xattrs were visible to the client, without any permissions.
Comment 4 Anand Avati 2014-11-11 09:25:40 UTC 
REVIEW: http://review.gluster.org/9085 (gNFS: Allow reading ACLs even without read permissions on the file.) posted (#2) for review on master by Meghana M (mmadhusu)
Comment 5 Anand Avati 2014-11-12 12:19:45 UTC 
REVIEW: http://review.gluster.org/9085 (gNFS: Allow reading ACLs even without read permissions on the file.) posted (#3) for review on master by Meghana M (mmadhusu)
Comment 6 Anand Avati 2014-11-12 15:24:47 UTC 
REVIEW: http://review.gluster.org/9085 (gNFS: Allow reading ACLs even without read permissions on the file.) posted (#4) for review on master by Niels de Vos (ndevos)
Comment 7 Anand Avati 2014-11-13 06:32:27 UTC 
REVIEW: http://review.gluster.org/9085 (gNFS: Allow reading ACLs even without read permissions on the file.) posted (#5) for review on master by Meghana M (mmadhusu)
Comment 8 Anand Avati 2014-11-13 09:00:54 UTC 
REVIEW: http://review.gluster.org/9085 (gNFS: Allow reading ACLs even without read permissions on the file.) posted (#6) for review on master by Meghana M (mmadhusu)
Comment 9 Anand Avati 2014-11-13 14:01:01 UTC 
REVIEW: http://review.gluster.org/9085 (gNFS: Allow reading ACLs even without read permissions on the file.) posted (#7) for review on master by Niels de Vos (ndevos)
Comment 10 Anand Avati 2014-11-13 19:58:04 UTC 
COMMIT: http://review.gluster.org/9085 committed in master by Niels de Vos (ndevos) 
------
commit 878b30420891c8b00043391cdce90930eaf18795
Author: Meghana Madhusudhan <mmadhusu>
Date:   Mon Nov 10 15:20:51 2014 +0530

    gNFS: Allow reading ACLs even without read permissions on the file.
    
    When root-squash is enabled or when no permissions are given to
    a file, NFS threw permission errors. According to the kernel-nfs
    behaviour, no permissions are required to read ACLs.
    
    When no ACLs are set, the system call sys_lgetxattr fails and
    returns a ENODATA error. This translates to ESERVERFAULT error
    in NFS. Fuse makes an exception to this error and returns a success
    case. Similar changes are made here to achieve the expected behaviour.
    
    Change-Id: I46b8f5911114eb087a3f8ca4e921b6b41e83f3b3
    BUG: 1161092
    Signed-off-by: Meghana Madhusudhan <mmadhusu>
    Signed-off-by: Niels de Vos <ndevos>
    Reviewed-on: http://review.gluster.org/9085
    Tested-by: Gluster Build System <jenkins.com>
Comment 11 Meghana 2014-11-14 05:12:59 UTC 
Thanks a lot to Niels and Soumya for their help and timely reviews!
Comment 12 Niels de Vos 2015-05-14 17:28:20 UTC 
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.0, please open a new bug report.

glusterfs-3.7.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/10939
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user
Comment 13 Niels de Vos 2015-05-14 17:35:41 UTC 
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.0, please open a new bug report.

glusterfs-3.7.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/10939
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user
Comment 14 Niels de Vos 2015-05-14 17:38:04 UTC 
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.0, please open a new bug report.

glusterfs-3.7.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/10939
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user
Comment 15 Niels de Vos 2015-05-14 17:44:36 UTC 
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.0, please open a new bug report.

glusterfs-3.7.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/10939
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user