Description of problem:
As discussed,Validated the usecase reported in upstream BZ[1],with current downstream bits,issue exist in downstream bits we well.Hence raising downstream BZ for the same.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1405147
While glusterfs volume is exported via ganesha,if the nfs-client mounting it, tries to copy an owned read-only file -not being root user-, a permission denied error is been observed.
The same use case is passed when the user is root.
Version-Release number of selected component (if applicable):
nfs-ganesha-gluster-2.5.5-7.el7rhgs.x86_64
nfs-ganesha-debuginfo-2.5.5-7.el7rhgs.x86_64
nfs-ganesha-2.5.5-7.el7rhgs.x86_64
glusterfs-ganesha-3.12.2-11.el7rhgs.x86_64
How reproducible:
2/2
Steps to Reproduce:
1.Create 6 node ganesha cluster
2.Create Distributed-Replicate volume
3.Export the volume via ganesha
4.Mount the volume on nfs client via v4 protocol
5.Change the mount dir permission to chmod 777
6.Create a user "mani"
7.Switch user to "mani" and create a file from user "mani".Change the permission of the file as readonly.Then copy the file to some other file from same user
Actual results:
A user who owns a file that is read-only cannot copy it to a new file.
Expected results:
A user who owns a file that is read-only should be able to copy it even if it is read-only.
Additional info:
When tested on NFS client from non-root-user -Test FAILED
--------------
[mani@rhs-client6 ganesha]$ ls
[mani@rhs-client6 ganesha]$ rm -f kk.txt 444.txt; echo "prueba" > 444.txt; chmod 444 444.txt; cp -p 444.txt kk.txt; ls -ld 444.txt kk.txt
cp: failed to close ‘kk.txt’: Permission denied
-r--r--r--. 1 mani mani 7 May 20 2018 444.txt
-r--r--r--. 1 mani mani 0 May 20 2018 kk.txt
--------------
When tested on NFS client from root user -Test PASSED
-------------
[root@rhs-client6 ganesha]# rm -f kk.txt 444.txt; echo "prueba" > 444.txt; chmod 444 444.txt; cp -p 444.txt kk.txt; ls -ld 444.txt kk.txt
-r--r--r--. 1 root root 7 May 20 2018 444.txt
-r--r--r--. 1 root root 7 May 20 2018 kk.txt
[root@rhs-client6 ganesha]#
---------------
When tested with Glusterfs client mount with non-root user - Test PASSED
-----------------
[root@rhs-client6 mnt]# su mani
[mani@rhs-client6 mnt]$ cd glusterfs-mount/
[mani@rhs-client6 glusterfs-mount]$ ls
[mani@rhs-client6 glusterfs-mount]$ rm -f kk.txt 444.txt; echo "prueba" > 444.txt; chmod 444 444.txt; cp -p 444.txt kk.txt; ls -ld 444.txt kk.txt
-r--r--r--. 1 mani mani 7 May 20 2018 444.txt
-r--r--r--. 1 mani mani 7 May 20 2018 kk.txt
[mani@rhs-client6 glusterfs-mount]$
[mani@rhs-client6 glusterfs-mount]$ exit
exit
[root@rhs-client6 mnt]# df -hT
Filesystem Type Size Used Avail Use% Mounted on
/dev/mapper/rhel_rhs--client6-root xfs 50G 1.6G 49G 4% /
devtmpfs devtmpfs 7.8G 0 7.8G 0% /dev
tmpfs tmpfs 7.8G 0 7.8G 0% /dev/shm
tmpfs tmpfs 7.8G 9.5M 7.8G 1% /run
tmpfs tmpfs 7.8G 0 7.8G 0% /sys/fs/cgroup
/dev/sda1 xfs 1014M 143M 872M 15% /boot
/dev/mapper/rhel_rhs--client6-home xfs 1.8T 133M 1.8T 1% /home
tmpfs tmpfs 1.6G 0 1.6G 0% /run/user/0
moonshine.lab.eng.blr.redhat.com:Ganeshavol1 fuse.glusterfs 4.6T 49G 4.6T 2% /mnt/glusterfs-mount
Ganesha-gfapi.logs for the test failure as non root user and nfs mount
----------------
[2018-05-20 11:35:41.965005] E [dht-helper.c:90:dht_fd_ctx_set] (-->/usr/lib64/glusterfs/3.12.2/xlator/cluster/replicate.so(+0x2e97c) [0x7f0c582fd97c] -->/usr/lib64/glusterfs/3.12.2/xlator/cluster/distribute.so(+0x6e91b) [0x7f0bc0bc691b] -->/usr/lib64/glusterfs/3.12.2/xlator/cluster/distribute.so(+0x7936) [0x7f0bc0b5f936] ) 1-Ganeshavol1-dht: invalid argument: fd [Invalid argument]
[2018-05-20 11:35:41.993847] E [MSGID: 114031] [client-rpc-fops.c:435:client3_3_open_cbk] 1-Ganeshavol1-client-9: remote operation failed. Path: /kk.txt (d2198cc5-acb5-4c36-9b58-9bb4e039521e) [Permission denied]
[2018-05-20 11:35:41.993883] E [MSGID: 114031] [client-rpc-fops.c:435:client3_3_open_cbk] 1-Ganeshavol1-client-11: remote operation failed. Path: /kk.txt (d2198cc5-acb5-4c36-9b58-9bb4e039521e) [Permission denied]
[2018-05-20 11:35:41.993931] E [MSGID: 114031] [client-rpc-fops.c:435:client3_3_open_cbk] 1-Ganeshavol1-client-10: remote operation failed. Path: /kk.txt (d2198cc5-acb5-4c36-9b58-9bb4e039521e) [Permission denied]
[2018-05-20 11:47:59.849296] E [dht-helper.c:90:dht_fd_ctx_set] (-->/usr/lib64/glusterfs/3.12.2/xlator/cluster/replicate.so(+0x2e97c) [0x7f0c582fd97c] -->/usr/lib64/glusterfs/3.12.2/xlator/cluster/distribute.so(+0x6e91b) [0x7f0bc0bc691b] -->/usr/lib64/glusterfs/3.12.2/xlator/cluster/distribute.so(+0x7936) [0x7f0bc0b5f936] ) 1-Ganeshavol1-dht: invalid argument: fd [Invalid argument]
-------------------
Comment 2Daniel Gryniewicz
2018-05-21 12:32:07 UTC
This appears to be a gfapi issue, not a ganesha issue, since the permission denied came from gfapi.
Its a bug in open-behind. NFS Ganesha is issuing a setattr call. Open-behind implements fsetattr, but not setattr. So, pending opens are not done before setattr resulting in open failures due to change in permissions. Component can be changed to Open-behind
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHSA-2018:2607