Description of problem: As discussed,Validated the usecase reported in upstream BZ[1],with current downstream bits,issue exist in downstream bits we well.Hence raising downstream BZ for the same. [1] https://bugzilla.redhat.com/show_bug.cgi?id=1405147 While glusterfs volume is exported via ganesha,if the nfs-client mounting it, tries to copy an owned read-only file -not being root user-, a permission denied error is been observed. The same use case is passed when the user is root. Version-Release number of selected component (if applicable): nfs-ganesha-gluster-2.5.5-7.el7rhgs.x86_64 nfs-ganesha-debuginfo-2.5.5-7.el7rhgs.x86_64 nfs-ganesha-2.5.5-7.el7rhgs.x86_64 glusterfs-ganesha-3.12.2-11.el7rhgs.x86_64 How reproducible: 2/2 Steps to Reproduce: 1.Create 6 node ganesha cluster 2.Create Distributed-Replicate volume 3.Export the volume via ganesha 4.Mount the volume on nfs client via v4 protocol 5.Change the mount dir permission to chmod 777 6.Create a user "mani" 7.Switch user to "mani" and create a file from user "mani".Change the permission of the file as readonly.Then copy the file to some other file from same user Actual results: A user who owns a file that is read-only cannot copy it to a new file. Expected results: A user who owns a file that is read-only should be able to copy it even if it is read-only. Additional info: When tested on NFS client from non-root-user -Test FAILED -------------- [mani@rhs-client6 ganesha]$ ls [mani@rhs-client6 ganesha]$ rm -f kk.txt 444.txt; echo "prueba" > 444.txt; chmod 444 444.txt; cp -p 444.txt kk.txt; ls -ld 444.txt kk.txt cp: failed to close ‘kk.txt’: Permission denied -r--r--r--. 1 mani mani 7 May 20 2018 444.txt -r--r--r--. 1 mani mani 0 May 20 2018 kk.txt -------------- When tested on NFS client from root user -Test PASSED ------------- [root@rhs-client6 ganesha]# rm -f kk.txt 444.txt; echo "prueba" > 444.txt; chmod 444 444.txt; cp -p 444.txt kk.txt; ls -ld 444.txt kk.txt -r--r--r--. 1 root root 7 May 20 2018 444.txt -r--r--r--. 1 root root 7 May 20 2018 kk.txt [root@rhs-client6 ganesha]# --------------- When tested with Glusterfs client mount with non-root user - Test PASSED ----------------- [root@rhs-client6 mnt]# su mani [mani@rhs-client6 mnt]$ cd glusterfs-mount/ [mani@rhs-client6 glusterfs-mount]$ ls [mani@rhs-client6 glusterfs-mount]$ rm -f kk.txt 444.txt; echo "prueba" > 444.txt; chmod 444 444.txt; cp -p 444.txt kk.txt; ls -ld 444.txt kk.txt -r--r--r--. 1 mani mani 7 May 20 2018 444.txt -r--r--r--. 1 mani mani 7 May 20 2018 kk.txt [mani@rhs-client6 glusterfs-mount]$ [mani@rhs-client6 glusterfs-mount]$ exit exit [root@rhs-client6 mnt]# df -hT Filesystem Type Size Used Avail Use% Mounted on /dev/mapper/rhel_rhs--client6-root xfs 50G 1.6G 49G 4% / devtmpfs devtmpfs 7.8G 0 7.8G 0% /dev tmpfs tmpfs 7.8G 0 7.8G 0% /dev/shm tmpfs tmpfs 7.8G 9.5M 7.8G 1% /run tmpfs tmpfs 7.8G 0 7.8G 0% /sys/fs/cgroup /dev/sda1 xfs 1014M 143M 872M 15% /boot /dev/mapper/rhel_rhs--client6-home xfs 1.8T 133M 1.8T 1% /home tmpfs tmpfs 1.6G 0 1.6G 0% /run/user/0 moonshine.lab.eng.blr.redhat.com:Ganeshavol1 fuse.glusterfs 4.6T 49G 4.6T 2% /mnt/glusterfs-mount Ganesha-gfapi.logs for the test failure as non root user and nfs mount ---------------- [2018-05-20 11:35:41.965005] E [dht-helper.c:90:dht_fd_ctx_set] (-->/usr/lib64/glusterfs/3.12.2/xlator/cluster/replicate.so(+0x2e97c) [0x7f0c582fd97c] -->/usr/lib64/glusterfs/3.12.2/xlator/cluster/distribute.so(+0x6e91b) [0x7f0bc0bc691b] -->/usr/lib64/glusterfs/3.12.2/xlator/cluster/distribute.so(+0x7936) [0x7f0bc0b5f936] ) 1-Ganeshavol1-dht: invalid argument: fd [Invalid argument] [2018-05-20 11:35:41.993847] E [MSGID: 114031] [client-rpc-fops.c:435:client3_3_open_cbk] 1-Ganeshavol1-client-9: remote operation failed. Path: /kk.txt (d2198cc5-acb5-4c36-9b58-9bb4e039521e) [Permission denied] [2018-05-20 11:35:41.993883] E [MSGID: 114031] [client-rpc-fops.c:435:client3_3_open_cbk] 1-Ganeshavol1-client-11: remote operation failed. Path: /kk.txt (d2198cc5-acb5-4c36-9b58-9bb4e039521e) [Permission denied] [2018-05-20 11:35:41.993931] E [MSGID: 114031] [client-rpc-fops.c:435:client3_3_open_cbk] 1-Ganeshavol1-client-10: remote operation failed. Path: /kk.txt (d2198cc5-acb5-4c36-9b58-9bb4e039521e) [Permission denied] [2018-05-20 11:47:59.849296] E [dht-helper.c:90:dht_fd_ctx_set] (-->/usr/lib64/glusterfs/3.12.2/xlator/cluster/replicate.so(+0x2e97c) [0x7f0c582fd97c] -->/usr/lib64/glusterfs/3.12.2/xlator/cluster/distribute.so(+0x6e91b) [0x7f0bc0bc691b] -->/usr/lib64/glusterfs/3.12.2/xlator/cluster/distribute.so(+0x7936) [0x7f0bc0b5f936] ) 1-Ganeshavol1-dht: invalid argument: fd [Invalid argument] -------------------
This appears to be a gfapi issue, not a ganesha issue, since the permission denied came from gfapi.
Its a bug in open-behind. NFS Ganesha is issuing a setattr call. Open-behind implements fsetattr, but not setattr. So, pending opens are not done before setattr resulting in open failures due to change in permissions. Component can be changed to Open-behind
upstream patch: https://review.gluster.org/#/c/20084/
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2018:2607