Description of problem: When a gluster volume is exported over NFS (tried both kernel nfs and nfs ganesha) and a client copies data onto the NFS mounted filesystem, it shows an error when copying a file that only has read rights for the owner and the file is not correctly copied. It is only 0 bytes long on the target. I've assigned this bug to nfs-ganesha but I don't think this is ganesha specific as I can reproduce the problem when the gluster filesystem is exported using kernel nfs. Version-Release number of selected component (if applicable): gluster 3.12.5 (fedora 27 packages) How reproducible: Always Steps to Reproduce: 1. Create a gluster volume and export it over NFS using either nfs-ganesha or kernel nfs 2. Mount it on a client and as a regular user execute the following commands on the nfs-mounted filesystem: mkdir source dd if=/dev/zero of=source/testfile-rw bs=1M count=2 chmod 600 source/testfile-rw cp source/testfile-rw source/testfile-ro chmod 400 source/testfile-ro cp -ar source target 3. the cp command will show: cp: failed to close ‘target/testfile-ro’: Permission denied Actual results: The above error message is shown and the target/testfile-ro file is now 0 bytes long. Expected results: No error message and the file copied correctly. Additional info: When the client uses the fuse client to mount the glusterfs filesystem, this works correctly.
I'm not sure what you mean by export a gluster volume by kernel nfs. That's not possible. Gluster volumes are exportable as native gluster w/ FUSE client, using legacy gluster NFS (NFSv3 only), and using nfs-ganesha. (I hope you're not exporting the gluster brick through kernel nfs, or with ganesha+FSAL_VFS.) Please attach your /etc/ganesha/ganesha.conf file
Hi, I first experienced the bug when using NFS ganesha. The export from ganesha.conf: EXPORT { Export_Id = 1; Path = "/"; FSAL { Name = GLUSTER; hostname = "ox-gluster.esat.kuleuven.be"; volume = "vol_rep3_arb"; } Access_Type = RW; Pseudo = "/vol_rep3_arb"; SecType = "sys"; CLIENT { Clients = @esat.nfs.dns; } } I then tried to reproduce the issue using a glusterfs mount on one of the brick servers, but was unable to reproduce it. Then I mounted it using the fuse client on one of the brick servers and exported the mountpoint using kernel nfs. When mounting it like this, I was able to reproduce it. So I can reproduce the issue with both NFS implementations, but not when using the fuse client. Regards, Rik
I've just reproduced it again using nfs-ganesha and the ganesha server logs the following error when I reproduce it: 13/02/2018 20:15:47 : epoch 5a83381b : tiger.esat.kuleuven.be : ganesha.nfsd-2308[work-106] glusterfs_close_my_fd :FSAL :CRIT :Error : close returns with Permission denied 13/02/2018 20:15:47 : epoch 5a83381b : tiger.esat.kuleuven.be : ganesha.nfsd-2308[work-112] glusterfs_process_acl :FSAL :CRIT :setattr acl is NULL 13/02/2018 20:15:47 : epoch 5a83381b : tiger.esat.kuleuven.be : ganesha.nfsd-2308[work-112] glusterfs_setattr2 :FSAL :CRIT :setattrs failed with error Success
Hi, This bug looks similar to the issue reported here: https://github.com/nfs-ganesha/nfs-ganesha/issues/262 Regards, Rik
I looked at this "briefly." When copying a 0444 file over gluster native/fuse, the destination file is created with open(...,O_CREAT)* and has acl perms that allow the open() to succeed. But when copying the same file over NFS (nfs-ganesha, fsal_gluster, gfapi) the file is created with creat() (create fop) and has insufficient acl perms to allow the creat() to succeed. In the time that I spent looking at it I was unable to find where in gluster those initial acl perms come from or why open(...,O_CREAT) has different default perms than creat().
Can you point me at where the initial acl perms are set? Thanks
I am not sure, may be one of the maintainers of posix-acl may know it. Adding needinfo on them.
I also meant to add that I may have open() and creat() reversed in comment 5. Regardless, one way gets the necessary acl perms to allow the create, the other one does not.
Release 3.12 has been EOLd and this bug was still found to be in the NEW state, hence moving the version to mainline, to triage the same and take appropriate actions.
Any update? This has been open since February.
The PR mentioned in bug will be rcaed in https://bugzilla.redhat.com/show_bug.cgi?id=1735480. Even I suspect using different fd can cause this issue than an issue with posix acl xlator. Hence adding the dependency on that bug. Will wait and see once required changes got merged then it can be tested again. And changing component to nfs-ganesha again
this was finally fixed in glusterfs-6 with a work-around pending the real fix in glusterfs-7 IIRC