Bug 1543996 - truncates read-only files on copy
Summary: truncates read-only files on copy
Alias: None
Product: GlusterFS
Classification: Community
Component: ganesha-nfs
Version: mainline
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
Assignee: Jiffin
QA Contact:
Depends On: 1735480 1753569
TreeView+ depends on / blocked
Reported: 2018-02-09 19:11 UTC by Rik Theys
Modified: 2019-11-22 16:04 UTC (History)
12 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed: 2019-11-22 16:04:56 UTC
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
spalai: needinfo-

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Github https://github.com/nfs-ganesha nfs-ganesha issues 262 0 None None None 2019-11-22 16:02:56 UTC
Red Hat Bugzilla 1405147 0 unspecified CLOSED glusterfs (posix-acl xlator layer) checks for "write permission" instead for "file owner" during open() when writing to ... 2021-02-22 00:41:40 UTC

Internal Links: 1405147

Description Rik Theys 2018-02-09 19:11:20 UTC
Description of problem:

When a gluster volume is exported over NFS (tried both kernel nfs and nfs ganesha) and a client copies data onto the NFS mounted filesystem, it shows an error when copying a file that only has read rights for the owner and the file is not correctly copied. It is only 0 bytes long on the target.

I've assigned this bug to nfs-ganesha but I don't think this is ganesha specific as I can reproduce the problem when the gluster filesystem is exported using kernel nfs.

Version-Release number of selected component (if applicable):
gluster 3.12.5 (fedora 27 packages)

How reproducible:

Steps to Reproduce:
1. Create a gluster volume and export it over NFS using either nfs-ganesha or kernel nfs
2. Mount it on a client and as a regular user execute the following commands on the nfs-mounted filesystem:

mkdir source
dd if=/dev/zero of=source/testfile-rw bs=1M count=2
chmod 600 source/testfile-rw
cp source/testfile-rw source/testfile-ro
chmod 400 source/testfile-ro
cp -ar source target

3. the cp command will show: 
cp: failed to close ‘target/testfile-ro’: Permission denied

Actual results:
The above error message is shown and the target/testfile-ro file is now 0 bytes long.

Expected results:
No error message and the file copied correctly.

Additional info:
When the client uses the fuse client to mount the glusterfs filesystem, this works correctly.

Comment 1 Kaleb KEITHLEY 2018-02-13 16:39:10 UTC
I'm not sure what you mean by export a gluster volume by kernel nfs. That's not possible.

Gluster volumes are exportable as native gluster w/ FUSE client, using legacy gluster NFS (NFSv3 only), and using nfs-ganesha.

(I hope you're not exporting the gluster brick through kernel nfs, or with ganesha+FSAL_VFS.)

Please attach your /etc/ganesha/ganesha.conf file

Comment 2 Rik Theys 2018-02-13 19:13:29 UTC

I first experienced the bug when using NFS ganesha. The export from ganesha.conf:

        Export_Id = 1;
        Path = "/";
        FSAL {
                Name = GLUSTER;
                hostname = "ox-gluster.esat.kuleuven.be";
                volume = "vol_rep3_arb";

        Access_Type = RW;
        Pseudo = "/vol_rep3_arb";
        SecType = "sys";

        CLIENT {
                Clients = @esat.nfs.dns;

I then tried to reproduce the issue using a glusterfs mount on one of the brick servers, but was unable to reproduce it.

Then I mounted it using the fuse client on one of the brick servers and exported the mountpoint using kernel nfs. When mounting it like this, I was able to reproduce it.

So I can reproduce the issue with both NFS implementations, but not when using the fuse client.



Comment 3 Rik Theys 2018-02-13 19:16:46 UTC
I've just reproduced it again using nfs-ganesha and the ganesha server logs the following error when I reproduce it:

13/02/2018 20:15:47 : epoch 5a83381b : tiger.esat.kuleuven.be : ganesha.nfsd-2308[work-106] glusterfs_close_my_fd :FSAL :CRIT :Error : close returns with Permission denied
13/02/2018 20:15:47 : epoch 5a83381b : tiger.esat.kuleuven.be : ganesha.nfsd-2308[work-112] glusterfs_process_acl :FSAL :CRIT :setattr acl is NULL
13/02/2018 20:15:47 : epoch 5a83381b : tiger.esat.kuleuven.be : ganesha.nfsd-2308[work-112] glusterfs_setattr2 :FSAL :CRIT :setattrs failed with error Success

Comment 4 Rik Theys 2018-03-03 20:53:00 UTC

This bug looks similar to the issue reported here:



Comment 5 Kaleb KEITHLEY 2018-03-05 13:17:20 UTC
I looked at this "briefly."

When copying a 0444 file over gluster native/fuse, the destination file is created with open(...,O_CREAT)* and has acl perms that allow the open() to succeed.

But when copying the same file over NFS (nfs-ganesha, fsal_gluster, gfapi) the file is created with creat() (create fop) and has insufficient acl perms to allow the creat() to succeed.

In the time that I spent looking at it I was unable to find where in gluster those initial acl perms come from or why open(...,O_CREAT) has different default perms than creat().

Comment 6 Kaleb KEITHLEY 2018-03-05 13:20:48 UTC
Can you point me at where the initial acl perms are set? Thanks

Comment 7 Pranith Kumar K 2018-03-05 13:25:05 UTC
I am not sure, may be one of the maintainers of posix-acl may know it. Adding needinfo on them.

Comment 8 Kaleb KEITHLEY 2018-03-05 13:29:15 UTC
I also meant to add that I may have open() and creat() reversed in comment 5. Regardless, one way gets the necessary acl perms to allow the create, the other one does not.

Comment 9 Shyamsundar 2018-10-23 14:54:52 UTC
Release 3.12 has been EOLd and this bug was still found to be in the NEW state, hence moving the version to mainline, to triage the same and take appropriate actions.

Comment 10 Kaleb KEITHLEY 2019-08-19 14:10:26 UTC
Any update? This has been open since February.

Comment 12 Jiffin 2019-09-18 04:35:57 UTC
The PR mentioned in bug will be rcaed in https://bugzilla.redhat.com/show_bug.cgi?id=1735480. Even I suspect using different fd can cause this issue than an issue with posix acl xlator.
Hence adding the dependency on that bug. Will wait and see once required changes got merged then it can be tested again. And changing component to nfs-ganesha again

Comment 13 Kaleb KEITHLEY 2019-11-22 16:04:56 UTC
this was finally fixed in glusterfs-6 with a work-around pending the real fix in glusterfs-7 IIRC

Note You need to log in before you can comment on or make changes to this bug.