Bug 1543996

I'm not sure what you mean by export a gluster volume by kernel nfs. That's not possible.

Gluster volumes are exportable as native gluster w/ FUSE client, using legacy gluster NFS (NFSv3 only), and using nfs-ganesha.

(I hope you're not exporting the gluster brick through kernel nfs, or with ganesha+FSAL_VFS.)

Please attach your /etc/ganesha/ganesha.conf file

Hi,

I first experienced the bug when using NFS ganesha. The export from ganesha.conf:

EXPORT {
        Export_Id = 1;
        Path = "/";
        
        FSAL {
                Name = GLUSTER;
                hostname = "ox-gluster.esat.kuleuven.be";
                volume = "vol_rep3_arb";
        }

        Access_Type = RW;
        Pseudo = "/vol_rep3_arb";
        SecType = "sys";

        CLIENT {
                Clients = @esat.nfs.dns;
        }
}

I then tried to reproduce the issue using a glusterfs mount on one of the brick servers, but was unable to reproduce it.

Then I mounted it using the fuse client on one of the brick servers and exported the mountpoint using kernel nfs. When mounting it like this, I was able to reproduce it.

So I can reproduce the issue with both NFS implementations, but not when using the fuse client.

Regards,

Rik

I've just reproduced it again using nfs-ganesha and the ganesha server logs the following error when I reproduce it:

13/02/2018 20:15:47 : epoch 5a83381b : tiger.esat.kuleuven.be : ganesha.nfsd-2308[work-106] glusterfs_close_my_fd :FSAL :CRIT :Error : close returns with Permission denied
13/02/2018 20:15:47 : epoch 5a83381b : tiger.esat.kuleuven.be : ganesha.nfsd-2308[work-112] glusterfs_process_acl :FSAL :CRIT :setattr acl is NULL
13/02/2018 20:15:47 : epoch 5a83381b : tiger.esat.kuleuven.be : ganesha.nfsd-2308[work-112] glusterfs_setattr2 :FSAL :CRIT :setattrs failed with error Success

Hi,

This bug looks similar to the issue reported here:
https://github.com/nfs-ganesha/nfs-ganesha/issues/262

Regards,

Rik

I looked at this "briefly."

When copying a 0444 file over gluster native/fuse, the destination file is created with open(...,O_CREAT)* and has acl perms that allow the open() to succeed.

But when copying the same file over NFS (nfs-ganesha, fsal_gluster, gfapi) the file is created with creat() (create fop) and has insufficient acl perms to allow the creat() to succeed.

In the time that I spent looking at it I was unable to find where in gluster those initial acl perms come from or why open(...,O_CREAT) has different default perms than creat().

Can you point me at where the initial acl perms are set? Thanks

I am not sure, may be one of the maintainers of posix-acl may know it. Adding needinfo on them.

I also meant to add that I may have open() and creat() reversed in comment 5. Regardless, one way gets the necessary acl perms to allow the create, the other one does not.

Release 3.12 has been EOLd and this bug was still found to be in the NEW state, hence moving the version to mainline, to triage the same and take appropriate actions.

Any update? This has been open since February.

The PR mentioned in bug will be rcaed in https://bugzilla.redhat.com/show_bug.cgi?id=1735480. Even I suspect using different fd can cause this issue than an issue with posix acl xlator.
Hence adding the dependency on that bug. Will wait and see once required changes got merged then it can be tested again. And changing component to nfs-ganesha again

this was finally fixed in glusterfs-6 with a work-around pending the real fix in glusterfs-7 IIRC