Bug 1030443

Summary: NFS_ACL : Inconsistent behaviour of setfacl and getfacl.
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Anil Shah <ashah>
Component: glusterdAssignee: santosh pradhan <spradhan>
Status: CLOSED ERRATA QA Contact: Anil Shah <ashah>
Severity: urgent Docs Contact:
Priority: high    
Version: 2.1CC: grajaiya, rtalur, spradhan, vagarwal, vbellur, vraman
Target Milestone: ---Keywords: Reopened, TestBlocker, ZStream
Target Release: RHGS 2.1.2   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: 3.4.0.47.1u2rhs-1 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1035218 (view as bug list) Environment:
Last Closed: 2014-02-25 08:03:42 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1032984, 1035218    

Description Anil Shah 2013-11-14 13:05:29 UTC
Description of problem:

While trying to run setfacl and getfacl command NFS ACLs behaviour is inconsistent
.
Version-Release number of selected component (if applicable):
glusterfs 3.4.0.43.1u2rhs

How reproducible:

create user acltest_user2
create file dot
Mount the gluster as NFS
mount -t nfs -o acl,vers=3 10.70.36.231:dist-rep2 /mnt/acl2/

[root@rhsauto010 acl2]# touch dot
[root@rhsauto010 acl2]# setfacl -m u:acltest_user2:rwx dot
[root@rhsauto010 acl2]# getfacl dot
getfacl: dot: Invalid argument
[root@rhsauto010 acl2]# setfacl -m u:acltest_user2:rwx dot
setfacl: dot: Invalid argument


Actual results:

getfacl and setfacl behaviour is inconsistent.

Expected results:

getfacl and setfacl command should execute successfully. 

Additional info:


[root@rhsauto001 ~]# gluster vol info
 
Volume Name: dist-rep2
Type: Distributed-Replicate
Volume ID: 65a2f89e-7d92-4632-b10e-bfc7119a8f9a
Status: Started
Number of Bricks: 6 x 2 = 12
Transport-type: tcp
Bricks:
Brick1: 10.70.36.236:/rhs/brick1/d2r12
Brick2: 10.70.36.237:/rhs/brick1/d2r22
Brick3: 10.70.36.236:/rhs/brick1/d4r12
Brick4: 10.70.36.237:/rhs/brick1/d4r22
Brick5: 10.70.36.236:/rhs/brick1/d6r12
Brick6: 10.70.36.237:/rhs/brick1/d6r22
Brick7: 10.70.36.231:/rhs/brick1/d1r12
Brick8: 10.70.36.233:/rhs/brick1/d1r22
Brick9: 10.70.36.231:/rhs/brick1/d3r12
Brick10: 10.70.36.233:/rhs/brick1/d3r22
Brick11: 10.70.36.231:/rhs/brick1/d5r12
Brick12: 10.70.36.233:/rhs/brick1/d5r22
Options Reconfigured:
nfs.export-dir: /test(localhost)
nfs.export-volumes: on
nfs.acl: on


[root@rhsauto001 ~]# rpcinfo -p
   program vers proto   port  service
    100000    4   tcp    111  portmapper
    100000    3   tcp    111  portmapper
    100000    2   tcp    111  portmapper
    100000    4   udp    111  portmapper
    100000    3   udp    111  portmapper
    100000    2   udp    111  portmapper
    100005    3   tcp  38465  mountd
    100005    1   tcp  38466  mountd
    100003    3   tcp   2049  nfs
    100024    1   udp  51728  status
    100024    1   tcp  52310  status
    100021    4   tcp  38468  nlockmgr
    100227    3   tcp   2049  nfs_acl
    100021    1   udp    735  nlockmgr
    100021    1   tcp    738  nlockmgr

Comment 3 Raghavendra Talur 2013-11-18 12:09:38 UTC
Hi Anil,

Can you re-run the test after this command and see if it still fails?
Command: gluster volume set <VOLNAME> stat-prefetch off

Comment 4 Anil Shah 2013-11-18 12:28:03 UTC
No luck.

[root@rhsauto001 ~]# gluster volume set dist-rep2 stat-prefetch off
volume set: success


[root@rhsauto010 acl2]# touch dot
[root@rhsauto010 acl2]# getfacl dot
# file: dot
# owner: root
# group: root
user::rw-
group::r--
other::r--

[root@rhsauto010 acl2]# setfacl -m u:adm:rw dot 
[root@rhsauto010 acl2]# getfacl dot
# file: dot
# owner: root
# group: root
user::rw-
user:adm:rw-
group::r--
mask::rw-
other::r--

[root@rhsauto010 acl2]# setfacl -m u:adm:rw dot
[root@rhsauto010 acl2]# getfacl dot
getfacl: dot: Invalid argument
[root@rhsauto010 acl2]# rm -rf dot
[root@rhsauto010 acl2]# touch dot
[root@rhsauto010 acl2]# setfacl -m u:acltest_user2:rwx dot
[root@rhsauto010 acl2]# getfacl dot
getfacl: dot: Invalid argument

Comment 5 santosh pradhan 2013-11-21 10:53:43 UTC
The pcap shows that the permissions are getting set improperly by posix-acl i.e. in glusterfs backend (brick). The NFS is working as expected. Except permissions, everything else look good.

The permissions needs to be masked properly in the backend/brick process.

Comment 6 santosh pradhan 2013-11-21 11:31:13 UTC
When I fixed the permissions properly the getfacl/setfacl worked in my local workstation. When I tested in the upstream code, the commands getfacl/sefacl just worked without any change. That means the issue is already fixed upstream. Little more code investigation shows the issue is really fixed upstream. 

Ported the fix downstream (see BZ 1032984) and its up for review:

https://code.engineering.redhat.com/gerrit/#/c/16121/

Comment 7 santosh pradhan 2013-11-27 11:07:42 UTC
Posted another FIX to this:
http://review.gluster.org/#/c/6368/

Comment 8 Anil Shah 2013-11-28 06:16:51 UTC
Tried running setfacl and getfacl on build glusterfs 3.4.0.44.1u2rhs.
However behaviour is still in-consistent. 

[root@rhsauto010 acl2]# setfacl -m u:adm:rw dot
[root@rhsauto010 acl2]# getfacl dot
# file: dot
# owner: root
# group: root
user::rw-
user:adm:rw-
group::r--
mask::rw-
other::r--

[root@rhsauto010 acl2]# getfacl dot
getfacl: dot: Invalid argument

Comment 9 santosh pradhan 2013-12-04 03:09:18 UTC
Posted the patch for review:

https://code.engineering.redhat.com/gerrit/16748

Comment 10 Anil Shah 2013-12-09 06:48:09 UTC
Behaviour is Consistent. 

[root@rhsauto010 acl2]# setfacl -m u:adm:rwx abc
[root@rhsauto010 acl2]# getfacl abc
# file: abc
# owner: root
# group: root
user::rw-
user:adm:rwx
group::r--
mask::rwx
other::r--

[root@rhsauto010 acl2]# setfacl -m u:adm:rwx abc
[root@rhsauto010 acl2]# getfacl abc
# file: abc
# owner: root
# group: root
user::rw-
user:adm:rwx
group::r--
mask::rwx
other::r--

[root@rhsauto010 acl2]# getfacl abc
# file: abc
# owner: root
# group: root
user::rw-
user:adm:rwx
group::r--
mask::rwx
other::r--

[root@rhsauto010 acl2]# getfacl abc
# file: abc
# owner: root
# group: root
user::rw-
user:adm:rwx
group::r--
mask::rwx
other::r--

Comment 11 Anil Shah 2013-12-09 06:49:31 UTC
Closing this Bug.

Comment 12 Gowrishankar Rajaiyan 2013-12-09 07:52:25 UTC
Changing state to ON_QA to have it addressed appropriately.

Comment 13 Anil Shah 2013-12-13 06:41:58 UTC
NFS ACL Behaviour is Consistent.

[root@rhsauto010 acl2]# setfacl -m u:adm:rwx dot
[root@rhsauto010 acl2]# getfacl dot
# file: dot
# owner: root
# group: root
user::rw-
user:adm:rwx
group::r--
mask::rwx
other::r--

[root@rhsauto010 acl2]# setfacl -m u:adm:rwx dot
[root@rhsauto010 acl2]# getfacl dot
# file: dot
# owner: root
# group: root
user::rw-
user:adm:rwx
group::r--
mask::rwx
other::r--

[root@rhsauto010 acl2]# getfacl dot
# file: dot
# owner: root
# group: root
user::rw-
user:adm:rwx
group::r--
mask::rwx
other::r--

[root@rhsauto010 acl2]# setfacl -m u:gopher:rw dot
[root@rhsauto010 acl2]# getfacl dot
# file: dot
# owner: root
# group: root
user::rw-
user:adm:rwx
user:gopher:rw-
group::r--
mask::rwx
other::r--

[root@rhsauto010 acl2]# setfacl -m u:gopher:rw dot
[root@rhsauto010 acl2]# getfacl dot
# file: dot
# owner: root
# group: root
user::rw-
user:adm:rwx
user:gopher:rw-
group::r--
mask::rwx
other::r--


Bug verified on build glusterfs 3.4.0.49rhs.

Comment 15 errata-xmlrpc 2014-02-25 08:03:42 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2014-0208.html