Bug 1317048

The error in setting the ACL is coming from this message:
[posix-helpers.c:1177:posix_handle_pair] 0-sftp_rep_vol01-posix: <filename>: key:system.posix_acl_accessflags: 0 length:212 [Invalid argument]

This indicates a failure in setxattr syscall, which is returning the error EINVAL here.

Tried a few things on my Fedora box (although not representative of the actual problem), and here are the results.

1) Created ~30 users (with names testuser0$i)
2) Used setfacl -m u:<username>:rwx <dirpath>

- When <dirpath> belonged to an ext4 filesystem this passed
  - getfacl also passed
  - but poking around using getfattr to see how the xattr was stored, resulted in it showing no xattr for this name.
  - I assume ext4 stores this differently
- When <dirpath> belonged to an xfs filesystem
  - The set failed after 20 users were updated.
  - The error thought was "Argument list too long", possibly due to this being Fedora, and hence having more recent packages
  - On examining the xattrs on the XFS directory directly using getfattr, it shows
    - system.posix_acl_access="\000..."
    and,
    - trusted.SGI_ACL_FILE="\000..."

Glusterfs prescribes XFS as its backed, and the error here is presumed to have been on the same filesystem. It is possible that the ACL encoding is exceedng the xattr size limit, and hence we are getting an EINVAL error as stated.

Additionally, it looks like this is not an XFS error, just that we exceed the xattr size. See [1]

Checking the XFS fs data here is what I see,
# xfs_db -r /dev/mapper/fedora_dhcp--41--97-mntd
xfs_db> inode 67110757
xfs_db> print
core.magic = 0x494e
core.mode = 040775
core.version = 2
core.format = 1 (local)
...
a.bmx[0] = [startoff,startblock,blockcount,extentflag] 0:[0,4194421,1,0]
xfs_db> ablock 0
xfs_db> p
hdr.info.forw = 0
hdr.info.back = 0
hdr.info.magic = 0xfbee
hdr.count = 4
hdr.usedbytes = 424
hdr.firstused = 3068
...
nvlist[0].valuelen = 16
nvlist[0].namelen = 4
nvlist[0].name = "gfid"
nvlist[0].value = "\3677\302LX3G@\251\320\033\'aQ\242\322"
nvlist[1].valuelen = 304
nvlist[1].namelen = 12
nvlist[1].name = "SGI_ACL_FILE"
nvlist[1].value = "\000\..."
nvlist[2].valuelen = 16
nvlist[2].namelen = 13
nvlist[2].name = "glusterfs.dht"
nvlist[2].value = "\000\000\000\001\000\000\000\000\277\377\377\375\377\377\377\377"
nvlist[3].valuelen = 37
nvlist[3].namelen = 7
nvlist[3].name = "selinux"
nvlist[3].value = "unconfined_u:object_r:unlabeled_t:s0\000"

The above does not seem to be 64K in size, so unsure what limits we are hitting, or if there is a simple setting that can overcome this limitation.

[1] http://oss.sgi.com/archives/xfs/2014-02/msg00024.html

On further discussion with Brian Foster in the XFS team, the issue is that on my Fedora machine, the XFS FS is at v4 superblock version, which supports only 25 ACLs. On counting the actual ACLs the failure is beyond 25 (there are a few default ACLs set already).

We need to following to ensure that the cu is on the same version:
xfs_info <mount> for the mount point being used as bricks.

The output would look as follows,
meta-data=/dev/mapper/fedora_dhcp--41--97-mntd isize=256    agcount=16, agsize=1474560 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=0        finobt=0
data     =                       bsize=4096   blocks=23592960, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=0
log      =internal               bsize=4096   blocks=11520, version=2

In the output, if crc=0 then it is v4, if that reads crc=1 then it is v5, which is limited by 64K value for the xattrs, as the ACL limits as well.

NOTE: XFS v5 ships from RHEL7 onwards.

Marking this as NEEDINFO to validate the above.

http://oss.sgi.com/archives/xfs/2007-12/msg00466.html

RHEL6 git code:

[rfreire@rf rhel6]$ grep XFS_ACL_MAX_ENTRIES fs/xfs/xfs_acl.h
#define XFS_ACL_MAX_ENTRIES 25
	} acl_entry[XFS_ACL_MAX_ENTRIES];

RHEL7:
[rfreire@rf rhel7]$ grep XFS_ACL_MAX_ENTRIES fs/xfs/xfs_acl.h
[rfreire@rf rhel7]$

OK, I'm closing it as NOTABUG.

Use RHEL7 and create the filesystem using -m crc=1 and you will be able to use it. Otherwise, it will use the V4 superblock and behave like RHEL6.

Thanks Shyan for your help and patience!

[root@localhost ~]# dd if=/dev/zero of=XFS bs=1G count=1
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 12.0768 s, 88.9 MB/s
[root@localhost ~]# mkfs.xfs -m crc=1 XFS
meta-data=XFS                    isize=512    agcount=4, agsize=65536 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=0
data     =                       bsize=4096   blocks=262144, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
log      =internal log           bsize=4096   blocks=2560, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
[root@localhost ~]# losetup /dev/loop0 XFS
[root@localhost ~]# mount -o rw,noatime,nouuid,attr2,delaylog,inode64,noquota /dev/loop0 /mnt
[root@localhost ~]# xfs_info /mnt
meta-data=/dev/loop0             isize=512    agcount=4, agsize=65536 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=0
data     =                       bsize=4096   blocks=262144, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
log      =internal               bsize=4096   blocks=2560, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
[root@localhost ~]# mkdir /mnt/bla
[root@localhost ~]# wc -l /etc/passwd
32
[root@localhost ~]# for i in `cat /etc/passwd | awk -F : '{print $1}' `; do echo $i ; setfacl -m u:"$i":rwx /mnt/bla && echo OK || echo FAIL ;  done
root
OK
bin
OK
daemon
OK
adm
OK
lp
OK
sync
OK
shutdown
OK
halt
OK
mail
OK
operator
OK
games
OK
ftp
OK
nobody
OK
avahi-autoipd
OK
systemd-bus-proxy
OK
systemd-network
OK
dbus
OK
polkitd
OK
tss
OK
postfix
OK
sshd
OK
rpc
OK
memcached
OK
apache
OK
ods
OK
rpcuser
OK
nfsnobody
OK
sssd
OK
ntp
OK
hsqldb
OK
tomcat
OK
pkiuser
OK

Rodrigo,

What is the new ACL number limit in version 5?

Thank you.

(In reply to Nicolas Ecarnot from comment #15)
> Rodrigo,
> 
> What is the new ACL number limit in version 5?
> 
> Thank you.

Hi Nicolas,

As per https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit?id=5c87d4bc :

    The limit of 25 ACL entries is arbitrary, but baked into the on-disk
    format.  For version 5 superblocks, increase it to the maximum nuber
    of ACLs that can fit into a single xattr.

I hope that helps.

My best regards,

- RF.