Bug 1127457 - Setting security.* xattrs fails
Summary: Setting security.* xattrs fails
Alias: None
Product: GlusterFS
Classification: Community
Component: fuse
Version: 3.5.2
Hardware: x86_64
OS: Linux
Target Milestone: ---
Assignee: bugs@gluster.org
QA Contact:
Depends On: 1283103
TreeView+ depends on / blocked
Reported: 2014-08-06 22:03 UTC by Diego Woitasen
Modified: 2016-06-17 15:57 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 1283103 (view as bug list)
Last Closed: 2016-06-17 15:57:15 UTC
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:

Attachments (Terms of Use)

Description Diego Woitasen 2014-08-06 22:03:39 UTC
Description of problem:
I have an issue with xattrs with the security prefix. This command work in
all the servers involved (bricks, clients) in all the filesystems mounted
(including the bricks), but fails on the volume mounted in the clients.

# touch testfile && setfattr -n security.NTACL -v foo testfile
setfattr: testfile: Operation not supported

If I try "-n whatever", works.


Ubuntu 14.04

# uname -a
Linux storage01 3.13.0-32-generic #57-Ubuntu SMP Tue Jul 15 03:51:08 UTC
2014 x86_64 x86_64 x86_64 GNU/Linux

# glusterfs --version
glusterfs 3.5.2

All the boxes involved have the same version.

When I try that operation, I see the following in the brick logs:

[2014-08-06 14:39:06.231495] E [marker.c:2542:marker_removexattr_cbk]
0-gv0-marker: No data available occurred while creating symlinks
[2014-08-06 14:39:06.231543] I
[server-rpc-fops.c:727:server_removexattr_cbk] 0-gv0-server: 124:
REMOVEXATTR /testfile (473fd975-a619-47d7-9ebc-d2a077ed1ac8) of key
security.ima ==> (No data available)

Comment 1 Diego Woitasen 2014-08-06 22:09:37 UTC
More information, my fstab line:

storage01:/gv0 /srv/data glusterfs defaults,acl 0 0

Adding user_xattr doesn't work, I can't mount the volume with that option. BTW, I can set xattrs, the problem is with security prefixed xattrs.

Comment 2 Niels de Vos 2014-08-29 09:55:59 UTC
From xlators/mount/fuse/src/fuse-bridge.c:
3136         if (!priv->selinux) {
3137                 if (strncmp (name, "security.", 9) == 0) {
3138                         send_fuse_err (this, finh, EOPNOTSUPP);
3139                         GF_FREE (finh);
3140                         return;
3141                 }
3142         }

Checking a little further, it seems that you can mount with the 'selinux' option. In /etc/fstab this would then look like this:

  storage01:/gv0 /srv/data glusterfs defaults,acl,selinux 0 0

I understand that Ubuntu does not use selinux, but that does not matter for using this mount option. Could you test and report if that works for you?

Proposed fix:
We should not disallow setting security.* xattrs when selinux is not in use, selinux is not the only component that uses security.* xattrs. I think that instead of filtering out all security.* xattrs, only security.selinux should be denied.

Comment 3 Diego Woitasen 2014-08-29 11:39:36 UTC
Adding the selinux mount options it works.

Thanks for your help.

Comment 4 pille 2015-01-23 10:38:48 UTC
i'm currently ingesting several TB into our glusterfs-cluster and my logfile grows multiple GB/day because of (three times per file):

[2015-01-23 10:31:04.835822] E [marker.c:2542:marker_removexattr_cbk] 0-storage_www-marker: No data available occurred while creating symlinks
[2015-01-23 10:31:04.835850] I [server-rpc-fops.c:693:server_removexattr_cbk] 0-storage_www-server: 88351: REMOVEXATTR /file1234 (85c82646-7c31-4773-ae5f-fa7661452f67) of key security.ima ==> (No data available)

mounting using -o selinux doesn't help.

i'm on ubuntu 14.04, too. glusterfs is v3.6.1.

Comment 5 Niels de Vos 2015-01-23 11:26:06 UTC
The missing (No data available) security.ima extended attribute might be a kernel issue. I think this patch (or some follow up) fixes it:


It would be interesting to know if Ubuntu 14.04 has these patches applied.

Alternatively, you can probably disable the IMA function in Ubuntu to prevent these messages.

Could you let us know if that helps?

Comment 6 pille 2015-01-23 14:08:10 UTC
i couldn't figure out, whether these patches are applied to the stock ubuntu kernel, but found nothing in the changelog.

but i installed a 14.10 kernel (3.16), which should have these commits already in. unfortunately the behavior didn't change.

i didn't try to disable IMA yet.

Comment 7 Jeff Byers 2015-03-03 02:25:09 UTC
Using the -o selunux mount option does not seem to be a good work-around, at least not when selinux is not in use, and Gluster replica volumes are in use.

In GlusterFS 3.6.1, copying a directory tree takes more than 600% longer when
the FUSE mount has the selinux option than it does without it.

This seems to be due to all of the:

   fuse-bridge.c:3335:fuse_xattr_cbk ... GETXATTR(security.capability) ... (No
   data available)


Comment 8 prasun.gera 2015-08-06 19:31:45 UTC
My logs are filled with these messages too on RHS 3.0 on RHEL 6.7. Also, replacing a brick and heal is taking exceptionally long. Don't know if they are related. On replacing a brick in distributed replicated volume, it had managed to move only 100G in 24 hours.

Comment 9 Niels de Vos 2016-06-17 15:57:15 UTC
This bug is getting closed because the 3.5 is marked End-Of-Life. There will be no further updates to this version. Please open a new bug against a version that still receives bugfixes if you are still facing this issue in a more current release.

Note You need to log in before you can comment on or make changes to this bug.