Bug 1283103 - Setting security.* xattrs fails
Setting security.* xattrs fails
Status: CLOSED CURRENTRELEASE
Product: GlusterFS
Classification: Community
Component: fuse (Show other bugs)
mainline
x86_64 Linux
unspecified Severity high
: ---
: ---
Assigned To: Michael Adam
: EasyFix, Triaged
Depends On:
Blocks: 1292778 1292779 1127457 1283107
  Show dependency treegraph
 
Reported: 2015-11-18 04:38 EST by Niels de Vos
Modified: 2016-06-16 09:45 EDT (History)
2 users (show)

See Also:
Fixed In Version: glusterfs-3.8rc2
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1127457
: 1283107 (view as bug list)
Environment:
Last Closed: 2016-06-16 09:45:21 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Niels de Vos 2015-11-18 04:38:25 EST
+++ This bug was initially created as a clone of Bug #1127457 +++
+++                                                           +++
+++ Use this bug to provide a patch for the master branch.    +++

Description of problem:
I have an issue with xattrs with the security prefix. This command work in
all the servers involved (bricks, clients) in all the filesystems mounted
(including the bricks), but fails on the volume mounted in the clients.

# touch testfile && setfattr -n security.NTACL -v foo testfile
setfattr: testfile: Operation not supported

If I try "-n whatever", works.

Versions:

Ubuntu 14.04

# uname -a
Linux storage01 3.13.0-32-generic #57-Ubuntu SMP Tue Jul 15 03:51:08 UTC
2014 x86_64 x86_64 x86_64 GNU/Linux

# glusterfs --version
glusterfs 3.5.2

All the boxes involved have the same version.

When I try that operation, I see the following in the brick logs:

[2014-08-06 14:39:06.231495] E [marker.c:2542:marker_removexattr_cbk]
0-gv0-marker: No data available occurred while creating symlinks
[2014-08-06 14:39:06.231543] I
[server-rpc-fops.c:727:server_removexattr_cbk] 0-gv0-server: 124:
REMOVEXATTR /testfile (473fd975-a619-47d7-9ebc-d2a077ed1ac8) of key
security.ima ==> (No data available)

--- Additional comment from Diego Woitasen on 2014-08-07 00:09:37 CEST ---

More information, my fstab line:

storage01:/gv0 /srv/data glusterfs defaults,acl 0 0

Adding user_xattr doesn't work, I can't mount the volume with that option. BTW, I can set xattrs, the problem is with security prefixed xattrs.

--- Additional comment from Niels de Vos on 2014-08-29 11:55:59 CEST ---

From xlators/mount/fuse/src/fuse-bridge.c:
3136         if (!priv->selinux) {
3137                 if (strncmp (name, "security.", 9) == 0) {
3138                         send_fuse_err (this, finh, EOPNOTSUPP);
3139                         GF_FREE (finh);
3140                         return;
3141                 }
3142         }

Checking a little further, it seems that you can mount with the 'selinux' option. In /etc/fstab this would then look like this:

  storage01:/gv0 /srv/data glusterfs defaults,acl,selinux 0 0

I understand that Ubuntu does not use selinux, but that does not matter for using this mount option. Could you test and report if that works for you?


Proposed fix:
We should not disallow setting security.* xattrs when selinux is not in use, selinux is not the only component that uses security.* xattrs. I think that instead of filtering out all security.* xattrs, only security.selinux should be denied.

--- Additional comment from Diego Woitasen on 2014-08-29 13:39:36 CEST ---

Adding the selinux mount options it works.

Thanks for your help.

--- Additional comment from pille on 2015-01-23 11:38:48 CET ---

i'm currently ingesting several TB into our glusterfs-cluster and my logfile grows multiple GB/day because of (three times per file):

[2015-01-23 10:31:04.835822] E [marker.c:2542:marker_removexattr_cbk] 0-storage_www-marker: No data available occurred while creating symlinks
[2015-01-23 10:31:04.835850] I [server-rpc-fops.c:693:server_removexattr_cbk] 0-storage_www-server: 88351: REMOVEXATTR /file1234 (85c82646-7c31-4773-ae5f-fa7661452f67) of key security.ima ==> (No data available)

mounting using -o selinux doesn't help.

i'm on ubuntu 14.04, too. glusterfs is v3.6.1.

--- Additional comment from Niels de Vos on 2015-01-23 12:26:06 CET ---

The missing (No data available) security.ima extended attribute might be a kernel issue. I think this patch (or some follow up) fixes it:

    https://lkml.org/lkml/2014/7/11/259

It would be interesting to know if Ubuntu 14.04 has these patches applied.

Alternatively, you can probably disable the IMA function in Ubuntu to prevent these messages.

Could you let us know if that helps?

--- Additional comment from pille on 2015-01-23 15:08:10 CET ---

i couldn't figure out, whether these patches are applied to the stock ubuntu kernel, but found nothing in the changelog.

but i installed a 14.10 kernel (3.16), which should have these commits already in. unfortunately the behavior didn't change.

i didn't try to disable IMA yet.

--- Additional comment from Jeff Byers on 2015-03-03 03:25:09 CET ---

Using the -o selunux mount option does not seem to be a good work-around, at least not when selinux is not in use, and Gluster replica volumes are in use.

In GlusterFS 3.6.1, copying a directory tree takes more than 600% longer when
the FUSE mount has the selinux option than it does without it.

This seems to be due to all of the:

   fuse-bridge.c:3335:fuse_xattr_cbk ... GETXATTR(security.capability) ... (No
   data available)

failures.

--- Additional comment from  on 2015-08-06 21:31:45 CEST ---

My logs are filled with these messages too on RHS 3.0 on RHEL 6.7. Also, replacing a brick and heal is taking exceptionally long. Don't know if they are related. On replacing a brick in distributed replicated volume, it had managed to move only 100G in 24 hours.
Comment 1 Vijay Bellur 2015-11-30 07:28:02 EST
REVIEW: http://review.gluster.org/12826 (fuse: forbid only access to security.selinux xattr if not mounted with 'selinux') posted (#1) for review on master by Michael Adam (obnox@samba.org)
Comment 2 Vijay Bellur 2015-11-30 07:28:06 EST
REVIEW: http://review.gluster.org/12827 (tests:bugs:fuse: add test for bug #1283103 - selinux mount vs security xattrs) posted (#1) for review on master by Michael Adam (obnox@samba.org)
Comment 3 Vijay Bellur 2015-12-07 23:01:34 EST
REVIEW: http://review.gluster.org/12826 (fuse: forbid only access to security.selinux xattr if not mounted with 'selinux') posted (#2) for review on master by Vijay Bellur (vbellur@redhat.com)
Comment 4 Vijay Bellur 2015-12-08 10:05:50 EST
REVIEW: http://review.gluster.org/12826 (fuse: forbid only access to security.selinux xattr if not mounted with 'selinux') posted (#3) for review on master by Vijay Bellur (vbellur@redhat.com)
Comment 5 Vijay Bellur 2015-12-08 16:51:57 EST
REVIEW: http://review.gluster.org/12826 (fuse: forbid only access to security.selinux xattr if not mounted with 'selinux') posted (#4) for review on master by Vijay Bellur (vbellur@redhat.com)
Comment 6 Vijay Bellur 2015-12-09 03:19:48 EST
REVIEW: http://review.gluster.org/12826 (fuse: forbid only access to security.selinux xattr if not mounted with 'selinux') posted (#5) for review on master by Michael Adam (obnox@samba.org)
Comment 7 Vijay Bellur 2015-12-09 06:37:41 EST
REVIEW: http://review.gluster.org/12826 (fuse: forbid only access to security.selinux xattr if not mounted with 'selinux') posted (#6) for review on master by Michael Adam (obnox@samba.org)
Comment 8 Vijay Bellur 2015-12-09 07:21:43 EST
REVIEW: http://review.gluster.org/12826 (fuse: forbid only access to security.selinux xattr if not mounted with 'selinux') posted (#7) for review on master by Michael Adam (obnox@samba.org)
Comment 9 Vijay Bellur 2015-12-09 08:40:50 EST
REVIEW: http://review.gluster.org/12826 (fuse: forbid only access to security.selinux xattr if not mounted with 'selinux') posted (#8) for review on master by Michael Adam (obnox@samba.org)
Comment 10 Vijay Bellur 2015-12-10 01:12:44 EST
REVIEW: http://review.gluster.org/12826 (fuse: forbid only access to security.selinux xattr if not mounted with 'selinux') posted (#9) for review on master by Michael Adam (obnox@samba.org)
Comment 11 Vijay Bellur 2015-12-10 04:30:43 EST
REVIEW: http://review.gluster.org/12826 (fuse: forbid only access to security.selinux xattr if not mounted with 'selinux') posted (#10) for review on master by Michael Adam (obnox@samba.org)
Comment 12 Vijay Bellur 2015-12-11 00:34:56 EST
COMMIT: http://review.gluster.org/12826 committed in master by Raghavendra G (rgowdapp@redhat.com) 
------
commit 4fefa3d7dbcdad1e71c74db11113ac1e74b01656
Author: Michael Adam <obnox@samba.org>
Date:   Wed Nov 18 09:01:30 2015 +0100

    fuse: forbid only access to security.selinux xattr if not mounted with 'selinux'
    
    Originally, all selinux.* xattrs were forbidden, causing
    for example Samba's acl_xattr module which uses security.NTACL
    to fail without the 'selinux' mount option, which is confusing
    at least. This change specializes the check to the security.selinux
    attribute, so other selinux.* attributes work with or without the option.
    
    Change-Id: I9d3083123efbf403f20572cfb325a300ce2e90d9
    BUG: 1283103
    Signed-off-by: Michael Adam <obnox@samba.org>
    Reviewed-on: http://review.gluster.org/12826
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Tested-by: NetBSD Build System <jenkins@build.gluster.org>
    Reviewed-by: Rajesh Joseph <rjoseph@redhat.com>
    Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Comment 13 Vijay Bellur 2015-12-11 00:35:05 EST
REVIEW: http://review.gluster.org/12827 (tests:bugs:fuse: add test for bug #1283103 - selinux mount vs security xattrs) posted (#2) for review on master by Raghavendra G (rgowdapp@redhat.com)
Comment 14 Vijay Bellur 2015-12-11 04:48:09 EST
COMMIT: http://review.gluster.org/12827 committed in master by Raghavendra G (rgowdapp@redhat.com) 
------
commit 8df88683c334c0652770e8e0fcec59dbc8bb5748
Author: Michael Adam <obnox@samba.org>
Date:   Mon Nov 30 10:51:13 2015 +0100

    tests:bugs:fuse: add test for bug #1283103 - selinux mount vs security xattrs
    
    BUG: 1283103
    Change-Id: Ic4485d650275f67eb6b0b8382a92eb829c06e27c
    Signed-off-by: Michael Adam <obnox@samba.org>
    Reviewed-on: http://review.gluster.org/12827
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Reviewed-by: Niels de Vos <ndevos@redhat.com>
    Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Comment 15 Niels de Vos 2016-06-16 09:45:21 EDT
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.0, please open a new bug report.

glusterfs-3.8.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://blog.gluster.org/2016/06/glusterfs-3-8-released/
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user

Note You need to log in before you can comment on or make changes to this bug.