Bug 764245 (GLUSTER-2513)

(In reply to comment #0)
> hi,
> 
> Glusterfs posix backend stores data in extended file attributes, specifically
> Trusted extended attributes.
> Those require CAP_SYS_ADMIN capability to access/change.
> 
> This requirement makes it impractical to run glusterfs in light-weight
> virtualization environments (lxc/openvz/linux vserver), since delegating
> CAP_SYS_ADMIN to guests is usually undesirable.
> 
Problem is the other way round, CAP_SYS_ADMIN is a standard mechanism where a given application is allowed to behave in Linux in 'extended' attribute space. 

Now that lxc/openvz/linux vserver fail with this is their problem rather than GlusterFS itself. 

For better virtualization environments recommended is KVM. 

lxc/openvz/linux vserver are not tested in-house, in-turn calls for unwarranted issues.

(In reply to comment #1)
> (In reply to comment #0)
> > Glusterfs posix backend stores data in extended file attributes, specifically
> > Trusted extended attributes.
> > Those require CAP_SYS_ADMIN capability to access/change.
> > 
> > This requirement makes it impractical to run glusterfs in light-weight
> > virtualization environments (lxc/openvz/linux vserver), since delegating
> > CAP_SYS_ADMIN to guests is usually undesirable.
> > 
> Problem is the other way round, CAP_SYS_ADMIN is a standard mechanism where a
> given application is allowed to behave in Linux in 'extended' attribute space. 
> Now that lxc/openvz/linux vserver fail with this is their problem rather than
> GlusterFS itself. 

I believe providing an option to use "user" extended attributes instead of "trusted" extended attributes could allow for better security trade-offs in some configurations using  light-weight virtualization environments. It's possible to allow CAP_SYS_ADMIN for each container, but in many configurations using "user" attributes instead would be a better choice.
  
> For better virtualization environments recommended is KVM.
> lxc/openvz/linux vserver are not tested in-house, in-turn calls for unwarranted issues.

Well, KVM and container-based virtualization are just two different beasts, you cannot say one is better than the other - they just have different purposes/profiles and user bases.

--igor

Using containers to host bricks requires writing the "trusted.*" xattrs to the underlaying filesystem. The brick processes need to have the CAP_SYS_ADMIN capability (see: man 7 capabilities). Containers are expected to have little privileges, and grating CAP_SYS_ADMIN to a container is frowned upon.

Proposed solution:
- instead of using "trusted.*" xattrs on the bricks, use "user.*" xattrs

Implementation details/notes/ideas:
- provide a volume or mount option to specify that "user.*" should be used
- it is impractical (and would hurt compatibility) when all the xattrs would be
  replaced by "user.*" throughout the whole sources. It would be simpler and more
  compatible to have the posix-xlator handle the volume/mount option. A fallback
  on checking for the other xattr prefix would be more efficient there too.

because of the large number of bugs filed against mainline version\ is ambiguous and about to be removed as a choice.

If you believe this is still a bug, please change the status back to NEW and choose the appropriate, applicable version for it.

I have written a tiny library which you can load with LD_PRELOAD that translates attribute names - trusted. into user.tr. - and will test how well GlusterFS behaves when glusterd is run for all bricks in an unprivileged / no CAP_SYS_ADMIN docker container, with xattr names under user.

Initial testing of it hasn't found anything broken by it.

Here is my library in case other people want to try it: https://gitlab.com/A1kmm/fake-trusted-xattr

You could consider either incorporating this approach into the Docker container, or potentially making it an option in GlusterFS itself.