1356810 – Fliesystem extended attributes in trusted namespace are not persisted after node reboot

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1356810 - Fliesystem extended attributes in trusted namespace are not persisted after node reboot

Summary: Fliesystem extended attributes in trusted namespace are not persisted after ...

Keywords:
Status:	CLOSED NOTABUG
Alias:	None
Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	docker
Sub Component:
Version:	7.2
Hardware:	x86_64
OS:	Linux
Priority:	unspecified
Severity:	urgent
Target Milestone:	rc
Target Release:	---
Assignee:	Daniel Walsh
QA Contact:	atomic-bugs@redhat.com
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2016-07-15 06:03 UTC by Mohamed Ashiq
Modified:	2019-03-06 02:02 UTC (History)
CC List:	5 users (show)
Fixed In Version:
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:	2016-10-18 14:24:41 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Description Mohamed Ashiq 2016-07-15 06:03:50 UTC

Description of problem:

We have a container running on RHEL 7.2 latest update where we set extended attribute on a file from the container and after node reboot these extended atributes are not seen on subject file.

Version-Release number of selected component (if applicable):

libattr-2.4.46-12.el7.x86_64
pyxattr-0.5.1-5.el7.x86_64
attr-2.4.46-12.el7.x86_64


[root@dhcp42-69 ~]# uname -a
Linux dhcp42-69.lab.eng.blr.redhat.com 3.10.0-327.18.2.el7.x86_64 #1 SMP Fri Apr 8 05:09:53 EDT 2016 x86_64 x86_64 x86_64 GNU/Linux

[root@dhcp42-69 ~]# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.2 (Maipo)

How reproducible:

#docker run -v /run -v /etc/glusterfs:/etc/glusterfs:z -v /var/lib/glusterd:/var/lib/glusterd:z -v /var/log/glusterfs:/var/log/glusterfs:z -v /sys/fs/cgroup:/sys/fs/cgroup:ro -d --privileged=true --net=host -v /dev:/dev brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhgs3/rhgs-server-rhel7

# Create a PV?VG/ and LV

# Format it with the XFS FS

# mount LV inside the container

# Make this mount happening in the container whenever container is spawned. We have a custom fstab entry 

# set xattr on trusted namespace. 

# listxattr from the container
 
 -  make sure the xattr can be retrieved.

# Reboot the node

# Spawn the container again

# Check the xattr again
  - At this stage we are not able to see the xattr in subjected file.


Steps to Reproduce:

Same as above

Actual results:

The extended attributes in trusted namespace are vanished after node reboot.
 
Expected results:

The xattr has to be available even after the reboot of the node.

Additional info:

BUgzilla ID # https://bugzilla.redhat.com/show_bug.cgi?id=1355801

Comment 2 Kamil Dudka 2016-07-15 09:15:28 UTC

This is either problem with the file system implementation or the containerized environment.  There is no flag that the libattr users-pace library could use to tell kernel whether the attributes should persist a reboot or not.

Not a bug in the attr package for sure.

Comment 3 Humble Chirammal 2016-07-16 09:35:26 UTC

(In reply to Kamil Dudka from comment #2)
> This is either problem with the file system implementation or the
> containerized environment.  There is no flag that the libattr users-pace
> library could use to tell kernel whether the attributes should persist a
> reboot or not.
> 
> Not a bug in the attr package for sure.

Yes, We do suspect it may be something wrong with the underlying FS. In this context it is XFS. We also did some isolation here and we noticed that the attributes we set from the container was not visible at times when we inspect from the HOST system. Thats one of the reason for opening this bug. But, you were fast to respond before updating this second scenario :). Can you please confirm libattr dont have a role in this scenario as well ? We are doing more isolation wrt container environment and also with underlying FS/XFS. We would like to eliminate the possibilities of libattr in both the scenarios.

Comment 4 Kamil Dudka 2016-07-16 11:05:30 UTC

Sure I meant the attr package with all its subpackages, including libattr.  It just implements library interface on top of the corresponding syscalls.  There is nothing in libattr that could influence whether the extended attributes are propagated to the host file system or not.

Comment 5 Kamil Dudka 2016-07-18 11:24:10 UTC

I am switching the component to docker as it takes care of the bind mounts.

Comment 6 Daniel Walsh 2016-07-19 10:25:00 UTC

The SELinux xattrs are set at container creation time.  It is up to the file system to maintain the Xattrs.

As an experiment outside of docker, could you check the contexts on these directories before reboot?


After creating the container

ls -ldZ /etc/glusterfs /var/lib/glusterd /var/log/glusterfs

After reboot, run the same command.  If these labels are changing, then something in the boot is changing the labels. or the file system is broken.

Nothing in docker should change them after the container creation?

Comment 7 Daniel Walsh 2016-08-19 22:39:46 UTC

Kamil any update on this?

Comment 8 Kamil Dudka 2016-08-20 11:09:46 UTC

(In reply to Daniel Walsh from comment #7)
> Kamil any update on this?

I am just the maintainer of libattr.  This bug was assigned to me by mistake.  Setting needinfo on Mohamed, who reported it originally...

Comment 9 Mohamed Ashiq 2016-08-23 11:45:55 UTC

(In reply to Daniel Walsh from comment #6)
> The SELinux xattrs are set at container creation time.  It is up to the file
> system to maintain the Xattrs.
> 
> As an experiment outside of docker, could you check the contexts on these
> directories before reboot?
> 
> 
> After creating the container
> 
> ls -ldZ /etc/glusterfs /var/lib/glusterd /var/log/glusterfs
> 
> After reboot, run the same command.  If these labels are changing, then
> something in the boot is changing the labels. or the file system is broken.
> 
> Nothing in docker should change them after the container creation?

Dan, sorry for really late reply. I am able to see the labels on the host and container in these(/etc/glusterfs /var/lib/glusterd /var/log/glusterfs). The problem is on the LV's which was created inside the container and mounted in the container using a custom fstab. After reboot, the container is spawned instantly and few mountpoints are sometimes read-only and sometimes not mounted.

Comment 10 Mohamed Ashiq 2016-08-23 11:51:18 UTC

(In reply to Mohamed Ashiq from comment #9)
> (In reply to Daniel Walsh from comment #6)
> > The SELinux xattrs are set at container creation time.  It is up to the file
> > system to maintain the Xattrs.
> > 
> > As an experiment outside of docker, could you check the contexts on these
> > directories before reboot?
> > 
> > 
> > After creating the container
> > 
> > ls -ldZ /etc/glusterfs /var/lib/glusterd /var/log/glusterfs
> > 
> > After reboot, run the same command.  If these labels are changing, then
> > something in the boot is changing the labels. or the file system is broken.
> > 
> > Nothing in docker should change them after the container creation?
> 
> Dan, sorry for really late reply. I am able to see the labels on the host
> and container in these(/etc/glusterfs /var/lib/glusterd /var/log/glusterfs).
> The problem is on the LV's which was created inside the container and
> mounted in the container using a custom fstab. After reboot, the container
> is spawned instantly and few mountpoints are sometimes read-only and
> sometimes not mounted.

So these mountpoints were mounted using custom fstab. We were checking the mount points which was not mounted. As glusterfs was still setting few xattrs for checking(which was something unexpected), We thought the rest of the xattrs are lost.

Comment 11 Daniel Walsh 2016-08-24 12:00:39 UTC

Does this mean there is no bug?  Or at least on bug in docker?

Note You need to log in before you can comment on or make changes to this bug.