Bug 1309075 - [f23] atomic updates-testing rebase selinux error
Summary: [f23] atomic updates-testing rebase selinux error
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: selinux-policy
Version: 23
Hardware: Unspecified
OS: Unspecified
urgent
high
Target Milestone: ---
Assignee: Lukas Vrabec
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1349045 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-02-16 18:51 UTC by Dusty Mabe
Modified: 2016-07-15 10:26 UTC (History)
13 users (show)

Fixed In Version: selinux-policy-3.13.1-158.21.fc23
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-07-14 00:23:58 UTC
Type: Bug


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1349045 0 unspecified CLOSED selinux error rebasing f23 atomic host to f24 2021-02-22 00:41:40 UTC

Internal Links: 1349045

Description Dusty Mabe 2016-02-16 18:51:45 UTC
Description of problem:
I am getting an error when trying to rebase to the updates-testing
ostree. It is a similar fsetxattr error we have seen in the past:


# Go to known reproducer starting point:
-bash-4.3# rpm-ostree deploy 23.54 && reboot

# After reboot:
-bash-4.3# rpm-ostree rebase fedora-atomic:fedora-atomic/f23/x86_64/testing/docker-host

Receiving objects: 92% (3773/4069) 859.0 kB/s 99.6 MB         
error: fsetxattr: Invalid argument


From Giuseppe:
"""
I can reproduce the issue and the problem seems to be that the
/usr/libexec/rpm-ostreed file is not properly labeled.  It must have the
same label as as rpm-ostree:

bash-4.3# ls -Z -1 /usr/libexec/rpm-ostreed  /usr/bin/rpm-ostree 
system_u:object_r:install_exec_t:s0 /usr/bin/rpm-ostree
         system_u:object_r:bin_t:s0 /usr/libexec/rpm-ostreed
"""



Version-Release number of selected component (if applicable):
The version I am coming from:
2016-01-28 18:08:54     23.54     33166bf33b     fedora-atomic     fedora-atomic:fedora-atomic/f23/x86_64/docker-host


The version I am going to:
2016-02-15 04:56:09     23.68     6c739b2306     fedora-atomic     fedora-atomic:fedora-atomic/f23/x86_64/testing/docker-host

How reproducible:


Steps to Reproduce:
1. rpm-ostree deploy 23.54 --reboot
2. rpm-ostree rebase fedora-atomic:fedora-atomic/f23/x86_64/testing/docker-host

Since the HEAD of the updates-testing branch will move over time this reproducer may not work in the future. At the time of writing the version at the head of the branch is:
2016-02-15 04:56:09     23.68     6c739b2306     fedora-atomic     fedora-atomic:fedora-atomic/f23/x86_64/testing/docker-host


Additional info:
You can workaround by disabling selinux or using this workaround from giuseppe:

# systemctl stop rpm-ostreed
# cp /usr/libexec/rpm-ostreed /tmp
# chcon system_u:object_r:install_exec_t:s0 /tmp/rpm-ostreed
# /tmp/rpm-ostreed

Comment 1 Christoph Görn 2016-02-17 16:01:25 UTC
This does also happen on atomic host upgrade:

$ sudo atomic host status
  TIMESTAMP (UTC)         VERSION   ID             OSNAME            REFSPEC                                                
* 2016-01-26 18:16:33     23.53     22f0b303da     fedora-atomic     fedora-atomic:fedora-atomic/f23/x86_64/docker-host     
$ sudo atomic host upgrade
Updating from: fedora-atomic:fedora-atomic/f23/x86_64/docker-host

Receiving objects: 25% (1/4) 2.7 MB/s 8.1 MB
error: fsetxattr: Invalid argument

But I guess its the same root cause?!

Comment 3 Colin Walters 2016-02-18 13:47:11 UTC
So unfortunately, now that the selinux-policy update made it into stable with new types on disk, in order to fix this on the server side, we would have to:

 - Temporarily revert the update
 - Create a new package which has the policy fix for rpm-ostreed
 - Use that new policy only for Atomic Host content, and freeze it - don't pull in any other policy updates
 - Let time pass for clients to upgrade
 - Eventually unfreeze and return to policy updates
 - At this point anyone who wants to upgrade could use `atomic host deploy` to pick a fixed version, or alternatively setenforce 0.

Comment 4 Miroslav Grepl 2016-02-18 14:51:23 UTC
There is no update for this issue. Just committed in dist-git.

Comment 5 Miroslav Grepl 2016-02-18 15:02:17 UTC

(In reply to Colin Walters from comment #3)
> So unfortunately, now that the selinux-policy update made it into stable
> with new types on disk, in order to fix this on the server side, we would
> have to:

And it happens very often if we add new labels to the policy.

> 
>  - Temporarily revert the update
>  - Create a new package which has the policy fix for rpm-ostreed
>  - Use that new policy only for Atomic Host content, and freeze it - don't
> pull in any other policy updates
>  - Let time pass for clients to upgrade
>  - Eventually unfreeze and return to policy updates
>  - At this point anyone who wants to upgrade could use `atomic host deploy`
> to pick a fixed version, or alternatively setenforce 0.

Comment 6 Colin Walters 2016-02-18 15:06:34 UTC
I'll elaborate a bit here on how this bug occurred.

See https://bugzilla.redhat.com/show_bug.cgi?id=1061064 for some background.

Anything without specific policy runs as unconfined_t, where SELinux allows *almost* anything.  The key here being "almost".  One thing that's not allowed is for a process to write labels that are not known by the current policy.  In order to do that, the install_t domain (via install_exec_t) is used, which has "mac_admin".

The "unknown types" scenario can occur when we're downloading an updated tree commit with a new selinux policy that added types.  But we're still running with the old policy.

However, for an upgrade which doesn't add any new types, things work fine.
Which is how this passed our testing.

This bug will hopefully not reoccur, but just as a sanity check, it might help if rpm-ostree checked on startup that it has the mac_admin permission?

Comment 7 Miroslav Grepl 2016-02-19 10:24:20 UTC
My understanding is /usr/libexec/rpm-ostreed is involved here and it runs as unconfined_service_t so we get this error because of new types in the policy and no "mac_admin" for unconfined_service_t.

It is the same what we had for /usr/bin/rpm-ostree running as unconfined_service_t without mac_admin.


Note: Another reason to have seatomic policy coming from Atomic where we could avoid it.

Comment 8 Colin Walters 2016-02-19 15:22:58 UTC
Yeah.  We should have thought of this when creating the daemon, it's just a very subtle issue...things worked with local testing.

Agreed on seatomic policy.

Comment 9 Miroslav Grepl 2016-02-19 15:34:13 UTC
(In reply to Colin Walters from comment #8)
> Yeah.  We should have thought of this when creating the daemon, it's just a
> very subtle issue...things worked with local testing.

Ok the questions is what is our solution here. We will "fail" in both cases - with/without install_t policy for /usr/libexec/rpm-ostreed.

Comment 10 Luke Macken 2016-03-27 02:01:39 UTC
I'm hitting this issue with the latest "stable" Fedora Atomic Host

https://download.fedoraproject.org/pub/alt/atomic/stable/Cloud_Atomic/x86_64/iso/Fedora-Cloud_Atomic-x86_64-23-20160308.iso

Comment 11 Micah Abbott 2016-03-28 18:41:20 UTC
Like Luke in the comment #10, I'm seeing this error again.

Going from 23.90 to 23.91.


# rpm-ostree status
  TIMESTAMP (UTC)         VERSION   ID             OSNAME            REFSPEC                                                
* 2016-03-25 01:15:28     23.90     648b1538c6     fedora-atomic     fedora-atomic:fedora-atomic/f23/x86_64/docker-host     
  2016-03-07 03:45:47     23.79     1a68c7b9c0     fedora-atomic     fedora-atomic:fedora-atomic/f23/x86_64/docker-host     
-bash-4.3# rpm-ostree deploy 23.91
Resolving version '23.91'
error: fsetxattr: Invalid argument


Relevant messages from journal:

Mar 28 18:37:45 micah-fedora-ah-23-vm1.localdomain audit[1174]: AVC avc:  denied  { mac_admin } for  pid=1174 comm="pool" capability=33  scontext=system_u:system_r:unconfined_service_t:s0 tcontext=system_u:system_r:unconfined_service_t:s0 tclass=capability2 permissive=0
Mar 28 18:37:45 micah-fedora-ah-23-vm1.localdomain audit: SELINUX_ERR op=setxattr invalid_context="system_u:object_r:systemd_hwdb_exec_t:s0"
Mar 28 18:37:49 micah-fedora-ah-23-vm1.localdomain audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-hostnamed comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'


The workaround in comment #1 gets around the issue again:

-bash-4.3# systemctl stop rpm-ostreed
-bash-4.3# cp /usr/libexec/rpm-ostreed /tmp
-bash-4.3# chcon system_u:object_r:install_exec_t:s0 /tmp/rpm-ostreed
-bash-4.3# /tmp/rpm-ostreed &
[1] 1208
-bash-4.3# rpm-ostree deploy 23.91
Resolving version '23.91'

15 metadata, 32 content objects fetched; 95955 KiB transferred in 4 seconds
Copying /etc changes: 24 modified, 0 removed, 49 added
Transaction complete; bootconfig swap: yes deployment count change: 0
Freed objects: 200.0 MB
Changed:
  selinux-policy 3.13.1-158.9.fc23 -> 3.13.1-158.11.fc23
  selinux-policy-targeted 3.13.1-158.9.fc23 -> 3.13.1-158.11.fc23
Run "systemctl reboot" to start a reboot

Comment 13 Colin Walters 2016-03-29 18:54:54 UTC
One thing we can do on the server side is (temporarily) revert selinux-policy back to what's in the last image, and add the patch to fix the label.

Comment 14 Miroslav Grepl 2016-04-01 07:40:58 UTC
Ok, see your #c3 as the reaction on the change. If you are ok with that, no problem to add it back. This change has never been shipped.

Comment 15 Colin Walters 2016-04-01 12:27:51 UTC
(In reply to Miroslav Grepl from comment #14)
> Ok, see your #c3 as the reaction on the change. If you are ok with that, no
> problem to add it back. This change has never been shipped.

Yes, we need the label change.

Comment 16 Fedora Update System 2016-04-11 11:03:36 UTC
selinux-policy-3.13.1-158.14.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2016-3fade554b0

Comment 17 Fedora Update System 2016-04-13 09:28:32 UTC
selinux-policy-3.13.1-158.14.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-3fade554b0

Comment 18 Fedora Update System 2016-04-19 01:21:39 UTC
selinux-policy-3.13.1-158.14.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report.

Comment 19 Colin Walters 2016-06-22 15:09:17 UTC
This still doesn't work.

# rpm-ostree status
  TIMESTAMP (UTC)         VERSION     ID             OSNAME            REFSPEC                                           
* 2016-06-20 20:15:43     23.141      e3e06c5186     fedora-atomic     fedora-atomic:fedora-atomic/f23/x86_64/docker-host
  2016-06-07 01:00:21     23.131      aadcfe51ac     fedora-atomic     fedora-atomic:fedora-atomic/f23/x86_64/docker-host
# rpm -q selinux-policy
selinux-policy-3.13.1-158.15.fc23.noarch
# ps axZ|grep rpm-os
system_u:system_r:init_t:s0      1137 ?        Ssl    0:00 /usr/libexec/rpm-ostreed

Comment 20 Eric Paris 2016-06-22 15:13:47 UTC
somehow rpm-ostreed got back to init_t...

Comment 21 Daniel Walsh 2016-06-22 15:19:01 UTC
How is rpm-ostreed labeled

Comment 22 Micah Abbott 2016-06-22 15:25:53 UTC
# rpm-ostree status
  TIMESTAMP (UTC)         VERSION     ID             OSNAME            REFSPEC                                           
* 2016-06-20 20:15:43     23.141      e3e06c5186     fedora-atomic     fedora-atomic:fedora-atomic/f23/x86_64/docker-host
  2016-05-24 01:15:41     23.125      5668f0613e     fedora-atomic     fedora-atomic:fedora-atomic/f23/x86_64/docker-host

# rpm -q rpm-ostree
rpm-ostree-2016.1-3.fc23.x86_64

# ls -Z /usr/libexec/rpm-ostreed
system_u:object_r:install_exec_t:s0 /usr/libexec/rpm-ostreed

Comment 23 Micah Abbott 2016-06-22 15:30:49 UTC
*** Bug 1349045 has been marked as a duplicate of this bug. ***

Comment 24 Daniel Walsh 2016-06-22 15:59:33 UTC
That is the correct label.  Do we have a transition rule from init_t -> install_t?

sesearch -T -s init_t -t install_exec_t

Comment 25 Micah Abbott 2016-06-22 16:10:54 UTC
Doesn't look that way.

-bash-4.3# sesearch -T -s init_t -t install_exec_t


-bash-4.3#

Comment 26 Daniel Walsh 2016-06-22 16:18:28 UTC
In Rawhide Policy I have

sesearch -T -t install_exec_t
Found 2 semantic te rules:
   type_transition unconfined_t install_exec_t : process install_t; 
   type_transition init_t install_exec_t : process install_t; 

We need this back ported to F23/F24 I guess.

Comment 27 Lukas Vrabec 2016-06-27 14:43:52 UTC
backported.

Comment 28 Fedora Update System 2016-07-02 20:54:25 UTC
selinux-policy-3.13.1-158.21.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-7bed6e7c72

Comment 29 Fedora Update System 2016-07-14 00:23:13 UTC
selinux-policy-3.13.1-158.21.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report.

Comment 30 Colin Walters 2016-07-15 10:26:37 UTC
To work around this, just:

setenforce 0

Then perform rpm-ostree upgrade/rebase etc.   Then reboot, and if you have this new policy, you should still be in enforcing after the reboot and won't need to setenforce 0 again.


Note You need to log in before you can comment on or make changes to this bug.