Bug 1290659 - selinux policy move to /var conflicts with ostree
selinux policy move to /var conflicts with ostree
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: selinux-policy (Show other bugs)
24
Unspecified Linux
high Severity urgent
: ---
: ---
Assigned To: Lukas Vrabec
Fedora Extras Quality Assurance
: Reopened
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-12-11 00:15 EST by Peter
Modified: 2017-11-15 21:23 EST (History)
22 users (show)

See Also:
Fixed In Version: selinux-policy-3.13.1-191.16.fc24 selinux-policy-3.13.1-218.fc25
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-10-08 22:42:56 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Do a factory reset when there's no policy.kern file in a store (5.50 KB, text/plain)
2016-09-13 05:45 EDT, Petr Lautrbach
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
GNOME Bugzilla 761620 None None None 2016-10-07 15:17 EDT

  None (edit)
Description Peter 2015-12-11 00:15:41 EST
Description of problem:

On the latest fedora atomic, running a semanage command errors:

Version-Release number of selected component (if applicable):

OS: Fedora Atomic, version 23.30, ID d147058d67

libsemanage-2.4-4.fc23.x86_64
libsemanage-python-2.4-4.fc23.x86_64
libsemanage-python3-2.4-4.fc23.x86_64

How reproducible:

Everytime

Steps to Reproduce:
1. semanage port -a -t ssh_port_t -p tcp 2222

Actual results:

libsemanage.semanage_direct_get_module_info: Unable to open docker module lang ext file at /var/lib/selinux/targeted/tmp/modules/400/docker/lang_ext. (No such file or directory).

Expected results:

Selinux allows ssh connections on port 2222.

Additional info:
Comment 1 Miroslav Grepl 2015-12-11 03:22:26 EST
Could you try to reinstall docker-selinux package?
Comment 2 Peter 2015-12-11 09:42:32 EST
On atomic you can't really just install packages, you update the system and it pulls the prebuilt tree. 
The currently installed package is docker-selinux-1.9.1-4.git6ec29ef.fc23.x86_64, which does appear to be the latest for f23. Since with the same package on standard fedora 23 this is working, I assume that this issues is caused by something getting removed during the tree compose process.
Comment 3 Daniel Walsh 2015-12-11 10:08:48 EST
I am not sure you are allowed to run semanage on atomic host.
Comment 4 Peter 2015-12-11 10:19:13 EST
Is that by design? Or just something that hasn't been fully implemented yet? I think that users probably should be able to change the ssh port on a atomic system
Comment 5 Lukas Vrabec 2015-12-11 10:20:48 EST
(In reply to Peter from comment #2)
> On atomic you can't really just install packages, you update the system and
> it pulls the prebuilt tree. 
> The currently installed package is
> docker-selinux-1.9.1-4.git6ec29ef.fc23.x86_64, which does appear to be the
> latest for f23.
Agree with Peter. I already tried it. F23 workstation looks fine with actual selinux-policy and docker-policy packages. 

In Atomic, problem is that selinux modules in /var/lib/selinux/targeted/active/modules/100 / /var/lib/selinux/targeted/active/modules/400/ are empty. 


bash-4.3# ls /var/lib/selinux/targeted/active/modules/100/openvpn
bash-4.3# ls /var/lib/selinux/targeted/active/modules/100/abrt
bash-4.3# ls /var/lib/selinux/targeted/active/modules/400/docker


> Since with the same package on standard fedora 23 this is
> working, 
> I assume that this issues is caused by something getting removed
> during the tree compose process.
Atomic guys, any hint here?
Comment 6 Colin Walters 2015-12-11 10:23:42 EST
This is at least partly due to the fact that the move of SELinux policy to /var simply conflicts with the OSTree design of having /var empty.

I mentioned this in https://lists.fedoraproject.org/pipermail/devel/2015-June/211308.html 

The phase
"However, we'll probably need to provide  systemd-tmpfiles units in each
selinux-policy-* subpackage to create necessary directory structure."
was probably not implemented.  I honestly think it'd be a lot saner to keep the sources in /usr (again), and have /etc be overrides.
Comment 7 Daniel Walsh 2015-12-11 10:28:45 EST
Should the defaults be stored in /usr/lib/selinux and then semanage would look in /var/lib/selinux first and fall back to /usr/lib/selinux?
Comment 8 Colin Walters 2015-12-11 10:29:17 EST
(In reply to Lukas Vrabec from comment #5)

> > Since with the same package on standard fedora 23 this is
> > working, 
> > I assume that this issues is caused by something getting removed
> > during the tree compose process.
> Atomic guys, any hint here?

Also there really shouldn't be a separate "Atomic guys" - the model is part of Fedora the same way that other parts are - and Atomic Host is also part of the Fedora release and is just as "standard".

There are just two update models now, and we need to work to align them.
Comment 9 Petr Lautrbach 2015-12-11 10:36:25 EST
(In reply to Daniel Walsh from comment #7)
> Should the defaults be stored in /usr/lib/selinux and then semanage would
> look in /var/lib/selinux first and fall back to /usr/lib/selinux?

It won't help if you want your changes to be persistent after reboot.
Comment 10 Colin Walters 2015-12-11 10:36:32 EST
(In reply to Daniel Walsh from comment #7)
> Should the defaults be stored in /usr/lib/selinux and then semanage would
> look in /var/lib/selinux first and fall back to /usr/lib/selinux?

I don't understand why they're in /var now - the systemd model of /usr is immutable defaults and /etc is overrides is quite good IMO.

If we can do that and just add some backwards compatibility to "look in /var first, then /usr", that seems like it'd work.
Comment 11 Colin Walters 2015-12-11 10:39:30 EST
(In reply to Petr Lautrbach from comment #9)
> (In reply to Daniel Walsh from comment #7)
> > Should the defaults be stored in /usr/lib/selinux and then semanage would
> > look in /var/lib/selinux first and fall back to /usr/lib/selinux?
> 
> It won't help if you want your changes to be persistent after reboot.

Okay, a bit of confusion here - Atomic Host/OSTree wants to support a "factory reset" by deleting /var, but by default /var *is* persistent.

The most important thing to understand is rpm-ostree converts /var from RPM content into systemd-tmpfiles units only for directories as a nod to some backwards compatiblity.  Many packages include stub directories, but a lot fewer had/have static files in /var.
Comment 12 Colin Walters 2015-12-11 11:01:21 EST
So, would it be possible to teach the userspace tools about the "systemd model" of /usr = ro defaults, /etc = persistent overrides?  (And maybe one step farther of /run = transient override)?
Comment 13 Lukas Vrabec 2015-12-11 11:04:29 EST
(In reply to Colin Walters from comment #11)
> (In reply to Petr Lautrbach from comment #9)
> > (In reply to Daniel Walsh from comment #7)
> > > Should the defaults be stored in /usr/lib/selinux and then semanage would
> > > look in /var/lib/selinux first and fall back to /usr/lib/selinux?
> > 
> > It won't help if you want your changes to be persistent after reboot.
> 
> Okay, a bit of confusion here - Atomic Host/OSTree wants to support a
> "factory reset" by deleting /var, but by default /var *is* persistent.
> 
I'm able reproduce the issue with fresh created atomic host.

> The most important thing to understand is rpm-ostree converts /var from RPM
> content into systemd-tmpfiles units only for directories as a nod to some
> backwards compatiblity.  Many packages include stub directories, but a lot
> fewer had/have static files in /var.
Comment 14 Petr Lautrbach 2015-12-11 11:43:06 EST

(In reply to Colin Walters from comment #12)
> So, would it be possible to teach the userspace tools about the "systemd
> model" of /usr = ro defaults, /etc = persistent overrides?  (And maybe one
> step farther of /run = transient override)?


To change libsemanage to create the /var/lib/selinux store from the system default /usr/lib/selinux when the /var/lib/selinux is empty should be feasible. I'll take a look at that.

Or there's a possibility to set store_path=/etc/selinux in /etc/selinux/semanage.conf but it would need to have a special version of selinux-policy-targetd  with the modules stored there.
Comment 15 Petr Lautrbach 2015-12-11 12:20:01 EST
(In reply to Colin Walters from comment #6)
> This is at least partly due to the fact that the move of SELinux policy to
> /var simply conflicts with the OSTree design of having /var empty.
> 
> I mentioned this in
> https://lists.fedoraproject.org/pipermail/devel/2015-June/211308.html 
> 
> The phase
> "However, we'll probably need to provide  systemd-tmpfiles units in each
> selinux-policy-* subpackage to create necessary directory structure."

That's actually something that could work.

selinux-policy-targeted would ship /usr/lib/selinux/targeted and /usr/lib/tmpfiles.d/selinux.conf with:

C /var/lib/selinux/targeted -    -    -    -  /usr/lib/selinux/targeted

So that after a factory reset when /var/lib/selinux/targeted doesn't exist, all files from /usr/lib/selinux/targeted would be copied to /var/lib/selinux/targeted.
Comment 16 Colin Walters 2015-12-11 12:30:53 EST
(In reply to Petr Lautrbach from comment #15)

> selinux-policy-targeted would ship /usr/lib/selinux/targeted and
> /usr/lib/tmpfiles.d/selinux.conf with:
> 
> C /var/lib/selinux/targeted -    -    -    -  /usr/lib/selinux/targeted
> 
> So that after a factory reset when /var/lib/selinux/targeted doesn't exist,
> all files from /usr/lib/selinux/targeted would be copied to
> /var/lib/selinux/targeted.

I think the concern then is - do you change the RPM to start shipping the files in /usr?  I wonder if RPM would delete the old ones on upgrades?

That aside, this would be an easy change to make now.  

The downsides though: It means we double the disk usage though even if you haven't made any policy customizations.  Which is again why I think the systemd model is best - you only use disk space proportionate to your modifications, it's easy to see what changes you have, etc.
Comment 17 Miroslav Grepl 2015-12-11 12:37:34 EST
Late on the party :).

Yes, easy change for now. Or we could ship a new subpackage - selinux-policy-atomic which would ship policy files in /usr/lib and would be based on selinux-policy-minimum until we get seatomic working.
Comment 18 Lukas Vrabec 2015-12-11 12:42:56 EST
(In reply to Miroslav Grepl from comment #17)
> Late on the party :).
> 
> Yes, easy change for now. Or we could ship a new subpackage -
> selinux-policy-atomic which would ship policy files in /usr/lib and would be
> based on selinux-policy-minimum until we get seatomic working.
Exactly, 
Petr mentioned this in comment 14, just called selinux-policy-atomic as  " special targeted version".
Comment 19 Miroslav Grepl 2015-12-11 13:19:39 EST
(In reply to Lukas Vrabec from comment #18)
> (In reply to Miroslav Grepl from comment #17)
> > Late on the party :).
> > 
> > Yes, easy change for now. Or we could ship a new subpackage -
> > selinux-policy-atomic which would ship policy files in /usr/lib and would be
> > based on selinux-policy-minimum until we get seatomic working.
> Exactly, 
> Petr mentioned this in comment 14, just called selinux-policy-atomic as  "
> special targeted version".

Yeap and I would be more strict than Targeted policy is. It would be a great first step to have minimized policy on Atomic.
Comment 20 Stef Walter 2015-12-13 01:01:20 EST
Besides affecting any user who wants to change the port of a service, this affects Cockpit integration tests that try to change ssh the port. We'll likely mark these test failures as known issues until the issue is resolved.

https://github.com/cockpit-project/cockpit/issues/3326
Comment 21 Petr Lautrbach 2015-12-14 07:09:23 EST
(In reply to Colin Walters from comment #16)
> (In reply to Petr Lautrbach from comment #15)
> 
> > selinux-policy-targeted would ship /usr/lib/selinux/targeted and
> > /usr/lib/tmpfiles.d/selinux.conf with:
> > 
> > C /var/lib/selinux/targeted -    -    -    -  /usr/lib/selinux/targeted
> > 
> > So that after a factory reset when /var/lib/selinux/targeted doesn't exist,
> > all files from /usr/lib/selinux/targeted would be copied to
> > /var/lib/selinux/targeted.
> 
> I think the concern then is - do you change the RPM to start shipping the
> files in /usr?  I wonder if RPM would delete the old ones on upgrades?

It should.

> 
> That aside, this would be an easy change to make now.  
> 
> The downsides though: It means we double the disk usage though even if you
> haven't made any policy customizations. 

As a workaround, we can make the system modules directory /var/lib/selinux/targeted/active/modules/100 to be a symlink to /usr/lib/selinux/targeted/active/modules/100. It would break 'semodule -r' in cases where /usr is read-only but I guess that it's not widely used technique as these modules would be reinstalled on seliunx-policy update anyway.
Comment 22 Colin Walters 2015-12-15 12:29:51 EST
What the "systemd conffiles model" does for deletion is to have a symlink to /dev/null with the same filename.  See `systemctl mask`.
Comment 23 Miroslav Grepl 2015-12-15 16:47:26 EST
As Petr mentioned above we can use a different store location for policy files. We had a discussion with SELinux folks about that and we would be able to get selinux-policy-atomic.rpm package with policy store located back to /etc.

It would require a change in /etc/selinux/config to

SELINUXTYPE=atomic

and in /etc/selinux/semanage.conf

store-root=/etc/selinux

when Atomic host image is created together with requiring selinux-policy-atomic.

We would get a clean solution. We could play with selinux-policy-atomic to get a minimal set of policies until we have seatomic.

Does it make sense?
Comment 24 Colin Walters 2015-12-15 21:15:16 EST
I'm a little worried about conflating this bug with a switch to an entirely new policy.  I'm not strongly objecting to it if we think fixing the existing Fedora 23 stream is too hard, but it wasn't sounding above like it would be.

If we have to we can change the rpm-ostree compose to `mv /var/lib/selinux /usr/share/selinux` or so.
Comment 25 Miroslav Grepl 2015-12-16 06:30:34 EST
Actually this is not a new policy. It will be the same as selinux-policy-targeted but it will have a default store location in /etc. Only this change from a policy point of view.

So no hacks are needed. But maybe we are not aware about all compose implications with selinux-polic-atomic solution.
Comment 26 Miroslav Grepl 2015-12-17 11:30:30 EST
I am working on test builds with selinux-policy-atomic.rpm.

Also the point is we have broken policy because of this issue. So it is a good chance to start with a new policy - based on Targeted but with the module store located in /etc/selinux.
Comment 27 Miroslav Grepl 2015-12-18 02:34:36 EST
(In reply to Miroslav Grepl from comment #26)
> I am working on test builds with selinux-policy-atomic.rpm.

https://copr.fedoraproject.org/coprs/mgrepl/selinux-policy/build/148502/

> 
> Also the point is we have broken policy because of this issue. So it is a
> good chance to start with a new policy - based on Targeted but with the
> module store located in /etc/selinux.
Comment 28 Miroslav Grepl 2016-01-04 04:09:36 EST
The latest builds are available on

https://copr.fedoraproject.org/coprs/mgrepl/seatomic/build/151302/
Comment 29 Giovanni Tirloni 2016-05-30 18:27:45 EDT
Freshly installed F24 Atomic Host (2016-05-29 image).

# semanage port -a -t ssh_port_t -p tcp 4444
libsemanage.semanage_read_policydb: Could not open kernel policy /var/lib/selinux/targeted/active/policy.kern for reading. (No such file or directory).
FileNotFoundError: [Errno 2] No such file or directory

# rpm -qa | egrep '(atomic|selinux)' | sort
atomic-1.8-5.gitcc5997a.fc24.x86_64
atomic-devmode-0.3.3-1.fc24.noarch
docker-selinux-1.10.3-9.git667d6d1.fc24.x86_64
libatomic_ops-7.4.2-9.fc24.x86_64
libselinux-2.5-3.fc24.x86_64
libselinux-python-2.5-3.fc24.x86_64
libselinux-python3-2.5-3.fc24.x86_64
libselinux-utils-2.5-3.fc24.x86_64
rpm-plugin-selinux-4.13.0-0.rc1.27.fc24.x86_64
selinux-policy-3.13.1-189.fc24.noarch
selinux-policy-targeted-3.13.1-189.fc24.noarch

# getenforce 
Enforcing
Comment 30 Colin Walters 2016-07-25 09:50:14 EDT
(In reply to Miroslav Grepl from comment #26)
> I am working on test builds with selinux-policy-atomic.rpm.
> 
> Also the point is we have broken policy because of this issue. So it is a
> good chance to start with a new policy - based on Targeted but with the
> module store located in /etc/selinux.

/etc/selinux is OK, but the best implementation for ostree would be to have the defaults in /usr/share/selinux, and have /etc override, because by default content in /etc is implemented via physical copy, which makes upgrades slower and takes up more disk space.
Comment 31 Stef Walter 2016-09-09 02:39:27 EDT
This is still happening every day, and prevents a real world use case from working with SELinux enabled.
Comment 32 Daniel Walsh 2016-09-09 09:14:00 EDT
Stef what is the problem you are seeing today?
Comment 33 Stef Walter 2016-09-09 09:37:10 EDT
Users still cannot change the SSH port for the system on Fedora Atomic. This has been the case for a long long time.

Here's what happens during changing the SSH port on Fedora Atomic:

# atomic host status
● fedora-atomic:fedora-atomic/f23/x86_64/docker-host
       Version: 23.174 (2016-09-01 18:44:32)
        Commit: 8199da014de45a4bd23c45e772535101220d9991f54df74482b00d0efdb40580
        OSName: fedora-atomic
  GPGSignature: (unsigned)
# sed -i 's/.*Port .*/Port 2222/' /etc/ssh/sshd_config
# semanage port -a -t ssh_port_t -p tcp 2222
libsemanage.semanage_direct_get_module_info: Unable to open docker module lang ext file at /var/lib/selinux/targeted/tmp/modules/400/docker/lang_ext. (No such file or directory).

The above works on RHEL atomic. I can then do the following and have sshd listen on an alternate port:

# systemctl restart sshd
# netstat -an | grep 22
tcp        0      0 0.0.0.0:2222            0.0.0.0:*               LISTEN     
tcp6       0      0 :::2222                 :::*                    LISTEN     

We track this as a known issue in our integration tests here:

https://github.com/cockpit-project/cockpit/issues/3326

As noted at the above link, this bug report is the root cause for user facing issue. If this bug report is unrelated to the "can't change port" issue, then I can open a new bug.
Comment 34 Stef Walter 2016-09-09 11:22:02 EDT
Same thing on Fedora 24 Atomic:

# atomic host status
● fedora-atomic:fedora-atomic/24/x86_64/docker-host
       Version: 24.36 (2016-09-06 22:09:42)
        Commit: d9c8b8a31238e857f010c6fdc282f5f611d3c8af3e78caa891f7edb85822771b
        OSName: fedora-atomic
  GPGSignature: (unsigned)
# sed -i 's/.*Port .*/Port 2222/' /etc/ssh/sshd_config
# semanage port -a -t ssh_port_t -p tcp 2222
libsemanage.semanage_read_policydb: Could not open kernel policy /var/lib/selinux/targeted/active/policy.kern for reading. (No such file or directory).
FileNotFoundError: [Errno 2] No such file or directory
Comment 35 Daniel Walsh 2016-09-09 11:40:01 EDT
I am wondering if we are about to break this on RHEL also, since RHEL7.3 rebases a lot of the SELinux toolchain and selinux-policy.

Petr what is your current ideas on this?  Is this more of a policycoreutils issue or a selinux-policy issue?
Comment 36 Daniel Walsh 2016-09-09 11:40:38 EDT
This also blocks the permanent setting of booleans.
Comment 37 Petr Lautrbach 2016-09-09 12:15:54 EDT
(In reply to Daniel Walsh from comment #35)
> I am wondering if we are about to break this on RHEL also, since RHEL7.3
> rebases a lot of the SELinux toolchain and selinux-policy.

RHEL-7.3 uses store-root=/etc/selinux in order to prevent these kind of issues. So everything is in /etc/selinux and /var/lib/selinux is generally empty by default there.
 
> Petr what is your current ideas on this?  Is this more of a policycoreutils
> issue or a selinux-policy issue?

I'd consider this as an issue among SELinux userspace, which uses /var/lib/selinux as a permanent store for modules and local policy modifications; and ostree which deletes /var directory on factory reset.

We agreed on devconf that the best solution would be to change SELinux userspace to be able to use two types of store, read only in /usr/lib and a regular one in /var/lib/selinux. But, there hasn't been done any work on this yet.


What is feasible to do in few days is the solution from comment #c15 with a minor change. In order not to duplicate all files, we can only store cil module files in /usr/share/selinux/. It should be enough to recreate a store on boot and it would increase the size of selinux-policy-targeted by around 4M. It still has the size downside but at least it would work until there's a better solution.
Comment 38 Petr Lautrbach 2016-09-13 05:45 EDT
Created attachment 1200429 [details]
Do a factory reset when there's no policy.kern file in a store

It could be liked this. selinux-policy-targeted ships /usr/share/selinux/POLICYTYPE/default - a copy of /var/lib/selinux/POLICYTYPE/active
    
And there's selinux-factory-reset@targeted.service similar to autorelabel service which copies all files to the store and rebuilds policy.

You can test it using this scratch build:
http://koji.fedoraproject.org/koji/taskinfo?taskID=15611971

The problem can be simulated using this script:

find /var/lib/selinux/targeted -type f | xargs rm
Comment 39 Lukas Vrabec 2016-09-15 07:55:03 EDT
Patch from Petr was merged and will be part of next selinux-policy update for Fedora 24.

Thanks.
Comment 40 Fedora Update System 2016-09-15 20:53:10 EDT
selinux-policy-3.13.1-191.16.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-fe39b806b6
Comment 41 Fedora Update System 2016-09-21 20:24:02 EDT
selinux-policy-3.13.1-191.16.fc24 has been pushed to the Fedora 24 stable repository. If problems still persist, please make note of it in this bug report.
Comment 42 Jonathan Lebon 2016-10-04 15:49:54 EDT
This is causing the latest F25 composes to get stuck in a reboot loop. I haven't fleshed out the full story, but I see at least the following two issues:

1. Race condition with systemd-tmpfiles-setup: the selinux-factory-reset template service should have systemd-tmpfiles-setup.service in its After= list, so that the rpm-ostree tmpfiles conf has time to create /var/lib/selinux/targeted. Otherwise, the `cp` will just barf. 

2. The call to `semodule -B -n` actually fails with:
Oct 04 19:11:14 localhost.localdomain selinux-factory-reset[770]: libsemanage.semanage_direct_get_module_info: Unable to open docker module lang ext file at /var/lib/selinux/targeted/tmp/modules/400/docker/lang_ext. (No such file or directory).
Oct 04 19:11:14 localhost.localdomain selinux-factory-reset[770]: semodule:  Failed!

This is actually puzzling to me, though I don't know enough about SELinux modules to debug further. Seems like the factory reset script should also go through every pp in /usr/share/selinux/packages and install them?

At a higher level though, it provides a really sad user experience to have to reboot the system on first boot. Would it be too late to have a tailored selinux-policy-atomic package for F25?
Comment 43 Colin Walters 2016-10-04 15:53:54 EDT
Yeah, it's not going to be OK to do a reboot.

Let's back out this change until we can do more integration testing.
Comment 44 Colin Walters 2016-10-04 16:07:43 EDT
So yes, the immediate action here should be to go back to the previous state (pre-built selinux policy works, there's no systemd service, and semanage doesn't work).

Next, I'd like to scope in moving the policy to /usr, and having files in /etc override.  Like how systemd units work.

If we do a locally-compiled policy, it should also likely go in /etc, because otherwise rollbacks would get the *new* policy.

So basically /var isn't used for SELinux, which would bring us in line with what RHEL7/CentOS7 do today, except we support read-only defaults in /usr.

(Although this is just an *optimization* over what ostree already does automatically with /usr/etc which contains read-only copies of /etc, including the policy defaults)
Comment 45 Ian McLeod 2016-10-05 12:15:38 EDT
Marking as urgent since it appears to be blocking the creation of a working F25-based Atomic Host and will likely break any updated F24 trees.
Comment 47 Colin Walters 2016-10-05 15:02:46 EDT
In the future, can we please loop in the Atomic Host team to review/test these things before we ship?  You can email https://lists.fedoraproject.org/archives/list/cloud@lists.fedoraproject.org/ in general, but in this specific case I think we have everyone relevant on the bugzilla.

I was busy when the patch was posted in
https://bugzilla.redhat.com/show_bug.cgi?id=1290659#c38
and it just didn't get on my radar to test.  So let's be sure for the next patches we're on the same page.

(FWIW I test with a custom treecompose, and rebase a vagrant box to it)
Comment 48 Petr Lautrbach 2016-10-06 12:02:25 EDT
(In reply to Jonathan Lebon from comment #42)
> This is causing the latest F25 composes to get stuck in a reboot loop. I
> haven't fleshed out the full story, but I see at least the following two
> issues:

Reboot was chosen as the safest way how to reload a policy and apply all possible changes to already running processes. As there's only systemd running at the moment it's probably not the case and 'semodule -B -R' could be enough.

> 
> 1. Race condition with systemd-tmpfiles-setup: the selinux-factory-reset
> template service should have systemd-tmpfiles-setup.service in its After=
> list, so that the rpm-ostree tmpfiles conf has time to create
> /var/lib/selinux/targeted. Otherwise, the `cp` will just barf. 

It could be fixed directly in selinux-factory-reset which would prepare needed directory structure itself.

> 
> 2. The call to `semodule -B -n` actually fails with:
> Oct 04 19:11:14 localhost.localdomain selinux-factory-reset[770]:
> libsemanage.semanage_direct_get_module_info: Unable to open docker module
> lang ext file at /var/lib/selinux/targeted/tmp/modules/400/docker/lang_ext.
> (No such file or directory).
> Oct 04 19:11:14 localhost.localdomain selinux-factory-reset[770]: semodule: 
> Failed!

This is caused by the rpm-ostree-1-autovar.conf which creates an empty directory for docker module.

At the same time, it  reveals the fact that the factory SELinux policy is not only selinux-policy-targeted but it's docker policy too. And it would need to be reinstalled during boot after factory reset together with the system policy.

> 
> This is actually puzzling to me, though I don't know enough about SELinux
> modules to debug further. Seems like the factory reset script should also go
> through every pp in /usr/share/selinux/packages and install them?

Since it's called factory reset, it probably make sense.
 
> At a higher level though, it provides a really sad user experience to have
> to reboot the system on first boot. Would it be too late to have a tailored
> selinux-policy-atomic package for F25?

There were some concerns about transitions from -targeted to -atomic for existing users.

If the process which puts /usr/lib/tmpfiled.s/rpm-ostree* files can change /etc/selinux/semanage.conf to "store-root=/etc/selinux" and /etc/selinux/config to use "SELINUXTYPE=atomic" then we can simply build selinux-policy-atomic will all modules and files in /etc/selinux/atomic and it can be used instead of -targeted.
Comment 49 Colin Walters 2016-10-06 12:24:52 EDT
(In reply to Petr Lautrbach from comment #48)

> If the process which puts /usr/lib/tmpfiled.s/rpm-ostree* files can change
> /etc/selinux/semanage.conf to "store-root=/etc/selinux" 

Yeah, that's fairly easy.

> /etc/selinux/config to use "SELINUXTYPE=atomic" then we can simply build
> selinux-policy-atomic will all modules and files in /etc/selinux/atomic and
> it can be used instead of -targeted.

Do you think it'd be possible to just `mv /var/lib/selinux/targeted /etc/selinux/targeted` too in the treecompose?  Then we wouldn't need a new policy package.

Basically, I'd prefer for both of us to avoid the maintenance burden, and try to have the same thing work on yum-managed systems as well as ostree-managed systems.
Comment 50 Petr Lautrbach 2016-10-06 15:19:02 EDT
(In reply to Colin Walters from comment #49)
> Do you think it'd be possible to just `mv /var/lib/selinux/targeted
> /etc/selinux/targeted` too in the treecompose?  Then we wouldn't need a new
> policy package.
> 

It's possible to move files and directories from /var/lib/selinux/targeted/ not the targeted directory itself. I will confirm it again tomorrow, but simple test works:

# semanage port -a -t ssh_port_t -p tcp 22100

# semodule -l | wc -l
411

# echo 'store-root=/etc/selinux' >> /etc/selinux/semanage.conf

# mv -v /var/lib/selinux/targeted/* /etc/selinux/targeted/    
'/var/lib/selinux/targeted/active' -> '/etc/selinux/targeted/active'
'/var/lib/selinux/targeted/semanage.read.LOCK' -> '/etc/selinux/targeted/semanage.read.LOCK'
'/var/lib/selinux/targeted/semanage.trans.LOCK' -> '/etc/selinux/targeted/semanage.trans.LOCK'

# semodule -l | wc -l                                         
411

# semanage port -C -l
SELinux Port Type              Proto    Port Number

ssh_port_t                     tcp      22100

# ls -l /var/lib/selinux/targeted 
total 0
Comment 51 Fedora Update System 2016-10-07 02:24:49 EDT
selinux-policy-3.13.1-218.fc25 has been pushed to the Fedora 25 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-3b70b59f26
Comment 52 Colin Walters 2016-10-07 11:33:15 EDT
I worked on implementing comment #50 in https://github.com/projectatomic/rpm-ostree/pull/479

Offhand, it seems to work OK.
Comment 53 Jonathan Lebon 2016-10-07 12:45:19 EDT
Successfully rebased F24 to an F25 compose running the PR above and the latest selinux-policy Bodhi update. And able to manage policy.
Comment 54 Fedora Update System 2016-10-08 22:42:56 EDT
selinux-policy-3.13.1-218.fc25 has been pushed to the Fedora 25 stable repository. If problems still persist, please make note of it in this bug report.
Comment 55 aurelien 2017-01-12 03:38:32 EST
Seems it comes back for 2017!


Linux icesheet 4.8.15-300.fc25.x86_64 #1 SMP Thu Dec 15 23:10:23 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux


libsemanage.semanage_direct_get_module_info: Unable to read firewalld module lang ext file.
semodule:  Failed!

libsemanage.semanage_direct_get_module_info: Unable to read firewalld module lang ext file.
/usr/sbin/semodule:  Failed!
Comment 56 Dusty Mabe 2017-01-12 14:59:37 EST
(In reply to aurelien from comment #55)
> Seems it comes back for 2017!
> 

I don't see the issue:

```
-bash-4.3# rpm-ostree status
State: idle
Deployments:
● fedora-atomic:fedora-atomic/25/x86_64/docker-host
       Version: 25.30 (2017-01-06 07:27:20)
        Commit: 547fa56532850a388ff152fa543f05214f7d1da9d1dda471e3e04259a2f5ff86
        OSName: fedora-atomic
-bash-4.3# 
-bash-4.3# rpm -q selinux-policy-targeted 
selinux-policy-targeted-3.13.1-225.3.fc25.noarch
-bash-4.3# 
-bash-4.3# 
-bash-4.3# semanage port -a -t ssh_port_t -p tcp 2222
-bash-4.3# echo $?
0

```
Comment 57 Stef Walter 2017-07-18 04:05:55 EDT
Confirming that this no longer happens. Removing as a known issue from Cockpit integration tests.

Note You need to log in before you can comment on or make changes to this bug.