There are symptoms that some packages with SELinux modules, e. g. container-selinux, had not been properly installed before the update started. I was not able to reproduce this problem. My system was fully updated and I checked all SELinux modules were installed and active.

Do you know what was the system's state before the updating process started?

rpm -q container-selinux
semodule -lfull | grep container

My system:
Before update: container-selinux-2.177.0-1.fc35.noarch
After update: container-selinux-2.178.0-1.fc36.noarch

Thank you for checking it.

I suspect that one of these events (which I did before upgrading) could cause the issue:
1. I was most probably running Virtualbox VMs before and while upgrading.
2. I did not restart the system after performing the "sudo dnf upgrade --refresh" command, before proceeding with the upgrade.
3. I might have removed Nvidia drivers without restarting using "sudo dnf remove akmod-nvidia nvidia-*" command, before proceeding with the upgrade.
4. There were a few packages which could not be upgraded using the "sudo dnf system-upgrade download --release=36" command, and I had to add the option --allowerasing.

rpm -q container-selinux
container-selinux-2.178.0-1.fc36.noarch

semodule -lfull | grep container
200 container         pp          

I do not mind simply reinstalling the whole system, I am mostly reporting it because I thought that maybe someone else could experience the same issue.

(In reply to Miroslav Lakota from comment #2)
> Thank you for checking it.
> 
> I suspect that one of these events (which I did before upgrading) could
> cause the issue:
> 1. I was most probably running Virtualbox VMs before and while upgrading.
> 2. I did not restart the system after performing the "sudo dnf upgrade
> --refresh" command, before proceeding with the upgrade.
> 3. I might have removed Nvidia drivers without restarting using "sudo dnf
> remove akmod-nvidia nvidia-*" command, before proceeding with the upgrade.
> 4. There were a few packages which could not be upgraded using the "sudo dnf
> system-upgrade download --release=36" command, and I had to add the option
> --allowerasing.

At first glance none of them looks like related.

> 
> rpm -q container-selinux
> container-selinux-2.178.0-1.fc36.noarch
> 
> semodule -lfull | grep container
> 200 container         pp          
> 
> I do not mind simply reinstalling the whole system, I am mostly reporting it
> because I thought that maybe someone else could experience the same issue.

It really looks like the known issue which can be resolved with updating to the latest packages version.
If you are not experiencing any problems right now, you should also not need to do any further action.
Therefore closing this bz, but feel free to open a new bugzilla or to reopen this one in case of an outstanding issue.

I have an identical problem. F 36 updated from, also required allow erasing. 
I have remove podman and crun but flatpak won't update.


Total                                           3.4 MB/s |  15 MB     00:04     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                        1/1 
  Running scriptlet: flatpak-1.12.6-1.fc36.x86_64                           1/6 
  Upgrading        : flatpak-1.12.6-1.fc36.x86_64                           1/6 
error: lsetfilecon: (/usr/libexec/flatpak-system-helper;621e9348, system_u:object_r:flatpak_helper_exec_t:s0) Invalid argument
error: Plugin selinux: hook fsm_file_prepare failed


I am using updates repositories. 

[tim@fedora ~]$ rpm -q container-selinux
container-selinux-2.178.0-1.fc36.noarch

If this is not the latest version, why not? I removed it and reinstalled it.

(In reply to tim richardson from comment #4)
> I have an identical problem. F 36 updated from, also required allow erasing. 
> I have remove podman and crun but flatpak won't update.
> 
> 
> Total                                           3.4 MB/s |  15 MB     00:04 
> 
> Running transaction check
> Transaction check succeeded.
> Running transaction test
> Transaction test succeeded.
> Running transaction
>   Preparing        :                                                       
> 1/1 
>   Running scriptlet: flatpak-1.12.6-1.fc36.x86_64                          
> 1/6 
>   Upgrading        : flatpak-1.12.6-1.fc36.x86_64                          
> 1/6 
> error: lsetfilecon: (/usr/libexec/flatpak-system-helper;621e9348,
> system_u:object_r:flatpak_helper_exec_t:s0) Invalid argument
> error: Plugin selinux: hook fsm_file_prepare failed
> 
> 
> I am using updates repositories. 
> 
> [tim@fedora ~]$ rpm -q container-selinux
> container-selinux-2.178.0-1.fc36.noarch
> 
> If this is not the latest version, why not? I removed it and reinstalled it.

It may still be a result of flatpak module not correctly installed before the update. If you can see the following:

# matchpathcon /usr/libexec/flatpak-system-helper
/usr/libexec/flatpak-system-helper      system_u:object_r:flatpak_helper_exec_t:s0
# semodule -lfull | grep flatpak
200 flatpak           pp
# seinfo -xt flatpak_helper_exec_t

Types: 1
   type flatpak_helper_exec_t, application_exec_type, entry_type, exec_type, file_type, non_auth_file_type, non_security_file_type, direct_init_entry, systemprocess_entry;

then no other action should be needed. Note the latter command is from setools-console.

I have the same issue. Snapd, Flatpak, Podman etc failed to upgrade on Fedora 36. Someone on Reddit /r/fedora had the same issue.

(In reply to P D from comment #6)
> I have the same issue. Snapd, Flatpak, Podman etc failed to upgrade on
> Fedora 36. Someone on Reddit /r/fedora had the same issue.

Same problem here, upgraded to 36 today.

I have run into the same or similar issue here: https://bugzilla.redhat.com/show_bug.cgi?id=2069325

This is after an upgrade to F36.

Strangely, after removing all selinux-policy packages and rebooting I'm left with:

$ sudo semodule -l
container
flatpak
smartmon
snappy
swtpm
swtpm_svirt

When I try to remove a module manually, I receive an AST error.

$ sudo semodule -X200 -r snappy 
libsemanage.semanage_direct_remove_key: Removing last snappy module (no other snappy module exists at another priority).
Failed to resolve typealiasactual statement at /var/lib/selinux/targeted/tmp/modules/200/container/cil:6
Failed to resolve AST
semodule:  Failed!

Reinstalling selinux-policy and its dependencies:

$ sudo dnf install container-selinux                             
Last metadata expiration check: 0:14:24 ago on Mon 28 Mar 2022 01:48:45 PM EDT.
Dependencies resolved.
==============================================================================================================================================================================================================================================
 Package                                                             Architecture                                       Version                                                      Repository                                          Size
==============================================================================================================================================================================================================================================
Installing:
 container-selinux                                                   noarch                                             2:2.180.0-1.fc36                                             fedora                                              50 k
Installing dependencies:
 flatpak-selinux                                                     noarch                                             1.12.6-1.fc36                                                fedora                                              22 k
 rpm-plugin-selinux                                                  x86_64                                             4.17.0-10.fc36                                               fedora                                              21 k
 selinux-policy                                                      noarch                                             36.5-1.fc36                                                  fedora                                              71 k
 selinux-policy-targeted                                             noarch                                             36.5-1.fc36                                                  fedora                                             6.3 M
 smartmontools-selinux                                               noarch                                             1:7.2-12.fc36                                                fedora                                              23 k

Transaction Summary
==============================================================================================================================================================================================================================================
Install  6 Packages

Total download size: 6.5 M
Installed size: 18 M
Is this ok [y/N]: y
Downloading Packages:
(1/6): flatpak-selinux-1.12.6-1.fc36.noarch.rpm                                                                                                                                                                39 kB/s |  22 kB     00:00    
(2/6): rpm-plugin-selinux-4.17.0-10.fc36.x86_64.rpm                                                                                                                                                            34 kB/s |  21 kB     00:00    
(3/6): container-selinux-2.180.0-1.fc36.noarch.rpm                                                                                                                                                             81 kB/s |  50 kB     00:00    
(4/6): smartmontools-selinux-7.2-12.fc36.noarch.rpm                                                                                                                                                           267 kB/s |  23 kB     00:00    
(5/6): selinux-policy-36.5-1.fc36.noarch.rpm                                                                                                                                                                  412 kB/s |  71 kB     00:00    
(6/6): selinux-policy-targeted-36.5-1.fc36.noarch.rpm                                                                                                                                                          10 MB/s | 6.3 MB     00:00    
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                                         4.4 MB/s | 6.5 MB     00:01     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Running scriptlet: selinux-policy-targeted-36.5-1.fc36.noarch                                                                                                                                                                           1/1 
  Preparing        :                                                                                                                                                                                                                      1/1 
  Installing       : rpm-plugin-selinux-4.17.0-10.fc36.x86_64                                                                                                                                                                             1/6 
  Installing       : selinux-policy-36.5-1.fc36.noarch                                                                                                                                                                                    2/6 
  Running scriptlet: selinux-policy-36.5-1.fc36.noarch                                                                                                                                                                                    2/6 
  Running scriptlet: selinux-policy-targeted-36.5-1.fc36.noarch                                                                                                                                                                           3/6 
  Installing       : selinux-policy-targeted-36.5-1.fc36.noarch                                                                                                                                                                           3/6 
  Running scriptlet: selinux-policy-targeted-36.5-1.fc36.noarch                                                                                                                                                                           3/6 
Failed to resolve allow statement at /var/lib/selinux/targeted/tmp/modules/200/container/cil:1263
Failed to resolve AST
/usr/sbin/semodule:  Failed!

  Running scriptlet: container-selinux-2:2.180.0-1.fc36.noarch                                                                                                                                                                            4/6 
  Installing       : container-selinux-2:2.180.0-1.fc36.noarch                                                                                                                                                                            4/6 
  Running scriptlet: container-selinux-2:2.180.0-1.fc36.noarch                                                                                                                                                                            4/6 
Failed to resolve allow statement at /var/lib/selinux/targeted/tmp/modules/200/container/cil:1263
Failed to resolve AST
Failed to commit changes to booleans: Success
Failed to resolve allow statement at /var/lib/selinux/targeted/tmp/modules/200/snappy/cil:305
Failed to resolve AST
/usr/sbin/semodule:  Failed!

  Running scriptlet: smartmontools-selinux-1:7.2-12.fc36.noarch                                                                                                                                                                           5/6 
  Installing       : smartmontools-selinux-1:7.2-12.fc36.noarch                                                                                                                                                                           5/6 
  Running scriptlet: smartmontools-selinux-1:7.2-12.fc36.noarch                                                                                                                                                                           5/6 
Failed to resolve allow statement at /var/lib/selinux/targeted/tmp/modules/200/container/cil:1263
Failed to resolve AST
/usr/sbin/semodule:  Failed!

  Installing       : flatpak-selinux-1.12.6-1.fc36.noarch                                                                                                                                                                                 6/6 
  Running scriptlet: flatpak-selinux-1.12.6-1.fc36.noarch                                                                                                                                                                                 6/6 
Failed to resolve allow statement at /var/lib/selinux/targeted/tmp/modules/200/container/cil:1263
Failed to resolve AST
/usr/sbin/semodule:  Failed!

  Running scriptlet: selinux-policy-targeted-36.5-1.fc36.noarch                                                                                                                                                                           6/6 
  Running scriptlet: container-selinux-2:2.180.0-1.fc36.noarch                                                                                                                                                                            6/6 
  Running scriptlet: flatpak-selinux-1.12.6-1.fc36.noarch                                                                                                                                                                                 6/6 
  Verifying        : container-selinux-2:2.180.0-1.fc36.noarch                                                                                                                                                                            1/6 
  Verifying        : flatpak-selinux-1.12.6-1.fc36.noarch                                                                                                                                                                                 2/6 
  Verifying        : rpm-plugin-selinux-4.17.0-10.fc36.x86_64                                                                                                                                                                             3/6 
  Verifying        : selinux-policy-36.5-1.fc36.noarch                                                                                                                                                                                    4/6 
  Verifying        : selinux-policy-targeted-36.5-1.fc36.noarch                                                                                                                                                                           5/6 
  Verifying        : smartmontools-selinux-1:7.2-12.fc36.noarch                                                                                                                                                                           6/6 

Installed:
  container-selinux-2:2.180.0-1.fc36.noarch         flatpak-selinux-1.12.6-1.fc36.noarch        rpm-plugin-selinux-4.17.0-10.fc36.x86_64        selinux-policy-36.5-1.fc36.noarch        selinux-policy-targeted-36.5-1.fc36.noarch       
  smartmontools-selinux-1:7.2-12.fc36.noarch       

But SELinux is in a borken state.

I just tried to upgrade another machine and ended with the same issue again. The last time I had to reinstall the whole system. I found no solution.

I was able to upgrade without having the issue, and this is how:

Before upgrade, I removed snapd. I also reinstalled container-selinux, flatpak-selinux, selinux-policy, selinux-policy-targeted, etc.

I didn't enable updates-testing repos until after I upgraded.

After upgrade, I didn't reinstall snapd as I no longer need it.

I wish I had seen this before I upgraded my laptop this morning. I have removed podman, snapd and many things, but I can't remove flatpak, and I'm stuck.

Try removing the troublesome selinux modules together (semodule -X 200 -r container -r flatpak -r snapd) and reinstalling whatever you need.

Workaround for me: 'sudo semodule -X 200 -r snappy -r container -r flatpak -X 400 -r pcpupstream -r pcpupstream-container -X 100 -r pcp', then run 'sudo dnf upgrade' and I can finally upgrade podman and  it works.
Reinstalling 'dnf install -y container-selinux' fails with (takes a while too as it runs restorecon):
```
Failed to resolve typeattributeset statement at /var/lib/selinux/targeted/tmp/modules/400/pcpupstream-container/cil:4
Failed to resolve AST
semodule:  Failed!
```

Removed it with 'sudo dnf remove pcp' (this actually causes auditd and pcmd to spin at 100% CPU usage, stopped it with 'systemctl stop pmcd') and tried again. I could reinstall container-selinux and podman successfully now.
Containers failed to run, trying 'fixfiles -B onboot' again

However podman now fails to run using btrfs storage driver still with an SELinux error (even after 'podman system reset'), so there is still something wrong with SELinux:

Error: error creating container storage: error creating read-write layer with ID "db743395667be6b494116c7ce7e11c4afda29cc875e3567a7040d0804ab1c6b1": setxattr /home/edwin/.local/share/containers/storage/btrfs/subvolumes/db743395667be6b494116c7ce7e11c4afda29cc875e3567a7040d0804ab1c6b1/etc/alternatives/libnssckbi.so.x86_64: operation not permitted

Opened new bug here about the SELinux error with the btrfs driver: https://bugzilla.redhat.com/show_bug.cgi?id=2071065, the default one works.
There is still something wrong with container-selinux package, it shouldn't have failed like that on upgrade requiring manual 'semodule -r'. I think that this bug should at least be mentioned on https://fedoraproject.org/wiki/Common_F36_bugs

(In reply to Török Edwin from comment #15)
> There is still something wrong with container-selinux package, it shouldn't
> have failed like that on upgrade requiring manual 'semodule -r'. I think
> that this bug should at least be mentioned on
> https://fedoraproject.org/wiki/Common_F36_bugs

I have created a Common Bugs proposal:

https://ask.fedoraproject.org/t/dnf-upgrade-of-some-packages-fail-after-upgrade-from-f35/20983

The entry is not complete yet,
so if you have any input, in particular for Cause or Workaround sections,
please edit the proposal.

*** Bug 2071059 has been marked as a duplicate of this bug. ***

Thanks for the common bugs proposal, I've mentioned the workaround.

In my case the problem might've originated from the pcp-selinux package, since reinstalling that still fails:
```
Running transaction
  Preparing        :                                                                                                                          1/1 
  Installing       : pcp-selinux-5.3.6-2.fc36.x86_64                                                                                          1/4 
  Running scriptlet: pcp-selinux-5.3.6-2.fc36.x86_64                                                                                          1/4 
Failed to resolve typeattributeset statement at /var/lib/selinux/targeted/tmp/modules/400/pcpupstream/cil:5
Failed to resolve AST
semodule:  Failed!

Failed to resolve typeattributeset statement at /var/lib/selinux/targeted/tmp/modules/400/pcpupstream-container/cil:4
Failed to resolve AST
semodule:  Failed!

```

I've reported the pcp issue here: https://bugzilla.redhat.com/show_bug.cgi?id=2071127

*** Bug 2070942 has been marked as a duplicate of this bug. ***

Might have something to do with older version of files.

Fixed by removing all modules with complaints, in my case it was:

semodule -X 200 -r snappy -r container  -X 300 -r my-chown -X 400 -r my-chown -r my-systemctl
dnf reinstall -y container-selinux

*** Bug 2069325 has been marked as a duplicate of this bug. ***

(In reply to bryanhoop from comment #8)
> I have run into the same or similar issue here:
> https://bugzilla.redhat.com/show_bug.cgi?id=2069325
> 
> This is after an upgrade to F36.
> 
> Strangely, after removing all selinux-policy packages and rebooting I'm left
> with:
> 
> $ sudo semodule -l
> container
> flatpak
> smartmon
> snappy
> swtpm
> swtpm_svirt

This looks like none of the distribution policy modules (content of selinux-policy-targeted) are installed, which probably means that selinux-policy-targeted failed to updgrade (probably because of some of container-selinux).

> 
> When I try to remove a module manually, I receive an AST error.
> 
> $ sudo semodule -X200 -r snappy 
> libsemanage.semanage_direct_remove_key: Removing last snappy module (no
> other snappy module exists at another priority).
> Failed to resolve typealiasactual statement at
> /var/lib/selinux/targeted/tmp/modules/200/container/cil:6
> Failed to resolve AST
> semodule:  Failed!

Because of dependencies between the modules it is best to remove them all in a single transaction.

I wasn't able to reproduce the issue so far, but my troubleshooting steps would be:
1) remove container-selinux package
2) try to reinstall selinux-policy and selinux-policy-targeted
3) If ^^ fails, remove all custom modules (priority other than 100) manually
 # sudo semodule -lfull | grep -v 100
 # sudo semodule -X <priority> -r <module>
 e.g.
 # sudo semodule -X 200 -r container -X 400 -r restraint
 and retry step 2

I also saw this issue after upgrading to Fedora 36.

Removing all modules other than priority 100 as suggested worked, and I was able to do `dnf reinstall container-selinux` afterwards and get things back on track.

Prior to this I had these modules installed:

200 container         pp          
200 flatpak           pp          
200 smartmon          pp          
200 snappy            pp          
200 swtpm             pp          
200 swtpm_svirt       pp          
200 tabrmd            pp

When I tried to remove only the snappy module (the one the error messages I was seeing referenced) I got:

$ sudo semodule -X 200 -r snappy
libsemanage.semanage_direct_remove_key: Removing last snappy module (no other snappy module exists at another priority).
Failed to resolve allow statement at /var/lib/selinux/targeted/tmp/modules/200/container/cil:1263
Failed to resolve AST
semodule:  Failed!

Running the command below was successful:

$ sudo semodule -X 200 -r snappy -r container -r flatpak -r smartmon -r swtpm -r swtpm_svirt -r tabrmd

The conflict seems to be between container-selinux and snappy.

*** Bug 2070702 has been marked as a duplicate of this bug. ***

Thank you for the fix everybody and especially Vit for also providing context!
My system is in much better shape now.

I am trying to write a Common Bug entry for this problem,
so I would like to as for some more clarifications:

After following your instructions,
all the previously installed semodules are missing,
i.e. 'semodule -lfull | grep -v 100' output is empty.
I can fix this by reinstalling packages that provide those modules,
e.g. 'dnf reinstall  flatpak-selinux'.
Is this the correct next step?

Another problem is that I do not know which packages provide these two modules: swtpm, swtpm_svirt.
How can I find package name starting from semodule name?

Finally, to successfully boot in SELinux enforcing mode,
I had to relabel everything once more.
Otherwise I got black screen with blinking cursor on boot.
Is this is to be expected,
should I instruct all users to do 'fixfiles -F onboot && reboot' afterwards?

(In reply to Otto Urpelainen from comment #26)
> Thank you for the fix everybody and especially Vit for also providing
> context!
> My system is in much better shape now.
> 
> I am trying to write a Common Bug entry for this problem,
> so I would like to as for some more clarifications:
> 
> After following your instructions,
> all the previously installed semodules are missing,
> i.e. 'semodule -lfull | grep -v 100' output is empty.
> I can fix this by reinstalling packages that provide those modules,
> e.g. 'dnf reinstall  flatpak-selinux'.
> Is this the correct next step?

Yes.

> 
> Another problem is that I do not know which packages provide these two
> modules: swtpm, swtpm_svirt.
> How can I find package name starting from semodule name?


# dnf provides /usr/share/selinux/packages/swtpm.pp
shows swtpm package (it seems to contain both swtpm and swtpm_svirt).

> 
> Finally, to successfully boot in SELinux enforcing mode,
> I had to relabel everything once more.
> Otherwise I got black screen with blinking cursor on boot.
> Is this is to be expected,
> should I instruct all users to do 'fixfiles -F onboot && reboot' afterwards?

Yes. Misslabeled files are to be expected with such a severe policy issue.

For me 'sudo dnf reinstall selinux-policy-targeted' was also required to fix the remaining problems with pcp-selinux, see https://bugzilla.redhat.com/show_bug.cgi?id=2071127#c5

Created attachment 1871123 [details]
On my System the same Bug occured after Update to 36

zephyrus@zephs-fedora-wks:~$ upd-sys
Copr repo for PyCharm owned by phracek                                            8.7 kB/s | 3.6 kB     00:00    
balena-etcher                                                                     2.2 kB/s | 648  B     00:00    
balena-etcher-noarch                                                              2.2 kB/s | 648  B     00:00    
balena-etcher-source                                                              2.4 kB/s | 648  B     00:00    
Fedora 36 - x86_64                                                                 36 kB/s | 6.5 kB     00:00    
Fedora 36 - x86_64                                                                1.6 MB/s | 1.7 MB     00:01    
Fedora 36 openh264 (From Cisco) - x86_64                                          7.3 kB/s | 989  B     00:00    
Fedora Modular 36 - x86_64                                                         63 kB/s | 8.9 kB     00:00    
Fedora Modular 36 - x86_64                                                        324 kB/s | 159 kB     00:00    
Fedora 36 - x86_64 - Updates                                                      128 kB/s |  21 kB     00:00    
Fedora Modular 36 - x86_64 - Updates                                              137 kB/s |  20 kB     00:00    
Fedora 36 - x86_64 - Test Updates                                                  63 kB/s | 9.7 kB     00:00    
Fedora Modular 36 - x86_64 - Test Updates                                         113 kB/s |  18 kB     00:00    
google-chrome                                                                     6.0 kB/s | 1.3 kB     00:00    
RPM Fusion for Fedora 36 - Free                                                   9.2 kB/s | 3.9 kB     00:00    
RPM Fusion for Fedora 36 - Free tainted                                           8.9 kB/s | 3.6 kB     00:00    
RPM Fusion for Fedora 36 - Free - Test Updates                                    8.9 kB/s | 3.7 kB     00:00    
RPM Fusion for Fedora 36 - Nonfree                                                9.8 kB/s | 3.9 kB     00:00    
RPM Fusion for Fedora 36 - Nonfree - NVIDIA Driver                                 10 kB/s | 4.1 kB     00:00    
RPM Fusion for Fedora 36 - Nonfree - Steam                                         10 kB/s | 3.9 kB     00:00    
RPM Fusion for Fedora 36 - Nonfree - Test Updates                                 9.7 kB/s | 3.8 kB     00:00    
teams                                                                              15 kB/s | 3.0 kB     00:00    
Visual Studio Code                                                                 22 kB/s | 3.0 kB     00:00    
Abhängigkeiten sind aufgelöst.
==================================================================================================================
 Paket                     Architektur    Version                                   Paketquelle             Größe
==================================================================================================================
Aktualisieren:
 conmon                    x86_64         2:2.1.0-2.fc36                            fedora                   59 k
 containers-common         noarch         4:1-53.fc36                               fedora                   78 k
 crun                      x86_64         1.4.4-1.fc36                              fedora                  188 k
 flatpak                   x86_64         1.12.7-1.fc36                             fedora                  1.6 M
 podman                    x86_64         3:4.0.3-1.fc36                            updates-testing          12 M
 snapd                     x86_64         2.54.4-1.fc36                             fedora                   13 M
 swtpm                     x86_64         0.7.2-1.20220307git21c90c1.fc36           fedora                   42 k

Transaktionszusammenfassung
==================================================================================================================
Aktualisieren  7 Pakete

Gesamte Downloadgröße: 27 M
Ist dies in Ordnung? [j/N]: j
Pakete werden heruntergeladen:
(1/7): conmon-2.1.0-2.fc36.x86_64.rpm                                             133 kB/s |  59 kB     00:00    
(2/7): containers-common-1-53.fc36.noarch.rpm                                     174 kB/s |  78 kB     00:00    
(3/7): swtpm-0.7.2-1.20220307git21c90c1.fc36.x86_64.rpm                            92 kB/s |  42 kB     00:00    
(4/7): crun-1.4.4-1.fc36.x86_64.rpm                                               407 kB/s | 188 kB     00:00    
(5/7): flatpak-1.12.7-1.fc36.x86_64.rpm                                           2.2 MB/s | 1.6 MB     00:00    
(6/7): snapd-2.54.4-1.fc36.x86_64.rpm                                             5.5 MB/s |  13 MB     00:02    
(7/7): podman-4.0.3-1.fc36.x86_64.rpm                                             3.9 MB/s |  12 MB     00:03    
------------------------------------------------------------------------------------------------------------------
Gesamt                                                                            8.0 MB/s |  27 MB     00:03     
Transaktionsüberprüfung wird ausgeführt
Transaktionsüberprüfung war erfolgreich.
Transaktion wird getestet
Transaktionstest war erfolgreich.
Transaktion wird ausgeführt
  Vorbereitung läuft    :                                                                                     1/1 
  Aktualisieren         : crun-1.4.4-1.fc36.x86_64                                                           1/14 
Fehler: lsetfilecon: (/usr/bin/crun;624dcb38, system_u:object_r:container_runtime_exec_t:s0) Das Argument ist ungültig
Fehler: Plugin selinux: hook fsm_file_prepare failed

Error unpacking rpm package crun-1.4.4-1.fc36.x86_64
  Aktualisieren         : containers-common-4:1-53.fc36.noarch                                               2/14 
Fehler: Entpacken des Archivs fehlgeschlagen bei Datei /usr/bin/crun;624dcb38: cpio: (Fehler 0x2)
Fehler: crun-1.4.4-1.fc36.x86_64: installieren fehlgeschlagen
Fehler: lsetfilecon: (/var/lib/containers/sigstore, system_u:object_r:container_var_lib_t:s0) Das Argument ist ungültig
Fehler: Plugin selinux: hook fsm_file_prepare failed

Error unpacking rpm package containers-common-4:1-53.fc36.noarch
  Aktualisieren         : conmon-2:2.1.0-2.fc36.x86_64                                                       3/14 
Fehler: Entpacken des Archivs fehlgeschlagen bei Datei /var/lib/containers/sigstore: cpio: (Fehler 0x2)
Fehler: containers-common-4:1-53.fc36.noarch: installieren fehlgeschlagen
Fehler: lsetfilecon: (/usr/bin/conmon;624dcb38, system_u:object_r:conmon_exec_t:s0) Das Argument ist ungültig
Fehler: Plugin selinux: hook fsm_file_prepare failed

Error unpacking rpm package conmon-2:2.1.0-2.fc36.x86_64
  Aktualisieren         : podman-3:4.0.3-1.fc36.x86_64                                                       4/14 
Fehler: Entpacken des Archivs fehlgeschlagen bei Datei /usr/bin/conmon;624dcb38: cpio: (Fehler 0x2)
Fehler: conmon-2:2.1.0-2.fc36.x86_64: installieren fehlgeschlagen
Fehler: lsetfilecon: (/usr/bin/podman;624dcb38, system_u:object_r:container_runtime_exec_t:s0) Das Argument ist ungültig
Fehler: Plugin selinux: hook fsm_file_prepare failed

Error unpacking rpm package podman-3:4.0.3-1.fc36.x86_64
  Aktualisieren         : swtpm-0.7.2-1.20220307git21c90c1.fc36.x86_64                                       5/14 
Fehler: Entpacken des Archivs fehlgeschlagen bei Datei /usr/bin/podman;624dcb38: cpio: (Fehler 0x2)
Fehler: podman-3:4.0.3-1.fc36.x86_64: installieren fehlgeschlagen
Fehler: lsetfilecon: (/usr/bin/swtpm;624dcb38, system_u:object_r:swtpm_exec_t:s0) Das Argument ist ungültig
Fehler: Plugin selinux: hook fsm_file_prepare failed

Error unpacking rpm package swtpm-0.7.2-1.20220307git21c90c1.fc36.x86_64
  Aktualisieren         : snapd-2.54.4-1.fc36.x86_64                                                         6/14 
Fehler: Entpacken des Archivs fehlgeschlagen bei Datei /usr/bin/swtpm;624dcb38: cpio: (Fehler 0x2)
Fehler: swtpm-0.7.2-1.20220307git21c90c1.fc36.x86_64: installieren fehlgeschlagen
Fehler: lsetfilecon: (/etc/sysconfig/snapd;624dcb38, system_u:object_r:snappy_config_t:s0) Das Argument ist ungültig
Fehler: Plugin selinux: hook fsm_file_prepare failed

Error unpacking rpm package snapd-2.54.4-1.fc36.x86_64
  Ausgeführtes Scriptlet: flatpak-1.12.7-1.fc36.x86_64                                                       7/14 
Fehler: Entpacken des Archivs fehlgeschlagen bei Datei /etc/sysconfig/snapd;624dcb38: cpio: (Fehler 0x2)
Fehler: snapd-2.54.4-1.fc36.x86_64: installieren fehlgeschlagen

  Aktualisieren         : flatpak-1.12.7-1.fc36.x86_64                                                       7/14 
Fehler: lsetfilecon: (/usr/libexec/flatpak-system-helper;624dcb38, system_u:object_r:flatpak_helper_exec_t:s0) Das Argument ist ungültig
Fehler: Plugin selinux: hook fsm_file_prepare failed

Error unpacking rpm package flatpak-1.12.7-1.fc36.x86_64
  Überprüfung läuft     : conmon-2:2.1.0-2.fc36.x86_64                                                       1/14 
  Überprüfung läuft     : conmon-2:2.1.0-2.fc35.x86_64                                                       2/14 
  Überprüfung läuft     : containers-common-4:1-53.fc36.noarch                                               3/14 
  Überprüfung läuft     : containers-common-4:1-45.fc35.noarch                                               4/14 
  Überprüfung läuft     : crun-1.4.4-1.fc36.x86_64                                                           5/14 
  Überprüfung läuft     : crun-1.4.4-1.fc35.x86_64                                                           6/14 
  Überprüfung läuft     : flatpak-1.12.7-1.fc36.x86_64                                                       7/14 
  Überprüfung läuft     : flatpak-1.12.7-1.fc35.x86_64                                                       8/14 
  Überprüfung läuft     : snapd-2.54.4-1.fc36.x86_64                                                         9/14 
  Überprüfung läuft     : snapd-2.54.4-1.fc35.x86_64                                                        10/14 
  Überprüfung läuft     : swtpm-0.7.2-1.20220307git21c90c1.fc36.x86_64                                      11/14 
  Überprüfung läuft     : swtpm-0.7.2-1.20220307git21c90c1.fc35.x86_64                                      12/14 
  Überprüfung läuft     : podman-3:4.0.3-1.fc36.x86_64                                                      13/14 
  Überprüfung läuft     : podman-3:3.4.4-1.fc35.x86_64                                                      14/14 

Fehlgeschlagen:
  conmon-2:2.1.0-2.fc35.x86_64                            conmon-2:2.1.0-2.fc36.x86_64                           
  containers-common-4:1-45.fc35.noarch                    containers-common-4:1-53.fc36.noarch                   
  crun-1.4.4-1.fc35.x86_64                                crun-1.4.4-1.fc36.x86_64                               
  flatpak-1.12.7-1.fc35.x86_64                            flatpak-1.12.7-1.fc36.x86_64                           
  podman-3:3.4.4-1.fc35.x86_64                            podman-3:4.0.3-1.fc36.x86_64                           
  snapd-2.54.4-1.fc35.x86_64                              snapd-2.54.4-1.fc36.x86_64                             
  swtpm-0.7.2-1.20220307git21c90c1.fc35.x86_64            swtpm-0.7.2-1.20220307git21c90c1.fc36.x86_64           

Fehler: Transaktion fehlgeschlagen
flatpak-1.12.7-1.fc35.x86_64 hat fehlende Abhängigkeiten von (flatpak-selinux = 1.12.7-1.fc35 if selinux-policy-targeted)
flatpak-1.12.7-1.fc35.x86_64 hat fehlende Abhängigkeiten von flatpak-session-helper(x86-64) = 1.12.7-1.fc35
podman-3:3.4.4-1.fc35.x86_64 hat fehlende Abhängigkeiten von libsubid.so.3()(64bit)
snapd-2.54.4-1.fc35.x86_64 hat fehlende Abhängigkeiten von snap-confine(x86-64) = 2.54.4-1.fc35
snapd-2.54.4-1.fc35.x86_64 hat fehlende Abhängigkeiten von snapd-selinux = 2.54.4-1.fc35
swtpm-0.7.2-1.20220307git21c90c1.fc35.x86_64 hat fehlende Abhängigkeiten von swtpm-libs = 0.7.2-1.20220307git21c90c1.fc35
swtpm-tools-0.7.2-1.20220307git21c90c1.fc36.x86_64 hat fehlende Abhängigkeiten von swtpm = 0.7.2-1.20220307git21c90c1.fc36
Fehler: Check discovered 7 problem(s)
Letzte Prüfung auf abgelaufene Metadaten: vor 0:00:32 am Mi 06 Apr 2022 19:17:15 CEST.
Abhängigkeiten sind aufgelöst.
==================================================================================================================
 Paket                               Arch.     Version                                  Paketquelle         Größe
==================================================================================================================
Entfernen:
 bluez-obexd                         x86_64    5.64-1.fc36                              @fedora             622 k
 colord-gtk                          x86_64    0.3.0-1.fc36                             @fedora              74 k
 f35-backgrounds-base                noarch    35.0.1-3.fc36                            @fedora              20 M
 f35-backgrounds-gnome               noarch    35.0.1-3.fc36                            @fedora             925  
 flatpak-selinux                     noarch    1.12.7-1.fc36                            @fedora              12 k
 flatpak-session-helper              x86_64    1.12.7-1.fc36                            @fedora             104 k
 gnome-shell-extension-user-theme    noarch    42.0-1.fc36                              @fedora             6.9 k
 iptables-legacy-libs                x86_64    1.8.7-15.fc36                            @fedora              91 k
 java-11-openjdk-headless            x86_64    1:11.0.14.1.1-5.fc36                     @fedora             177 M
 libtpms                             x86_64    0.9.3-1.20220307gita63c51805e.fc36       @fedora             971 k
 lilv                                x86_64    0.24.12-4.fc36                           @fedora             102 k
 llvm-libs                           x86_64    14.0.0-1.fc36                            @updates-testing    104 M
 lv2                                 x86_64    1.18.2-2.fc36                            @fedora             399 k
 mozjs78                             x86_64    78.15.0-3.fc36                           @fedora              28 M
 neon                                x86_64    0.32.2-4.fc36                            @fedora             331 k
 pakchois                            x86_64    0.4-25.fc36                              @fedora              29 k
 podman-gvproxy                      x86_64    3:4.0.3-1.fc36                           @updates-testing     11 M
 podman-plugins                      x86_64    3:4.0.3-1.fc36                           @updates-testing    3.2 M
 radvd                               x86_64    2.19-5.fc36                              @fedora             170 k
 shadow-utils-subid                  x86_64    2:4.11.1-2.fc36                          @fedora              55 k
 snap-confine                        x86_64    2.54.4-1.fc36                            @fedora             8.1 M
 snapd-selinux                       noarch    2.54.4-1.fc36                            @fedora              44 k
 swtpm                               x86_64    0.7.2-1.20220307git21c90c1.fc35          @updates            218 k
 swtpm-libs                          x86_64    0.7.2-1.20220307git21c90c1.fc36          @fedora              99 k

Transaktionszusammenfassung
==================================================================================================================
Entfernen  24 Pakete

While most of the packages mentioned here are in the default Workstation installation of F35/F36, I wasn't able to replicate this issue just by installing+upgrading. So there needs to some other trigger, than just having those packages on the system.

I'm very concerned about how many people are affected by this issue. While our release criteria allow us to block the release only on issues affecting a default system install, I can at least propose this for a freeze exception, if a fix is found. I'm also proposing this a Prioritized Bug, due to the impact and numerous reports.

Zdenek, if this is not going to be fixed before F36 Final release, can you please look at this Common Issue proposal [1], read the description and especially the Workarounds section, and tell me whether the description is correct or something needs to be changed? Or write the best workaround here from scratch? We need to inform people what to do if they hit this situation. Thanks!

[1] https://ask.fedoraproject.org/t/dnf-upgrade-of-some-packages-fail-after-upgrade-from-f35/20983

Just a quick question - has it by now been confirmed that the issue is caused by snapd? I see that all reports (including my 2 machines which failed to upgrade) had snapd installed. I can try to run an upgrade on a clean installed F35 VM + snapd added later today, just wanted to check weather someone already tried it.

I believe I've managed to find the root cause: some security classes were removed in rawhide selinux-policy, Thu Feb 03 2022.
Problems occur when selinux-policy is updated to its F36 version before e. g. container-selinux was updated. The update leaves the selinux store in an undefined state which I think is a serious problem which should be fixed before GA. It does not affect new installations.

I am looking for all packages which can hit this problem, so far I am aware of container, flatpak, osbuild.
I will request a rebuild which should be sufficient.

`sudo dnf remove snapd` solved the problem for me

> The update leaves the selinux store in an undefined state which I think is a serious problem which should be fixed before GA.

Proposing as a blocker, then. The blocker discussion can be found at https://pagure.io/fedora-qa/blocker-review/issue/756

Hard to imagine how this issue would not preclude a Final Target Date, unless an installed-base is no matter.

Discussed in ticket: https://pagure.io/fedora-qa/blocker-review/issue/756

The decision to classify this bug as an AcceptedBlocker was made:

"This issue violates "The upgraded system must meet all release criteria." criterion as the upgraded system isn't able to fulfill "The installed system must be able appropriately to install, remove, and update software with the default tool for the relevant software type...""

I can confirm I had/have snapd installed.

Also the latest dnf update has completely destroyed my machine's ability to get to a login screen. A bunch of network manager/dbus errors. Not sure if its related. 

At this point I'm wondering what I should be doing to fix it.

(In reply to Nathanael Noblet from comment #37)
> I can confirm I had/have snapd installed.
> 
> Also the latest dnf update has completely destroyed my machine's ability to
> get to a login screen. A bunch of network manager/dbus errors. Not sure if
> its related. 
> 
> At this point I'm wondering what I should be doing to fix it.

The same thing happened to me on one machine after I got the error the 1st time (comment #0). After 1-2 updates the system refused to get into login screen and was reporting dbus errors. I was not sure whether I broke it whle trying to fix the original issue, so I did not report it. I tried to fix it by chroot-ing into the system and reinstalling some components but in the end I gave up and clean installed F36.

I just managed to fix my 2nd machine which did not get logging issues by running "sudo semodule -X 200 -r snappy -r container -r flatpak -X 400 -r pcpupstream -r pcpupstream-container -X 100 -r pcp", preventively running "sudo dnf update", reinstalling snapd and reinstalling container-sellinux "dnf install -y container-selinux".

(In reply to Nathanael Noblet from comment #37)
> I can confirm I had/have snapd installed.
> 
> Also the latest dnf update has completely destroyed my machine's ability to
> get to a login screen. A bunch of network manager/dbus errors. Not sure if
> its related. 
> 
> At this point I'm wondering what I should be doing to fix it.

Boot with option `enforcing=0`:

Add 
enforcing=0
to the boot parameters, then you will be able to boot.

selinux-policy for F34 and F35 has been updated not to include socket classes removed in later releases:

selinux-policy-35.17-1.fc35
https://bodhi.fedoraproject.org/updates/FEDORA-2022-c5bee6b70f

selinux-policy-34.27-1.fc34
https://bodhi.fedoraproject.org/updates/FEDORA-2022-eaef082697

So far, I've managed to find 4 packages which depend on the update, there are these bz requests to rebuild the additional packages:

https://bugzilla.redhat.com/show_bug.cgi?id=2070764 container
https://bugzilla.redhat.com/show_bug.cgi?id=2071206 osbuild
https://bugzilla.redhat.com/show_bug.cgi?id=2070729 snapd
https://bugzilla.redhat.com/show_bug.cgi?id=2075651 flatpak

Keeping this bz open until these bugs are resolved, too.

Thank you everyone for the advice on how to get the system functioning again (I haven't tried it yet but I appreciate the guidance).

I originally didn't follow any of the workarounds detailed here as I wasn't sure if there was going to be an update that would 'rectify' the problem and testing would be necessary to make sure the fix was in. However I'm wondering at this point, is this a bug that will be fixed by other packages being fixed and in the released state so that others don't hit this issue? Or will there be package updates that if I ran dnf update would suddenly fix the problem and its worthwhile to the community to have someone whose system is in my state to install them to confirm the fix?

I have submitted a build of flatpak against the new selinux-policy, in side tag f35-build-side-52906. The rebuilds should not be done piecemeal, but should be coordinated, and bundled into a single update, as per https://docs.fedoraproject.org/en-US/fesco/Updates_Policy/#updating-inter-dependent-packages

(In reply to Zdenek Pytela from comment #40)
> selinux-policy for F34 and F35 has been updated not to include socket
> classes removed in later releases:
>…
> So far, I've managed to find 4 packages which depend on the update, there
> are these bz requests to rebuild the additional packages:
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=2070764 container
> https://bugzilla.redhat.com/show_bug.cgi?id=2071206 osbuild
> https://bugzilla.redhat.com/show_bug.cgi?id=2070729 snapd
> https://bugzilla.redhat.com/show_bug.cgi?id=2075651 flatpak
> 
> Keeping this bz open until these bugs are resolved, too.

I also see some discussion of this point (about build overrides) in bug 2071206. Please coordinate rebuilds so that they all happen in one side tag (feel free to use the side tag that I created, if you like), and that the selinux-policy bodhi update includes not only selinux-policy but all the builds of dependent packages. I am a provenpackager, and would be happy to help over the weekend if that would be useful.

FEDORA-2022-c5bee6b70f has been submitted as an update to Fedora 35. https://bodhi.fedoraproject.org/updates/FEDORA-2022-c5bee6b70f

I edited the rebuilds into the F35 selinux-policy update, and I will do the F34 rebuilds later today.

FEDORA-2022-c5bee6b70f has been pushed to the Fedora 35 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2022-c5bee6b70f`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-c5bee6b70f

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

*** Bug 2075997 has been marked as a duplicate of this bug. ***

Still broken.

I managed to update F35->F36 successfully today using packages from the updates-testing repo, all modules seem to be active and working.

$ rpm -q selinux-policy snapd-selinux container-selinux flatpak-selinux osbuild-selinux
selinux-policy-36.6-1.fc36.noarch
snapd-selinux-2.54.4-1.fc36.noarch
container-selinux-2.181.0-2.fc36.noarch
flatpak-selinux-1.12.7-2.fc36.noarch
osbuild-selinux-53-1.fc36.noarch

(In reply to brian connolly from comment #47)
> Still broken.

Can you describe what is not working for you?

I have broken system too.

1. Now I can't login without disabling selinux.
2. Can't install any F36 kernel. It just not appear to /boot partition.

How to solve it?

(In reply to Zdenek Pytela from comment #49)
> (In reply to brian connolly from comment #47)
> > Still broken.
> 
> Can you describe what is not working for you?

My bad.  Senior moment. I had not used the updates-testing repo.

(In reply to brian connolly from comment #51)
> My bad.  Senior moment. I had not used the updates-testing repo.

Brian,

It seems to be quite important to check with the testing repo if the update goes well now, please do so and report any outstanding problem. I haven't found any so far, but other installations may get to different experience.

(In reply to Vasiliy Glazov from comment #50)
> I have broken system too.
> 
> 1. Now I can't login without disabling selinux.
> 2. Can't install any F36 kernel. It just not appear to /boot partition.
> 
> How to solve it?

These issues do not seem to be related to this bz, so open new ones, one bz for each, and describe the problems, or use fedora mailing lists to discuss.

This is direct consequence of this bug.
I just made selinux autorelabel on /

Removing the prioritized bug nomination since this is an accepted F36 release blocker.

(In reply to Zdenek Pytela from comment #52)
> (In reply to brian connolly from comment #51)
> > My bad.  Senior moment. I had not used the updates-testing repo.
> 
> Brian,
> 
> It seems to be quite important to check with the testing repo if the update
> goes well now, please do so and report any outstanding problem. I haven't
> found any so far, but other installations may get to different experience.

Worked... sorta.  

- ran the fix in comment 45 under Fedora 35 and rebooted
- removed akmod-nvidia
- ran the upgrade
- booted into nouveau
- ran dnf upgrade and upgraded completely
- rebooted
- reinstalled akmod-nvidia
- rebooted, announced prerelease 36 kernel, and then got a black screen

Seems to have fixed the issues in this thread, only to uncover another.

(In reply to brian connolly from comment #56)
> - rebooted, announced prerelease 36 kernel, and then got a black screen
> 
> Seems to have fixed the issues in this thread, only to uncover another.

If you start with Grub2-Menu press "e" (for edit) and than add " enforcing=0" into the starting command, to disable selinux, which would block starting (DBUs, Network,...) and ending up with a black screen.

This should fix your issue.

Created attachment 1873565 [details]
how to fix a blank screen on startup with "enforcing=0"

First press e for edit, if you see the Grub2-Menu.

Same story here up to podman reinstall. Podman upgraded up to 4.0.3 but doesn't work - can't read container from the rootless user storage $HOME/.local/share/containers/storage/... i.e. from the same place where flatpack stores its data
$HOME is btrfs volume and mounted timely by systemd unit !!!!
SELinux monitor contains random error messages like:

SELinux is preventing gnome-shell from map access on the file /var/lib/flatpak/exports/share/icons/hicolor/icon-theme.cache.

*****  Plugin catchall_boolean (57.6 confidence) suggests   ******************

If you want to allow any process to mmap any file on system with attribute file_type.
Then you must tell SELinux about this by enabling the 'domain_can_mmap_files' boolean.

Do
setsebool -P domain_can_mmap_files 1

*****  Plugin catchall_labels (36.2 confidence) suggests   *******************

If you want to allow gnome-shell to have map access on the icon-theme.cache file
Then you need to change the label on /var/lib/flatpak/exports/share/icons/hicolor/icon-theme.cache
Do ...
Tried to relabel several times but stuck at the same place.
I also got Installed package container-selinux-2:2.181.0-1.fc36.noarch (from updates-testing) not available ?????

(In reply to brian connolly from comment #56)
> Worked... sorta.  
> 
> - ran the fix in comment 45 under Fedora 35 and rebooted
> - removed akmod-nvidia
> - ran the upgrade
> - booted into nouveau
> - ran dnf upgrade and upgraded completely
> - rebooted
> - reinstalled akmod-nvidia
> - rebooted, announced prerelease 36 kernel, and then got a black screen
> 
> Seems to have fixed the issues in this thread, only to uncover another.

Once you manage to log in, please add some data to investigate on, possibly create a new bz.

Some stuff in update-testing for 36 WS beta is too old. (nss-mdns) - version with corrected bug
I will check if libseccomp-2.5.3-2.fc36.x86_64 is OK. It looks like it has update for the new Kernel 5.17. It can be plain DevOps issue.

So this bug seems a bit confused now. Zdenek, is the issue you considered to be a serious one now definitely resolved? Everyone, is there any remaining clearly reproducible bug here which should be considered release-blocking?

Adam, see comment 40. I believe the remaining update to be pushed is this one:
https://bodhi.fedoraproject.org/updates/FEDORA-2022-c5bee6b70f

Note that that's for F35, so doesn't block F36 RC compose.

I'll try to test upgrade and look into upgrade logs, whether I can see some selinux-related problems even in other packages.

Created attachment 1874393 [details]
system-upgrade journal

I performed system upgrade from F35 with updates-testing enabled (therefore including https://bodhi.fedoraproject.org/updates/FEDORA-2022-c5bee6b70f ) to F36. The full upgrade journal is attached. It contains quite a lot of AVCs:

$ grep avc journal.txt | cut -d ' ' -f 5- | uniq

audit[4687]: AVC avc:  denied  { mac_admin } for  pid=4687 comm="restorecon" capability=33  scontext=system_u:system_r:setfiles_t:s0 tcontext=system_u:system_r:setfiles_t:s0 tclass=capability2 permissive=0
kernel: audit: type=1400 audit(1650629631.469:209): avc:  denied  { mac_admin } for  pid=4687 comm="restorecon" capability=33  scontext=system_u:system_r:setfiles_t:s0 tcontext=system_u:system_r:setfiles_t:s0 tclass=capability2 permissive=0
audit[4687]: AVC avc:  denied  { mac_admin } for  pid=4687 comm="restorecon" capability=33  scontext=system_u:system_r:setfiles_t:s0 tcontext=system_u:system_r:setfiles_t:s0 tclass=capability2 permissive=0
kernel: audit: type=1400 audit(1650629631.583:210): avc:  denied  { mac_admin } for  pid=4687 comm="restorecon" capability=33  scontext=system_u:system_r:setfiles_t:s0 tcontext=system_u:system_r:setfiles_t:s0 tclass=capability2 permissive=0
audit[4687]: AVC avc:  denied  { mac_admin } for  pid=4687 comm="restorecon" capability=33  scontext=system_u:system_r:setfiles_t:s0 tcontext=system_u:system_r:setfiles_t:s0 tclass=capability2 permissive=0
kernel: audit: type=1400 audit(1650629638.202:217): avc:  denied  { mac_admin } for  pid=4687 comm="restorecon" capability=33  scontext=system_u:system_r:setfiles_t:s0 tcontext=system_u:system_r:setfiles_t:s0 tclass=capability2 permissive=0
audit[4687]: AVC avc:  denied  { mac_admin } for  pid=4687 comm="restorecon" capability=33  scontext=system_u:system_r:setfiles_t:s0 tcontext=system_u:system_r:setfiles_t:s0 tclass=capability2 permissive=0
kernel: audit: type=1400 audit(1650629638.203:218): avc:  denied  { mac_admin } for  pid=4687 comm="restorecon" capability=33  scontext=system_u:system_r:setfiles_t:s0 tcontext=system_u:system_r:setfiles_t:s0 tclass=capability2 permissive=0
kernel: audit: type=1400 audit(1650629638.203:219): avc:  denied  { mac_admin } for  pid=4687 comm="restorecon" capability=33  scontext=system_u:system_r:setfiles_t:s0 tcontext=system_u:system_r:setfiles_t:s0 tclass=capability2 permissive=0
audit[4687]: AVC avc:  denied  { mac_admin } for  pid=4687 comm="restorecon" capability=33  scontext=system_u:system_r:setfiles_t:s0 tcontext=system_u:system_r:setfiles_t:s0 tclass=capability2 permissive=0
kernel: audit: type=1400 audit(1650629663.310:337): avc:  denied  { mac_admin } for  pid=4687 comm="restorecon" capability=33  scontext=system_u:system_r:setfiles_t:s0 tcontext=system_u:system_r:setfiles_t:s0 tclass=capability2 permissive=0
audit[4687]: AVC avc:  denied  { mac_admin } for  pid=4687 comm="restorecon" capability=33  scontext=system_u:system_r:setfiles_t:s0 tcontext=system_u:system_r:setfiles_t:s0 tclass=capability2 permissive=0
kernel: audit: type=1400 audit(1650629663.311:338): avc:  denied  { mac_admin } for  pid=4687 comm="restorecon" capability=33  scontext=system_u:system_r:setfiles_t:s0 tcontext=system_u:system_r:setfiles_t:s0 tclass=capability2 permissive=0
kernel: audit: type=1400 audit(1650629663.311:339): avc:  denied  { mac_admin } for  pid=4687 comm="restorecon" capability=33  scontext=system_u:system_r:setfiles_t:s0 tcontext=system_u:system_r:setfiles_t:s0 tclass=capability2 permissive=0
audit[4687]: AVC avc:  denied  { mac_admin } for  pid=4687 comm="restorecon" capability=33  scontext=system_u:system_r:setfiles_t:s0 tcontext=system_u:system_r:setfiles_t:s0 tclass=capability2 permissive=0
dnf[678]: uavc:  op=load_policy lsm=selinux seqno=2 res=1  Upgrading        : PackageKit-glib-1.2.5-1.fc36.x86_64               746/3508
audit[2462]: USER_MAC_POLICY_LOAD pid=2462 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc:  op=load_policy lsm=selinux seqno=2 res=1 exe=2F7573722F62696E2F646275732D62726F6B6572202864656C6574656429 sauid=81 hostname=? addr=? terminal=?'
dnf[678]:   Running scriptlet: container-selinux-2:2.183.0-1.fc36.noarch         835/3508uavc:  op=load_policy lsm=selinux seqno=3 res=1
audit[2462]: USER_MAC_POLICY_LOAD pid=2462 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc:  op=load_policy lsm=selinux seqno=3 res=1 exe=2F7573722F62696E2F646275732D62726F6B6572202864656C6574656429 sauid=81 hostname=? addr=? terminal=?'
kernel: audit: type=2310 audit(1650629690.557:387): pid=2462 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc:  op=load_policy lsm=selinux seqno=3 res=1 exe=2F7573722F62696E2F646275732D62726F6B6572202864656C6574656429 sauid=81 hostname=? addr=? terminal=?'
dnf[678]:   Running scriptlet: snapd-selinux-2.55.3-1.fc36.noarch               1304/3508uavc:  op=load_policy lsm=selinux seqno=4 res=1
dnf[678]:   Running scriptlet: flatpak-selinux-1.12.7-2.fc36.noarch             1305/3508uavc:  op=load_policy lsm=selinux seqno=5 res=1
dnf[678]:   Running scriptlet: swtpm-0.7.2-1.20220307git21c90c1.fc36.x86_64     1306/3508uavc:  op=load_policy lsm=selinux seqno=7 res=1
audit[2462]: USER_MAC_POLICY_LOAD pid=2462 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc:  op=load_policy lsm=selinux seqno=7 res=1 exe=2F7573722F62696E2F646275732D62726F6B6572202864656C6574656429 sauid=81 hostname=? addr=? terminal=?'
kernel: audit: type=2310 audit(1650629803.489:474): pid=2462 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc:  op=load_policy lsm=selinux seqno=7 res=1 exe=2F7573722F62696E2F646275732D62726F6B6572202864656C6574656429 sauid=81 hostname=? addr=? terminal=?'
dnf[678]:   Running scriptlet: osbuild-selinux-54-1.fc36.noarch                 1359/3508uavc:  op=load_policy lsm=selinux seqno=8 res=1
audit[2462]: USER_MAC_POLICY_LOAD pid=2462 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc:  op=load_policy lsm=selinux seqno=8 res=1 exe=2F7573722F62696E2F646275732D62726F6B6572202864656C6574656429 sauid=81 hostname=? addr=? terminal=?'
kernel: audit: type=2310 audit(1650629816.581:476): pid=2462 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc:  op=load_policy lsm=selinux seqno=8 res=1 exe=2F7573722F62696E2F646275732D62726F6B6572202864656C6574656429 sauid=81 hostname=? addr=? terminal=?'

Zdenek, do those look harmless or do we need to fix more F35 packages?

I don't know, it is needed to pair the mac_admin records with other types records with full auditing enabled. As I haven't seen this before, I will try to update a F35 system with packages set like yours.

What I am a bit worried though is this:

Apr 22 14:22:56 f35 dnf[678]: Downgraded:
Apr 22 14:22:56 f35 dnf[678]:   osbuild-54-1.fc36.noarch              osbuild-selinux-54-1.fc36.noarch
Apr 22 14:22:56 f35 dnf[678]:   python3-osbuild-54-1.fc36.noarch      snap-confine-2.55.3-1.fc36.x86_64
Apr 22 14:22:56 f35 systemd[1]: Stopped alsa-state.service - Manage Sound Card State (restore and store).
Apr 22 14:22:56 f35 dnf[678]:   snapd-2.55.3-1.fc36.x86_64            snapd-selinux-2.55.3-1.fc36.noarch
Apr 22 14:22:56 f35 dnf[678]:   tzdata-2022a-1.fc36.noarch            tzdata-java-2022a-1.fc36.noarch
Apr 22 14:22:56 f35 dnf[678]:   vim-data-2:8.2.4621-1.fc36.noarch     vim-minimal-2:8.2.4621-1.fc36.x86_64

Packages getting downgraded during a system upgrade is fairly common during freezes, because if the same version is sent to both F35 and F36, the F35 update may go stable while the F36 one gets stuck in the freeze. The 0-day stable push should clean up most such cases.

Assuming you're specifically concerned about snapd and snapd-selinux, that is indeed the case here - the F36 snapd update is still in updates-testing: https://bodhi.fedoraproject.org/updates/FEDORA-2022-2393f375a0 . Is that package implicated in this bug? i.e. do we need to push that F36 package stable to consider this bug resolved?

I still cannot see any actual problems with F35->F36 updates. I can confirm the denials reported by Kamil, will check it further. If any other issue appears, a new bz should be filed.

I currently don't think any other F36 update is needed but selinux-policy, there is now one build in testing and will be another one soon.

OK. We have run a release candidate compose that includes the selinux-policy update, but not the snapd one. I guess we'll push the selinux-policy update stable soon and then we'll have to see if there are still any clear reproducible bugs on upgrade after that and deal with them as they come up.

I reinstalled and updated libseccomp and the situation slightly improved: now podman fails because syscall setxattr refused when issued by rootless user and applied to the file in $HOME/.local/share/containers ... The files and directories under this point are labeled as unconfined_u:object_r:data_home_t:s0. I'm surprised that user can't set file attr inside own home area even running unprivileged process.

(In reply to Pavel Sosin from comment #69)
> I reinstalled and updated libseccomp and the situation slightly improved:
> now podman fails because syscall setxattr refused when issued by rootless
> user and applied to the file in $HOME/.local/share/containers ... The files
> and directories under this point are labeled as
> unconfined_u:object_r:data_home_t:s0. I'm surprised that user can't set file
> attr inside own home area even running unprivileged process.

Some if the stuff in ~/.local/share/containers is associated with with a uid/gid other than the uid/gid of the user. A rm -rf ~/.local/share/container usually also does not work (even in selinux  permissive mode) due to this.
Everything has to go through podman

Arguably a bug in Podman, because a user should be able to rm -rf ~/.local/share/containers, at least if no containeres are running.

(In reply to Kamil Páral from comment #63)
> I believe the remaining update to be pushed is this one:
> https://bodhi.fedoraproject.org/updates/FEDORA-2022-c5bee6b70f

Zdenek, can you please submit this stable? If this is not stable *long* before F36 is released, I'm afraid this might have disastrous consequences, because many people will start upgrading to F36 without having this installed.

(In reply to Kamil Páral from comment #71)
> Zdenek, can you please submit this stable? If this is not stable *long*
> before F36 is released, I'm afraid this might have disastrous consequences,
> because many people will start upgrading to F36 without having this
> installed.

The update is in this state since 2 days ago, I don't know why, but anyway I cannot push it.

dac.override The rm -rf failure is caused by user namespace.  Try `podman unshare rm -rf $HOME/.local/share/containers`

FEDORA-2022-c5bee6b70f has been pushed to the Fedora 35 stable repository.
If problem still persists, please make note of it in this bug report.

I afraid do make overall dnf update because Fedora WS is one of my main horses
But I can CRUD of files and directories manually as rootless user under $HOME/.local. In Podman the situation is different: it forks CRUN i.e. process context may switch. CRun label looks OK but is it enough?
Basic test bash crun --version & --> /usr/bin/crun: /usr/bin/crun: cannot execute binary file. So, Gnome-shell and podman inheriting its context can't run crun. It's exactly what I see when running container using podman.
Both podman and Crun are labeled as container_runtime, recursively.Strange.I expected that podman will be a regular exec on WS.

For the record, the update was not pushable because there was a gating test failure. It seems that snapd has a test suite configured in CI that always fails, and it was being considered a required test for some reason - I'm not sure why, as snapd has no gating.yml. Anyway, I just re-triggered the tests; that test failed again, but Greenwave no longer figured it as a required test, so gating passed and I could submit the update stable.

Adam,

Thanks for looking into it and resolving the problem.
Note this can also possibly be an issue in F34 batch:
https://bodhi.fedoraproject.org/updates/FEDORA-2022-eaef082697

I've re-submitted the tests for that update too. Note, you can do this as well, there's a button on the right-hand side of the page for doing it (if you have the power to edit the update).

Had this problem after trying to prematurely upgrade from F35 to F36 (release date a couple days away, should be safe, right? LOL) and finally was able to fix it (apparently) with this: 


sudo semodule -lfull | grep -v 100

200 container         pp          
200 flatpak           pp          
200 snappy            pp          
200 swtpm             pp          
200 swtpm_svirt       pp

sudo semodule -X 200 -r container -r flatpak -r snappy -r swtpm -r swtpm_svirt

(Adjust the module removal line according to whatever modules you happen to have installed.) 

sudo dnf reinstall container-selinux

sudo dnf update

sudo dnf install snapd flatpak

(Snap still broken with "too early for operation" error.)

sudo dnf reinstall snapd flatpak

(Seems to have fixed Snap, and was able to "sudo snap install btop".)

As noted somewhere above, trying to remove one module at a time kept failing with some sort of "AST" error. Putting everything on one line worked fine, then I've been able to reinstall Snap and there are no more errors installing packages after "sudo dnf update". 

/snap/bin was not in $PATH, but opening a new terminal fixed that.

Sorry for the trouble, RedBear! Can you check if you had the updates from https://bodhi.fedoraproject.org/updates/FEDORA-2022-c5bee6b70f installed before you upgraded?

(In reply to Adam Williamson from comment #80)
> Sorry for the trouble, RedBear! Can you check if you had the updates from
> https://bodhi.fedoraproject.org/updates/FEDORA-2022-c5bee6b70f installed
> before you upgraded?

Don't think so. 

I upgraded from F34 to F35 a couple of days ago, which went fine. Actually I did that by just pushing the button in the Software app. But after rebooting and seeing that everything was working, I followed some instructions somewhere online in the terminal to make it pull in F36 even though it hadn't released yet, which failed at first until I did the "--allowerasing" just like in the original post here. Pretty much this same procedure from the original post: 

How reproducible:
Upgrade from F35 to F36 using:
1. sudo dnf upgrade --refresh
2.  sudo dnf system-upgrade download --release=36
3.  sudo dnf system-upgrade download --release=36 --allowerasing
4.  sudo dnf system-upgrade reboot

This was a couple of days ago, if I remember right. Monday evening. 

I knew nothing at the time about this issue and so did nothing special to prevent it. I think I did the upgrade just before the patch was pushed to stable, if I'm reading the linked page correctly. Having no understanding of any of this, I think I tried to apply that patch after the fact, and enabled the "testing" repos, but was unable to fix the issue until I did the specific commands in my previous post, gleaned from posts further up in the thread. 

After rebooting and trying another "dnf update" everything seems to be OK at this point. 

It is rather distressing that something this major was allowed to still be an issue within a couple of days of what was supposed to be the final release date for F36. If I hadn't been able to figure this out by today I think I would have just wiped the drive and installed Ubuntu 22.04, which has been running with no issue on a couple other machines for at least a couple of weeks before its official release date. Without some of the instructions in this thread, found by googling the specific errors, I would have had NO IDEA how to fix any of this. 

I thought Rawhide was supposed to be the bleeding edge unstable Fedora where things like this might happen. 

A fresh F36 beta install on a different machine a few days earlier of course went perfectly fine.

(In reply to Adam Williamson from comment #80)
> Sorry for the trouble, RedBear! Can you check if you had the updates from
> https://bodhi.fedoraproject.org/updates/FEDORA-2022-c5bee6b70f installed
> before you upgraded?

Fwiw, it doesn't seem to be enough. Even with this update installed, I still see AST errors in the upgrade log when upgrading to F36. I *think* it might be because https://bodhi.fedoraproject.org/updates/FEDORA-2022-76963fee71 is still not in stable repos (despite what Bodhi says), because F36 composes are not happening/being stuck. I'll try an upgrade today with a side repo which will contain selinux-policy-36.7-1.fc36 so that I can confirm whether the AST and other selinux errors are gone or not.

Created attachment 1875586 [details]
upgrade journal with all current updates

This is the upgrade journal when I have fully updated F35 Workstation (containing selinux-policy-35.17-1.fc35) and upgrade to F36 *including* a side repo which contains selinux-policy-36.7-1.fc36 (currently waiting for a compose to push it stable). So this upgrade is the best possible outcome that we can achieve today.

I still see very concerning messages in the upgrade journal, see below. To my layman eye, it seems that things break when selinux-policy-targeted-36.7-1.fc36 gets updated ("Context XXX became invalid (unmapped)" messages), and get fixed when container-selinux-2:2.181.0-2.fc36 gets updated ("Context XXX became valid (mapped)" messages). Notice that the "Failed to resolve AST" error is still present as well in this log.

Zdenek, can you please tell us whether the upgrade messages look harmless or still problematic?

Snippets:

Apr 28 08:58:31 f35 dnf[680]:   Upgrading        : selinux-policy-36.7-1.fc36.noarch                 748/3503
Apr 28 08:58:39 f35 dnf[680]:   Running scriptlet: selinux-policy-36.7-1.fc36.noarch                 748/3503
Apr 28 08:58:39 f35 dnf[680]:   Running scriptlet: selinux-policy-targeted-36.7-1.fc36.noarch        749/3503
Apr 28 08:58:39 f35 dnf[680]:   Upgrading        : selinux-policy-targeted-36.7-1.fc36.noarch        749/3503
Apr 28 08:58:43 f35 kernel: SELinux:  Converting 449 SID table entries...
Apr 28 08:58:43 f35 kernel: SELinux:  Context system_u:object_r:snappy_unit_file_t:s0 became invalid (unmapped).
Apr 28 08:58:43 f35 kernel: SELinux:  Context system_u:object_r:container_var_lib_t:s0 became invalid (unmapped).
Apr 28 08:58:43 f35 kernel: SELinux:  Context system_u:object_r:snappy_var_lib_t:s0 became invalid (unmapped).
Apr 28 08:58:43 f35 kernel: SELinux:  Context system_u:object_r:snappy_config_t:s0 became invalid (unmapped).
Apr 28 08:58:43 f35 kernel: SELinux:  Context system_u:object_r:snappy_confine_exec_t:s0 became invalid (unmapped).
Apr 28 08:58:43 f35 kernel: SELinux:  Context system_u:object_r:snappy_exec_t:s0 became invalid (unmapped).
Apr 28 08:58:43 f35 kernel: SELinux:  Context system_u:object_r:snappy_mount_exec_t:s0 became invalid (unmapped).
Apr 28 08:58:43 f35 kernel: SELinux:  policy capability network_peer_controls=1
Apr 28 08:58:43 f35 kernel: SELinux:  policy capability open_perms=1
Apr 28 08:58:43 f35 kernel: SELinux:  policy capability extended_socket_class=1
Apr 28 08:58:43 f35 kernel: SELinux:  policy capability always_check_network=0
Apr 28 08:58:43 f35 kernel: SELinux:  policy capability cgroup_seclabel=1
Apr 28 08:58:43 f35 kernel: SELinux:  policy capability nnp_nosuid_transition=1
Apr 28 08:58:43 f35 kernel: SELinux:  policy capability genfs_seclabel_symlinks=1
Apr 28 08:58:43 f35 kernel: SELinux:  policy capability ioctl_skip_cloexec=0
Apr 28 08:58:43 f35 audit: MAC_POLICY_LOAD auid=4294967295 ses=4294967295 lsm=selinux res=1
Apr 28 08:58:43 f35 kernel: kauditd_printk_skb: 47 callbacks suppressed
Apr 28 08:58:43 f35 kernel: audit: type=1403 audit(1651129123.001:207): auid=4294967295 ses=4294967295 lsm=selinux res=1
Apr 28 08:58:43 f35 audit[2580]: SYSCALL arch=c000003e syscall=1 success=yes exit=3444113 a0=4 a1=7fd95ec39000 a2=348d91 a3=0 items=0 ppid=2575 pid=2580 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="load_policy" exe="/usr/sbin/load_policy" subj=system_u:system_r:load_policy_t:s0 key=(null)
Apr 28 08:58:43 f35 kernel: audit: type=1300 audit(1651129123.001:207): arch=c000003e syscall=1 success=yes exit=3444113 a0=4 a1=7fd95ec39000 a2=348d91 a3=0 items=0 ppid=2575 pid=2580 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="load_policy" exe="/usr/sbin/load_policy" subj=system_u:system_r:load_policy_t:s0 key=(null)
Apr 28 08:58:43 f35 kernel: audit: type=1327 audit(1651129123.001:207): proctitle="load_policy"
Apr 28 08:58:43 f35 audit: PROCTITLE proctitle="load_policy"
Apr 28 08:58:43 f35 audit[2929]: AVC avc:  denied  { mac_admin } for  pid=2929 comm="restorecon" capability=33  scontext=system_u:system_r:setfiles_t:s0 tcontext=system_u:system_r:setfiles_t:s0 tclass=capability2 permissive=0
Apr 28 08:58:43 f35 audit: SELINUX_ERR op=setxattr invalid_context="system_u:object_r:snappy_config_t:s0"
Apr 28 08:58:43 f35 kernel: audit: type=1400 audit(1651129123.361:208): avc:  denied  { mac_admin } for  pid=2929 comm="restorecon" capability=33  scontext=system_u:system_r:setfiles_t:s0 tcontext=system_u:system_r:setfiles_t:s0 tclass=capability2 permissive=0
Apr 28 08:58:43 f35 kernel: audit: type=1401 audit(1651129123.361:208): op=setxattr invalid_context="system_u:object_r:snappy_config_t:s0"
Apr 28 08:58:43 f35 audit[2929]: SYSCALL arch=c000003e syscall=189 success=no exit=-22 a0=7fff8335e270 a1=7ff4a4f20251 a2=5594c98bb9a0 a3=25 items=0 ppid=2582 pid=2929 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="restorecon" exe="/usr/sbin/setfiles" subj=system_u:system_r:setfiles_t:s0 key=(null)
Apr 28 08:58:43 f35 audit: PROCTITLE proctitle=2F7362696E2F726573746F7265636F6E002D65002F737973002D65002F70726F63002D65002F6D6E74002D65002F7661722F746D70002D65002F686F6D65002D65002F726F6F74002D65002F746D70002D69002D52002D66002D
Apr 28 08:58:43 f35 kernel: audit: type=1300 audit(1651129123.361:208): arch=c000003e syscall=189 success=no exit=-22 a0=7fff8335e270 a1=7ff4a4f20251 a2=5594c98bb9a0 a3=25 items=0 ppid=2582 pid=2929 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="restorecon" exe="/usr/sbin/setfiles" subj=system_u:system_r:setfiles_t:s0 key=(null)
Apr 28 08:58:43 f35 kernel: audit: type=1327 audit(1651129123.361:208): proctitle=2F7362696E2F726573746F7265636F6E002D65002F737973002D65002F70726F63002D65002F6D6E74002D65002F7661722F746D70002D65002F686F6D65002D65002F726F6F74002D65002F746D70002D69002D52002D66002D
Apr 28 08:58:43 f35 kernel: SELinux:  Context system_u:object_r:conmon_exec_t:s0 is not valid (left unmapped).
Apr 28 08:58:43 f35 kernel: SELinux:  Context system_u:object_r:container_runtime_exec_t:s0 is not valid (left unmapped).
Apr 28 08:58:43 f35 kernel: SELinux:  Context system_u:object_r:swtpm_exec_t:s0 is not valid (left unmapped).
Apr 28 08:58:43 f35 kernel: SELinux:  Context system_u:object_r:osbuild_exec_t:s0 is not valid (left unmapped).
Apr 28 08:58:43 f35 kernel: SELinux:  Context system_u:object_r:snappy_cli_exec_t:s0 is not valid (left unmapped).
Apr 28 08:58:43 f35 audit[2929]: AVC avc:  denied  { mac_admin } for  pid=2929 comm="restorecon" capability=33  scontext=system_u:system_r:setfiles_t:s0 tcontext=system_u:system_r:setfiles_t:s0 tclass=capability2 permissive=0
Apr 28 08:58:43 f35 audit: SELINUX_ERR op=setxattr invalid_context="system_u:object_r:conmon_exec_t:s0"
Apr 28 08:58:43 f35 audit[2929]: SYSCALL arch=c000003e syscall=189 success=no exit=-22 a0=7fff8335e270 a1=7ff4a4f20251 a2=5594c9931420 a3=23 items=0 ppid=2582 pid=2929 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="restorecon" exe="/usr/sbin/setfiles" subj=system_u:system_r:setfiles_t:s0 key=(null)
Apr 28 08:58:43 f35 audit: PROCTITLE proctitle=2F7362696E2F726573746F7265636F6E002D65002F737973002D65002F70726F63002D65002F6D6E74002D65002F7661722F746D70002D65002F686F6D65002D65002F726F6F74002D65002F746D70002D69002D52002D66002D
Apr 28 08:58:43 f35 kernel: audit: type=1400 audit(1651129123.479:209): avc:  denied  { mac_admin } for  pid=2929 comm="restorecon" capability=33  scontext=system_u:system_r:setfiles_t:s0 tcontext=system_u:system_r:setfiles_t:s0 tclass=capability2 permissive=0
Apr 28 08:58:43 f35 kernel: audit: type=1401 audit(1651129123.479:209): op=setxattr invalid_context="system_u:object_r:conmon_exec_t:s0"
Apr 28 08:58:43 f35 kernel: audit: type=1300 audit(1651129123.479:209): arch=c000003e syscall=189 success=no exit=-22 a0=7fff8335e270 a1=7ff4a4f20251 a2=5594c9931420 a3=23 items=0 ppid=2582 pid=2929 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="restorecon" exe="/usr/sbin/setfiles" subj=system_u:system_r:setfiles_t:s0 key=(null)
Apr 28 08:58:43 f35 audit[2929]: AVC avc:  denied  { mac_admin } for  pid=2929 comm="restorecon" capability=33  scontext=system_u:system_r:setfiles_t:s0 tcontext=system_u:system_r:setfiles_t:s0 tclass=capability2 permissive=0
Apr 28 08:58:43 f35 audit: SELINUX_ERR op=setxattr invalid_context="system_u:object_r:container_runtime_exec_t:s0"
Apr 28 08:58:43 f35 audit[2929]: SYSCALL arch=c000003e syscall=189 success=no exit=-22 a0=7fff8335e270 a1=7ff4a4f20251 a2=5594c9911ce0 a3=2e items=0 ppid=2582 pid=2929 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="restorecon" exe="/usr/sbin/setfiles" subj=system_u:system_r:setfiles_t:s0 key=(null)
Apr 28 08:58:43 f35 audit: PROCTITLE proctitle=2F7362696E2F726573746F7265636F6E002D65002F737973002D65002F70726F63002D65002F6D6E74002D65002F7661722F746D70002D65002F686F6D65002D65002F726F6F74002D65002F746D70002D69002D52002D66002D
Apr 28 08:58:43 f35 audit[2929]: AVC avc:  denied  { mac_admin } for  pid=2929 comm="restorecon" capability=33  scontext=system_u:system_r:setfiles_t:s0 tcontext=system_u:system_r:setfiles_t:s0 tclass=capability2 permissive=0


<snip, 600 more lines of AVC errors, see in the journal lines 3943 - 4589>


Apr 28 08:59:15 f35 kernel: SELinux:  Context system_u:object_r:snappy_var_t:s0 is not valid (left unmapped).
Apr 28 08:59:15 f35 dnf[680]:   Running scriptlet: selinux-policy-targeted-36.7-1.fc36.noarch        749/3503
Apr 28 08:59:15 f35 dnf[680]: Failed to resolve allow statement at /var/lib/selinux/targeted/tmp/modules/200/container/cil:1202
Apr 28 08:59:15 f35 dnf[680]: Failed to resolve AST
Apr 28 08:59:15 f35 dnf[680]: /usr/sbin/semodule:  Failed!


<snip>


Apr 28 08:59:32 f35 dnf[680]:   Running scriptlet: container-selinux-2:2.181.0-2.fc36.noarch         839/3503
Apr 28 08:59:32 f35 dnf[680]:   Downgrading      : container-selinux-2:2.181.0-2.fc36.noarch         839/3503
Apr 28 08:59:40 f35 kernel: SELinux:  Converting 573 SID table entries...
Apr 28 08:59:40 f35 kernel: SELinux:  Context system_u:object_r:snappy_unit_file_t:s0 became valid (mapped).
Apr 28 08:59:40 f35 kernel: SELinux:  Context system_u:object_r:container_var_lib_t:s0 became valid (mapped).
Apr 28 08:59:40 f35 kernel: SELinux:  Context system_u:object_r:snappy_var_lib_t:s0 became valid (mapped).
Apr 28 08:59:40 f35 kernel: SELinux:  Context system_u:object_r:snappy_config_t:s0 became valid (mapped).
Apr 28 08:59:40 f35 kernel: SELinux:  Context system_u:object_r:snappy_confine_exec_t:s0 became valid (mapped).
Apr 28 08:59:40 f35 kernel: SELinux:  Context system_u:object_r:snappy_exec_t:s0 became valid (mapped).
Apr 28 08:59:40 f35 kernel: SELinux:  Context system_u:object_r:snappy_mount_exec_t:s0 became valid (mapped).
Apr 28 08:59:40 f35 kernel: SELinux:  Context system_u:object_r:conmon_exec_t:s0 became valid (mapped).
Apr 28 08:59:40 f35 kernel: SELinux:  Context system_u:object_r:container_runtime_exec_t:s0 became valid (mapped).
Apr 28 08:59:40 f35 kernel: SELinux:  Context system_u:object_r:swtpm_exec_t:s0 became valid (mapped).
Apr 28 08:59:40 f35 kernel: SELinux:  Context system_u:object_r:osbuild_exec_t:s0 became valid (mapped).
Apr 28 08:59:40 f35 kernel: SELinux:  Context system_u:object_r:snappy_cli_exec_t:s0 became valid (mapped).
Apr 28 08:59:40 f35 kernel: SELinux:  Context system_u:object_r:flatpak_helper_exec_t:s0 became valid (mapped).
Apr 28 08:59:40 f35 kernel: SELinux:  Context system_u:object_r:snappy_var_cache_t:s0 became valid (mapped).
Apr 28 08:59:40 f35 kernel: SELinux:  Context system_u:object_r:snappy_var_t:s0 became valid (mapped).
Apr 28 08:59:40 f35 kernel: SELinux:  policy capability network_peer_controls=1
Apr 28 08:59:40 f35 kernel: SELinux:  policy capability open_perms=1
Apr 28 08:59:40 f35 kernel: SELinux:  policy capability extended_socket_class=1
Apr 28 08:59:40 f35 kernel: SELinux:  policy capability always_check_network=0
Apr 28 08:59:40 f35 kernel: SELinux:  policy capability cgroup_seclabel=1
Apr 28 08:59:40 f35 kernel: SELinux:  policy capability nnp_nosuid_transition=1
Apr 28 08:59:40 f35 kernel: SELinux:  policy capability genfs_seclabel_symlinks=1
Apr 28 08:59:40 f35 kernel: SELinux:  policy capability ioctl_skip_cloexec=0
Apr 28 08:59:40 f35 audit: MAC_POLICY_LOAD auid=4294967295 ses=4294967295 lsm=selinux res=1
Apr 28 08:59:40 f35 audit[3638]: SYSCALL arch=c000003e syscall=1 success=yes exit=3598606 a0=4 a1=7f7db0cc9000 a2=36e90e a3=0 items=0 ppid=3627 pid=3638 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="load_policy" exe="/usr/sbin/load_policy" subj=system_u:system_r:load_policy_t:s0 key=(null)
Apr 28 08:59:40 f35 kernel: kauditd_printk_skb: 10 callbacks suppressed
Apr 28 08:59:40 f35 kernel: audit: type=1403 audit(1651129180.751:385): auid=4294967295 ses=4294967295 lsm=selinux res=1
Apr 28 08:59:40 f35 kernel: audit: type=1300 audit(1651129180.751:385): arch=c000003e syscall=1 success=yes exit=3598606 a0=4 a1=7f7db0cc9000 a2=36e90e a3=0 items=0 ppid=3627 pid=3638 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="load_policy" exe="/usr/sbin/load_policy" subj=system_u:system_r:load_policy_t:s0 key=(null)
Apr 28 08:59:40 f35 kernel: audit: type=1327 audit(1651129180.751:385): proctitle="/usr/sbin/load_policy"
Apr 28 08:59:40 f35 audit: PROCTITLE proctitle="/usr/sbin/load_policy"
Apr 28 08:59:40 f35 dnf[680]:   Running scriptlet: container-selinux-2:2.181.0-2.fc36.noarch         839/3503uavc:  op=load_policy lsm=selinux seqno=3 res=1


<snip>


Apr 28 09:00:21 f35 dnf[680]:   Running scriptlet: snapd-selinux-2.54.4-1.fc36.noarch               1316/3503
Apr 28 09:00:21 f35 dnf[680]:   Downgrading      : snapd-selinux-2.54.4-1.fc36.noarch               1316/3503
Apr 28 09:00:29 f35 kernel: SELinux:  Converting 596 SID table entries...
Apr 28 09:00:29 f35 kernel: SELinux:  policy capability network_peer_controls=1
Apr 28 09:00:29 f35 kernel: SELinux:  policy capability open_perms=1
Apr 28 09:00:29 f35 kernel: SELinux:  policy capability extended_socket_class=1
Apr 28 09:00:29 f35 kernel: SELinux:  policy capability always_check_network=0
Apr 28 09:00:29 f35 kernel: SELinux:  policy capability cgroup_seclabel=1
Apr 28 09:00:29 f35 kernel: SELinux:  policy capability nnp_nosuid_transition=1
Apr 28 09:00:29 f35 kernel: SELinux:  policy capability genfs_seclabel_symlinks=1
Apr 28 09:00:29 f35 kernel: SELinux:  policy capability ioctl_skip_cloexec=0
Apr 28 09:00:29 f35 audit: MAC_POLICY_LOAD auid=4294967295 ses=4294967295 lsm=selinux res=1
Apr 28 09:00:29 f35 kernel: kauditd_printk_skb: 10 callbacks suppressed
Apr 28 09:00:29 f35 kernel: audit: type=1403 audit(1651129229.564:469): auid=4294967295 ses=4294967295 lsm=selinux res=1
Apr 28 09:00:29 f35 audit[5912]: SYSCALL arch=c000003e syscall=1 success=yes exit=3598594 a0=4 a1=7fd07a4f7000 a2=36e902 a3=0 items=0 ppid=5905 pid=5912 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="load_policy" exe="/usr/sbin/load_policy" subj=system_u:system_r:load_policy_t:s0 key=(null)
Apr 28 09:00:29 f35 kernel: audit: type=1300 audit(1651129229.564:469): arch=c000003e syscall=1 success=yes exit=3598594 a0=4 a1=7fd07a4f7000 a2=36e902 a3=0 items=0 ppid=5905 pid=5912 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="load_policy" exe="/usr/sbin/load_policy" subj=system_u:system_r:load_policy_t:s0 key=(null)
Apr 28 09:00:29 f35 kernel: audit: type=1327 audit(1651129229.564:469): proctitle="/usr/sbin/load_policy"
Apr 28 09:00:29 f35 audit: PROCTITLE proctitle="/usr/sbin/load_policy"
Apr 28 09:01:03 f35 dnf[680]:   Running scriptlet: snapd-selinux-2.54.4-1.fc36.noarch               1316/3503uavc:  op=load_policy lsm=selinux seqno=4 res=1
Apr 28 09:01:03 f35 dnf[680]:   Upgrading        : flatpak-selinux-1.12.7-2.fc36.noarch             1317/3503
Apr 28 09:01:10 f35 kernel: SELinux:  Converting 599 SID table entries...
Apr 28 09:01:10 f35 kernel: SELinux:  policy capability network_peer_controls=1
Apr 28 09:01:10 f35 kernel: SELinux:  policy capability open_perms=1
Apr 28 09:01:10 f35 kernel: SELinux:  policy capability extended_socket_class=1
Apr 28 09:01:10 f35 kernel: SELinux:  policy capability always_check_network=0
Apr 28 09:01:10 f35 kernel: SELinux:  policy capability cgroup_seclabel=1
Apr 28 09:01:10 f35 kernel: SELinux:  policy capability nnp_nosuid_transition=1
Apr 28 09:01:10 f35 kernel: SELinux:  policy capability genfs_seclabel_symlinks=1
Apr 28 09:01:10 f35 kernel: SELinux:  policy capability ioctl_skip_cloexec=0
Apr 28 09:01:10 f35 audit: MAC_POLICY_LOAD auid=4294967295 ses=4294967295 lsm=selinux res=1
Apr 28 09:01:10 f35 audit[6570]: SYSCALL arch=c000003e syscall=1 success=yes exit=3598606 a0=4 a1=7fba3f15b000 a2=36e90e a3=0 items=0 ppid=6563 pid=6570 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="load_policy" exe="/usr/sbin/load_policy" subj=system_u:system_r:load_policy_t:s0 key=(null)
Apr 28 09:01:11 f35 kernel: audit: type=1403 audit(1651129270.969:470): auid=4294967295 ses=4294967295 lsm=selinux res=1
Apr 28 09:01:11 f35 kernel: audit: type=1300 audit(1651129270.969:470): arch=c000003e syscall=1 success=yes exit=3598606 a0=4 a1=7fba3f15b000 a2=36e90e a3=0 items=0 ppid=6563 pid=6570 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="load_policy" exe="/usr/sbin/load_policy" subj=system_u:system_r:load_policy_t:s0 key=(null)
Apr 28 09:01:11 f35 kernel: audit: type=1327 audit(1651129270.969:470): proctitle="/usr/sbin/load_policy"
Apr 28 09:01:10 f35 audit: PROCTITLE proctitle="/usr/sbin/load_policy"
Apr 28 09:01:11 f35 dnf[680]:   Running scriptlet: flatpak-selinux-1.12.7-2.fc36.noarch             1317/3503uavc:  op=load_policy lsm=selinux seqno=5 res=1
Apr 28 09:01:11 f35 dnf[680]:   Upgrading        : swtpm-0.7.2-1.20220307git21c90c1.fc36.x86_64     1318/3503
Apr 28 09:01:18 f35 kernel: SELinux:  Converting 599 SID table entries...
Apr 28 09:01:18 f35 kernel: SELinux:  policy capability network_peer_controls=1
Apr 28 09:01:18 f35 kernel: SELinux:  policy capability open_perms=1
Apr 28 09:01:18 f35 kernel: SELinux:  policy capability extended_socket_class=1
Apr 28 09:01:18 f35 kernel: SELinux:  policy capability always_check_network=0
Apr 28 09:01:18 f35 kernel: SELinux:  policy capability cgroup_seclabel=1
Apr 28 09:01:18 f35 kernel: SELinux:  policy capability nnp_nosuid_transition=1
Apr 28 09:01:18 f35 kernel: SELinux:  policy capability genfs_seclabel_symlinks=1
Apr 28 09:01:18 f35 kernel: SELinux:  policy capability ioctl_skip_cloexec=0
Apr 28 09:01:18 f35 audit: MAC_POLICY_LOAD auid=4294967295 ses=4294967295 lsm=selinux res=1
Apr 28 09:01:18 f35 kernel: audit: type=1403 audit(1651129278.417:471): auid=4294967295 ses=4294967295 lsm=selinux res=1
Apr 28 09:01:18 f35 audit[6583]: SYSCALL arch=c000003e syscall=1 success=yes exit=3598606 a0=4 a1=7f96d6d57000 a2=36e90e a3=0 items=0 ppid=6575 pid=6583 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="load_policy" exe="/usr/sbin/load_policy" subj=system_u:system_r:load_policy_t:s0 key=(null)
Apr 28 09:01:18 f35 audit: PROCTITLE proctitle="/usr/sbin/load_policy"
Apr 28 09:01:18 f35 kernel: audit: type=1300 audit(1651129278.417:471): arch=c000003e syscall=1 success=yes exit=3598606 a0=4 a1=7f96d6d57000 a2=36e90e a3=0 items=0 ppid=6575 pid=6583 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="load_policy" exe="/usr/sbin/load_policy" subj=system_u:system_r:load_policy_t:s0 key=(null)
Apr 28 09:01:18 f35 kernel: audit: type=1327 audit(1651129278.417:471): proctitle="/usr/sbin/load_policy"
Apr 28 09:01:25 f35 kernel: SELinux:  Converting 599 SID table entries...
Apr 28 09:01:25 f35 kernel: SELinux:  policy capability network_peer_controls=1
Apr 28 09:01:25 f35 kernel: SELinux:  policy capability open_perms=1
Apr 28 09:01:25 f35 kernel: SELinux:  policy capability extended_socket_class=1
Apr 28 09:01:25 f35 kernel: SELinux:  policy capability always_check_network=0
Apr 28 09:01:25 f35 kernel: SELinux:  policy capability cgroup_seclabel=1
Apr 28 09:01:25 f35 kernel: SELinux:  policy capability nnp_nosuid_transition=1
Apr 28 09:01:25 f35 kernel: SELinux:  policy capability genfs_seclabel_symlinks=1
Apr 28 09:01:25 f35 kernel: SELinux:  policy capability ioctl_skip_cloexec=0
Apr 28 09:01:25 f35 audit: MAC_POLICY_LOAD auid=4294967295 ses=4294967295 lsm=selinux res=1
Apr 28 09:01:25 f35 audit[6591]: SYSCALL arch=c000003e syscall=1 success=yes exit=3598606 a0=4 a1=7f0ab3843000 a2=36e90e a3=0 items=0 ppid=6575 pid=6591 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="load_policy" exe="/usr/sbin/load_policy" subj=system_u:system_r:load_policy_t:s0 key=(null)
Apr 28 09:01:26 f35 kernel: audit: type=1403 audit(1651129285.971:472): auid=4294967295 ses=4294967295 lsm=selinux res=1
Apr 28 09:01:26 f35 kernel: audit: type=1300 audit(1651129285.971:472): arch=c000003e syscall=1 success=yes exit=3598606 a0=4 a1=7f0ab3843000 a2=36e90e a3=0 items=0 ppid=6575 pid=6591 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="load_policy" exe="/usr/sbin/load_policy" subj=system_u:system_r:load_policy_t:s0 key=(null)
Apr 28 09:01:26 f35 kernel: audit: type=1327 audit(1651129285.971:472): proctitle="/usr/sbin/load_policy"
Apr 28 09:01:25 f35 audit: PROCTITLE proctitle="/usr/sbin/load_policy"
Apr 28 09:01:26 f35 dnf[680]:   Running scriptlet: swtpm-0.7.2-1.20220307git21c90c1.fc36.x86_64     1318/3503uavc:  op=load_policy lsm=selinux seqno=7 res=1
Apr 28 09:01:26 f35 dnf[680]:   Upgrading        : swtpm-tools-0.7.2-1.20220307git21c90c1.fc36.x8   1319/3503

Created attachment 1875596 [details]
upgrade journal from default F35 install (outdated)

For comparison, this is the upgrade journal when I upgrade a default F35 Workstation install (without any updates installed) containing selinux-policy-35.3-1.20211019git94970fc.fc35 to F36 including selinux-policy-36.7-1.fc36. Interestingly, it contains fewer AVC and other errors, but they are still there.

I'm including this because I'm afraid we'll need to have a working solution for people who upgrade to F36, but don't have an up-to-date selinux-policy.fc35 when they do. Either there needs to be a package update that fixes it automatically (preferred), or at least we need to have good and safe documentation of commands to run in order to fix this. Otherwise this problem might affect a large portion of our user base. And we still don't know how to fix this properly for people who are already affected. Numerous people post 'this magic command worked for me' reports, but we need something safe, verified and universal.

Zdenek, please look at comment 83 and 84, thank you.

(In reply to Kamil Páral from comment #85)
> Zdenek, please look at comment 83 and 84, thank you.

I have been troubleshooting it already since you reported it, one problem found is container-selinux still contains the removed classes, checking further.

(In reply to Kamil Páral from comment #84)
> Created attachment 1875596 [details]
> upgrade journal from default F35 install (outdated)
> 
> For comparison, this is the upgrade journal when I upgrade a default F35
> Workstation install (without any updates installed) containing
> selinux-policy-35.3-1.20211019git94970fc.fc35 to F36 including
> selinux-policy-36.7-1.fc36. Interestingly, it contains fewer AVC and other
> errors, but they are still there.
> 
> I'm including this because I'm afraid we'll need to have a working solution
> for people who upgrade to F36, but don't have an up-to-date
> selinux-policy.fc35 when they do. Either there needs to be a package update
> that fixes it automatically (preferred), or at least we need to have good
> and safe documentation of commands to run in order to fix this. Otherwise
> this problem might affect a large portion of our user base. And we still
> don't know how to fix this properly for people who are already affected.
> Numerous people post 'this magic command worked for me' reports, but we need
> something safe, verified and universal.
I don't think there is another solution than have the system fully updated, namely https://bodhi.fedoraproject.org/updates/FEDORA-2022-c5bee6b70f
The update instructions include dnf upgrade --refresh
which is enough.

I updated F35->F36 many times, following instructions and not making any other hacks. Until yesterday it kept failing, today it started to work. The system after update looks good, all modules seem to be working.

The only outstanding problem are the failed statements during upgrade which are in journal only, not audited on the disk, so you need to make an extra effort to see them.
Problem is that container-selinux hasn't been updated not to refer to classes which would be removed during update, I'll create a bz for that. Anyway the resulting state is okay, even with container-selinux.

From my point of view there is no serious problem in the updating process now.

(In reply to Zdenek Pytela from comment #87)
> I don't think there is another solution than have the system fully updated,
> namely https://bodhi.fedoraproject.org/updates/FEDORA-2022-c5bee6b70f

If somebody upgrades to F36 from an outdated F35, what advice should we give him? Is there a better advice than reinstalling the system from scratch?

> The update instructions include dnf upgrade --refresh
> which is enough.

That's the commandline approach. But GUI apps (gnome-software) don't ask users to fully update beforehand.

> I updated F35->F36 many times, following instructions and not making any
> other hacks. Until yesterday it kept failing, today it started to work. The
> system after update looks good, all modules seem to be working.

How can I determine if the system is in a good or bad state?

> Problem is that container-selinux hasn't been updated not to refer to
> classes which would be removed during update, I'll create a bz for that.

Please link it here, thank you.

> Anyway the resulting state is okay, even with container-selinux.
> From my point of view there is no serious problem in the updating process
> now.

Ok, that's great to hear. Sorry for pestering you with additional questions, but we need to cover users who upgrade from an outdated system somehow, at least with good documentation.

(In reply to Zdenek Pytela from comment #87)
> I don't think there is another solution than have the system fully updated,
> namely https://bodhi.fedoraproject.org/updates/FEDORA-2022-c5bee6b70f

When discussing the blocker status of this bug, I'd like stakeholders to address:
1) whether we feel comfortable releasing in this state (tldr: Problem fixed in F35 updates 2 days ago, but many people won't have the update installed when upgrading to F36. The affected system might be in an inconsistent state, preventing further system updates. We currently have no recommendation on how to fix affected systems, safely and universally.)
2) whether we want to postpone the release by another week and try to implement some quick hack in gnome-software to advise people to fully update their F35 system first before commencing with the upgrade to F36 (basically a very basic implementation of a feature requested in bug 1336435).

I just succeed to upgrade to 36 but Podman exposed the new problem: unresolved libtinfo that prevents to test it. Is it a part of ncurses? The state of ncurses update looks OK and the libtinfo as dependency is listed. But libtinfo.so(6) is missed and I can't find its provider. This is very frequently reported issue for other distros. I hope that only Podman still uses curses today. I opened the separate issue on Podman.
After startup systemd shows 1 failed unit - kata. It is obviously not a release blocker.

For the record, at go/no-go today we decided to slip another week, so we'll have an extra week for this update to get through to F35 users before they upgrade.

I just want to note that all rpm-ostree systems that include both selinux-policy-targeted and container-selinux in the base image then inherently build things as an atomic, transactional unit.
All filesystem labels are computed server side.  We also (only currently for Fedora CoreOS) actually boot and test that fully formed image in CI in a variety of ways before it is ever shipped to humans.  Your system is always only running the combined policy version A or B, never "new selinux-policy-targeted but old container-selinux" etc.

(In reply to Kamil Páral from comment #88)
> (In reply to Zdenek Pytela from comment #87)
> > I don't think there is another solution than have the system fully updated,
> > namely https://bodhi.fedoraproject.org/updates/FEDORA-2022-c5bee6b70f
> 
> If somebody upgrades to F36 from an outdated F35, what advice should we give
> him? Is there a better advice than reinstalling the system from scratch?
dnf reinstall selinux-policy-targeted swtpm snapd-selinux flatpak-selinux container-selinux osbuild-selinux
should do it, but it depends on the actual state.

> > The update instructions include dnf upgrade --refresh
> > which is enough.
> 
> That's the commandline approach. But GUI apps (gnome-software) don't ask
> users to fully update beforehand.
I was not aware of this.

> > I updated F35->F36 many times, following instructions and not making any
> > other hacks. Until yesterday it kept failing, today it started to work. The
> > system after update looks good, all modules seem to be working.
> 
> How can I determine if the system is in a good or bad state?
I can't figure out a single command to check as there can be different states of the system.
# matchpathcon /var/lib/containers
/var/lib/containers     system_u:object_r:container_var_lib_t:s0
(good)
# matchpathcon /var/lib/containers
/var/lib/containers     system_u:object_r:unlabeled_t:s0
(bad)

With setools-console installed, 
# seinfo -xt container_file_t

Types: 1
   type container_file_t alias { svirt_sandbox_file_t svirt_lxc_file_t }, device_node, file_type, filesystem_type, mountpoint, non_auth_file_type, non_security_file_type, noxattrfs, ptynode, svirt_file_type;
(good)
# seinfo -xt container_file_t

Types: 0
(bad)

> > Problem is that container-selinux hasn't been updated not to refer to
> > classes which would be removed during update, I'll create a bz for that.
> 
> Please link it here, thank you.
https://bugzilla.redhat.com/show_bug.cgi?id=2079800

> > Anyway the resulting state is okay, even with container-selinux.
> > From my point of view there is no serious problem in the updating process
> > now.
> 
> Ok, that's great to hear. Sorry for pestering you with additional questions,
> but we need to cover users who upgrade from an outdated system somehow, at
> least with good documentation.
There are some workarounds which should work in most cases, but in the others there will hardly be a universal set of fixing commands.

The errors from comment 83 seem resolved in bug 2079800 , great.

(In reply to Zdenek Pytela from comment #93)
> dnf reinstall selinux-policy-targeted swtpm snapd-selinux flatpak-selinux
> container-selinux osbuild-selinux
> should do it, but it depends on the actual state.

If the transaction happens to print some errors (e.g. like in comment 0), suggesting users to try it again with `setenforce 0` shouldn't hurt anything, right?

Zdenek, I tried to create a Common Issues entry describing this problem for people who hit it (there will inevitably be some):
https://ask.fedoraproject.org/t/common-issues/21867

Can you please proof-read it and correct it, if needed? You can either post corrections into Ask as comments, here, or directly to me over email. Thanks!

Zdenek, we have a person here who was affected and the reinstall command didn't help, according to him:
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org/thread/K7G3MEPQWPBMOFZQAP5AHUEKXWX4BDLQ/

(In reply to Kamil Páral from comment #95)
> Zdenek, I tried to create a Common Issues entry describing this problem for
> people who hit it (there will inevitably be some):
> https://ask.fedoraproject.org/t/common-issues/21867
> 
> Can you please proof-read it and correct it, if needed? You can either post
> corrections into Ask as comments, here, or directly to me over email. Thanks!

It looks fine. A few comments:

The "might" word applies when packages with custom selinux-policy modules are installed and they use socket_class_set in raw rules which expands to all currently defined classes. The same would happen if there was such a local policy created by the administrator.

I believe setenforce 0 will not make any change. Instead, the semodule -r command may help, but I cannot think of all possible "going bad" scenarios to test some minimum set. It possibly can be:

  semodule -X 200 -r snappy -r container -r flatpak -r osbuild -r swtpm -r swtpm_svirt
  dnf reinstall selinux-policy-targeted swtpm snapd-selinux flatpak-selinux container-selinux osbuild-selinux

From SELinux PoV, reboot is not needed after any update. What may be needed is fixfiles -F onboot if the system was in this undefined state for some time, then also reboot would be necessary.

Thanks, Zdenek. I updated the guide. You can see the diff by clicking the orange pencil-on-paper icon.

Just to register here, that Fedora-Workstation-Live-x86_64-36-1.5.iso and all other RC 1.5 too are with selinux-policy-36.7-1.fc36.noarch. not the last 36.8-1.fc36.
Is that a problem?

It's unfortunate but I don't think it's relevant to this bug. We intended to pull in 36.8-1 for a different FE. AFAIK 36.7-1 should be new enough for this bug. It'd be good if Zdenek can confirm, though.

(In reply to Adam Williamson from comment #100)
> It's unfortunate but I don't think it's relevant to this bug. We intended to
> pull in 36.8-1 for a different FE. AFAIK 36.7-1 should be new enough for
> this bug. It'd be good if Zdenek can confirm, though.

Yes it is, 36.8-1 contains additional important improvements.

The F34 and F35 updates are in stable now and have been for some time. I don't think there's anything more we can really do here outside of documentation.

*** Bug 2133042 has been marked as a duplicate of this bug. ***