Bug 1893552

After a bit more searching (still with "selinux=0 audit=0" booted), I found this in my syslog, a slightly more verbose message, formatted for readability:


Oct 31 20:17:41 horus setroubleshoot[5405]: SELinux is preventing abrt-action-sav from write access on the file rpmdb.sqlite-wal. For complete SELinux messages run: 
                sealert -l 2641517c-35a9-4628-a089-f41c646f38d7
Oct 31 20:17:41 horus setroubleshoot[5405]: SELinux is preventing abrt-action-sav from write access on the file rpmdb.sqlite-wal.#012#012*****  
                Plugin catchall_labels (83.8 confidence) suggests   *******************#012#012
                If you want to allow abrt-action-sav to have write access on the rpmdb.sqlite-wal file#012
                Then you need to change the label on rpmdb.sqlite-wal#012Do#012
                # semanage fcontext -a -t FILE_TYPE 'rpmdb.sqlite-wal'#012
                where FILE_TYPE is one of the following: abrt_etc_t, abrt_tmp_t, abrt_upload_watch_tmp_t, abrt_var_cache_t, abrt_var_log_t, 
                abrt_var_run_t, afs_cache_t, initrc_tmp_t, kdump_crash_t, mail_home_rw_t, mock_var_lib_t, postfix_postdrop_t, puppet_tmp_t,
                rhsmcertd_var_run_t, rpm_log_t, rpm_var_cache_t, rpm_var_run_t, sysfs_t, user_cron_spool_t, user_tmp_t, usr_t.#012
                Then execute:#012restorecon -v 'rpmdb.sqlite-wal'#012#012#012
                *****  Plugin catchall (17.1 confidence) suggests   **************************#012#012
                If you believe that abrt-action-sav should be allowed write access on the rpmdb.sqlite-wal file by default.#012
                Then you should report this as a bug.#012
                You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012
                # ausearch -c 'abrt-action-sav' --raw | audit2allow -M my-abrtactionsav#012
                # semodule -X 300 -i my-abrtactionsav.pp#012
Oct 31 20:17:41 horus setroubleshootd[5852]: error: Unable to open sqlite database /var/lib/rpm/rpmdb.sqlite: unable to open database file
Oct 31 20:17:41 horus setroubleshoot[5405]: failed to retrieve rpm info for /etc/selinux/targeted
Oct 31 20:17:41 horus setroubleshootd[5852]: error: cannot open Packages index using sqlite - Operation not permitted (1)
Oct 31 20:17:41 horus setroubleshootd[5852]: error: Unable to open sqlite database /var/lib/rpm/rpmdb.sqlite: unable to open database file
Oct 31 20:17:41 horus setroubleshootd[5852]: error: cannot open Name index using sqlite - Operation not permitted (1)
[...]


And then on it goes with thousands of messages from setroubleshootd.


This is how /var/lib/rpm/ looks now:

# ls -lZd /var{,/lib{,/rpm{,/rpmdb.sqlite*}}}
drwxr-xr-x. 1 root root system_u:object_r:var_t:s0           206 Oct 31 20:16 /var
drwxr-xr-x. 1 root root system_u:object_r:var_lib_t:s0       918 Oct 31 21:36 /var/lib
drwxr-xr-x. 1 root root system_u:object_r:var_lib_t:s0       106 Oct 31 20:26 /var/lib/rpm
-rw-r--r--. 1 root root system_u:object_r:var_lib_t:s0 145240064 Nov  1 18:00 /var/lib/rpm/rpmdb.sqlite
-rw-r--r--. 1 root root system_u:object_r:var_lib_t:s0     32768 Nov  1 19:46 /var/lib/rpm/rpmdb.sqlite-shm
-rw-r--r--. 1 root root system_u:object_r:var_lib_t:s0         0 Nov  1 18:00 /var/lib/rpm/rpmdb.sqlite-wal

I'll try the suggestion from above and report back.

I opted for "restorecon" and now have:

-rw-r--r--. 1 root root system_u:object_r:abrt_var_cache_t:s0 145240064 Nov  1 20:20 /var/lib/rpm/rpmdb.sqlite
-rw-r--r--. 1 root root system_u:object_r:abrt_var_cache_t:s0     32768 Nov  1 20:44 /var/lib/rpm/rpmdb.sqlite-shm
-rw-r--r--. 1 root root system_u:object_r:abrt_var_cache_t:s0         0 Nov  1 20:20 /var/lib/rpm/rpmdb.sqlite-wal

But even after rebooting without "selinux=0 audit=0" (and the forced relabelling) both setroubleshootd and audit would not stop complaining. I'm starting to think this upgrade is botched and I should just reinstall from scratch. Some stats from each attempt to get a working system with SELinux enabled:

Attempt #1
   * 90605 audit messages in 12 minutes
   * 92859 setroubleshootd messages in 12 minutes

  $ awk '/SELinux is preventing/ {print $6,$7,$8,$9}' journalctl.out | sort | uniq -c | sort -n
      2 SELinux is preventing abrtd
      2 SELinux is preventing alsactl
      2 SELinux is preventing gssproxy
      2 SELinux is preventing libvirt_leasesh
      6 SELinux is preventing accounts-daemon
      8 SELinux is preventing cupsd
      8 SELinux is preventing NetworkManager
     10 SELinux is preventing (stubby)
     18 SELinux is preventing sadc
    480 SELinux is preventing sssd_nss

 In this log file, sealert would make suggestions which I followed:

 # /sbin/restorecon -v /var/cache/cups/job.cache
 # /sbin/restorecon -v /var/cache/stubby
 # /sbin/restorecon -v /var/lib/alsa/asound.state
 # /sbin/restorecon -v /var/lib/libvirt/dnsmasq/virbr0.status
 # /sbin/restorecon -v /var/log/sa/sa01


Attempt #2
   * 23193 audit messages in 3 minutes
   * 23946 setroubleshootd messages in 3 minutes

  # awk '/SELinux is preventing/ {print $6,$7,$8,$9}' j.3 | sort | uniq -c | sort -n
      2 SELinux is preventing cupsd
      2 SELinux is preventing gssproxy
      4 SELinux is preventing accounts-daemon
      6 SELinux is preventing NetworkManager
    124 SELinux is preventing mandb

 # /sbin/restorecon -v /var/cache/cups/job.cache.O
 # /sbin/restorecon -v /var/cache/man/sk/index.db
 # /sbin/restorecon -v /var/cache/man/sr/index.db
 # /sbin/restorecon -v /var/cache/man/zh_CN/index.db
 
 In fact, I have just removed those cached files to make SELinux shut up about those.

Attempt #3
   * 518 audit messages in 7 minutes
   *   0 setroubleshootd messages in 3 minutes

 # awk '/SELinux is preventing/ {print $6,$7,$8,$9}' j.4 | sort | uniq -c | sort -n
      2 SELinux is preventing gssproxy
      4 SELinux is preventing accounts-daemon
      6 SELinux is preventing NetworkManager
      8 SELinux is preventing bluetoothd

Despite no more messages from "setroubleshootd", the system would still not come up completely, the Gnome login window did not appear. Rebooted again with "selinux=0 audit=0" to have a working F33 system :(

Created attachment 1725605 [details]
messages archive from all 3 attempts

Hi,

Please undo all customization steps, this needs fixing in selinux-policy. To fix it immediately, run restorecon, but with default rules only.

  # restorecon -Rv /var/lib/rpm

*** This bug has been marked as a duplicate of bug 1461313 ***

What do you mean with "customization steps"? I have no custom SELinux rules loaded. I've run "restorecon -Rv /var/lib/rpm" multiple times, along with multiple runs of whole system relabling via "touch /.autorelabel" and also "fixfiles -B onboot", all w/o luck. The system just won't come back online and I'm stuck with "selinux=0 audit=0" to at least boot the system into a usable state.

If you are sure you haven't made any, there is no need for an action. This line indicates though that you have changed the type:

-rw-r--r--. 1 root root system_u:object_r:abrt_var_cache_t:s0 145240064 Nov  1 20:20 /var/lib/rpm/rpmdb.sqlite

The correct one is rpm_var_lib_t.

*** This bug has been marked as a duplicate of bug 1461313 ***

(In reply to Christian Kujau from comment #5)
> What do you mean with "customization steps"? I have no custom SELinux rules
> loaded. I've run "restorecon -Rv /var/lib/rpm" multiple times, along with
> multiple runs of whole system relabling via "touch /.autorelabel" and also
> "fixfiles -B onboot", all w/o luck. The system just won't come back online
> and I'm stuck with "selinux=0 audit=0" to at least boot the system into a
> usable state.

Run:

  sudo semanage fcontext --list --locallist

That would show what customization has been made.

Hm, this command indeed lists some objects:


$ semanage fcontext --list --locallist
SELinux fcontext                                   type               Context
/usr/lib/chrome-sandbox                            all files          system_u:object_r:chrome_sandbox_exec_t:s0 
/usr/lib/chromium-browser                          all files          system_u:object_r:bin_t:s0 
/usr/lib/chromium-browser/chromium-browser.sh      all files          system_u:object_r:bin_t:s0 
/var/lib/rpm/rpmdb.sqlite                          all files          system_u:object_r:abrt_var_cache_t:s0 
/var/lib/rpm/rpmdb.sqlite-shm                      all files          system_u:object_r:abrt_var_cache_t:s0 
/var/lib/rpm/rpmdb.sqlite-wal                      all files          system_u:object_r:abrt_var_cache_t:s0 


But...why?



$ cat /etc/selinux/targeted/contexts/files/file_contexts.local
# This file is auto-generated by libsemanage
# Do not edit directly.

/usr/lib/chrome-sandbox    system_u:object_r:chrome_sandbox_exec_t:s0
/usr/lib/chromium-browser    system_u:object_r:bin_t:s0
/usr/lib/chromium-browser/chromium-browser.sh    system_u:object_r:bin_t:s0
/var/lib/rpm/rpmdb.sqlite    system_u:object_r:abrt_var_cache_t:s0
/var/lib/rpm/rpmdb.sqlite-shm    system_u:object_r:abrt_var_cache_t:s0
/var/lib/rpm/rpmdb.sqlite-wal    system_u:object_r:abrt_var_cache_t:s0

$ rpm -qf /etc/selinux/targeted/contexts/files/file_contexts.local
selinux-policy-targeted-3.14.6-29.fc33.noarch


I don't recall running semanage on these objects, weird.

So, the etc version of this file is pulled from /var/lib/selinux/targeted/active/file_contexts.local,
removing that, NULL'ing its etc version and re-installing selinux-policy-targeted makes 
"semanage fcontext --list --locallist" to no longer return anything, yay! OK, let's hope this helps.