Created attachment 1511680 [details]
dmesg output from the shutdown dracut shell

Created attachment 1511681 [details]
rdsosreport.txt from shutdown dracut shell

I ran audit2allow -i dmesg_dracut_shutdown_1.txt -M dracut-systemd-shutdown
sudo semodule -i dracut-systemd-shutdown.pp
to create and install a local policy module with the rules
allow init_t fsadm_exec_t:file execute_no_trans;
allow init_t fsadm_t:process2 nosuid_transition;
allow init_t lvm_exec_t:file execute_no_trans;
allow init_t lvm_t:process2 nosuid_transition;
allow init_t mount_exec_t:file execute_no_trans;
allow init_t mount_t:process2 nosuid_transition;
allow init_t root_t:dir create;

I didn't see the shutdown errors I reported above after installing that policy module. I noted the following denials in the dmesg output from two subsequent shutdown dracut shells.

audit: type=1400 audit(1544159727.187:336): avc:  denied  { create } for  pid=1 comm="shutdown" name=".profile" scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:root_t:s0 tclass=file permissive=0

audit: type=1400 audit(1544161611.534:358): avc:  denied  { open } for  pid=1 comm="shutdown" path="/.profile" dev="tmpfs" ino=39404 scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:root_t:s0 tclass=file permissive=0

I added the following two rules to the module and installed it.
allow init_t root_t:file create;
allow init_t root_t:file open;

The dmesg output in the next shutdown dracut shell showed no denials, but more denials might have been missed due to ratelimiting of the messages as in
systemd-shutdow: 28 output lines suppressed due to ratelimiting
shutdown: 7 output lines suppressed due to ratelimiting

I've previously seen denials involving init_t labelled files like /usr/lib/systemd/systemd-user-runtime-dir which should have been systemd_logind_exec_t described in https://bugzilla.redhat.com/show_bug.cgi?id=1644313 
I don't know if systemd-shutdown and shutdown should be labelled init_t or something else while dracut is running at shutdown.
ls -laZ /usr/lib/systemd/systemd-shutdown showed
-rwxr-xr-x. 1 root root system_u:object_r:init_exec_t:s0 50896 Oct 28 21:01 /usr/lib/systemd/systemd-shutdown

The denials involved nosuid_transition and execute_no_trans with mount_t, fsadm_t, lvm_t targets on the files /usr/bin/mount, /usr/sbin/losetup, and /usr/sbin/dmsetup. All the mount points shown by cat /proc/self/mountinfo in the rdsosreport.txt I attached had nosuid which might be related to the denials.
I'm reassigning this report to selinux-policy for the reasons above.

I downgraded to dracut-049-11.git20181024.fc29 before installing the policy modules above, and I saw no shutdown errors with that build. In the shutdown dracut shell, I got permission denied errors when running dmesg and mounting /boot to copy rdsosreport.txt to. rdsosreport.txt showed only / mounted with nosuid. 

dracut-049-25.git20181204.fc29 had 0016.patch with the subject dracut-initramfs-restore: set selinux labels at
https://src.fedoraproject.org/rpms/dracut/c/02dec613593d9f053e67d5c10e5db60dca07c984?branch=f29
That patch might be related to the denials in question.

I have this bug as well, with the same symptoms. I haven't tried the audit2allow stuff yet.

Should this be filed against selinux-policy? It seems to be an SELinux problem.

(In reply to Russell Golden from comment #4)
> I have this bug as well, with the same symptoms. I haven't tried the
> audit2allow stuff yet.
> 
> Should this be filed against selinux-policy? It seems to be an SELinux
> problem.

Russell, I reassigned this report from dracut to selinux-policy in comment 3 since the policy module I created fixed the errors and they also involved systemd-shutdown. I haven't seen these shutdown errors again since I installed the policy module I described. If you want to try to install the same policy module, you could create a file dmesg_dracut_shutdown_2.txt containing 

audit: type=1400 audit(1544159727.187:336): avc:  denied  { create } for  pid=1 comm="shutdown" name=".profile" scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:root_t:s0 tclass=file permissive=0

audit: type=1400 audit(1544161611.534:358): avc:  denied  { open } for  pid=1 comm="shutdown" path="/.profile" dev="tmpfs" ino=39404 scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:root_t:s0 tclass=file permissive=0

You could download the attachment I labelled dmesg output from the shutdown dracut shell and name it dmesg_dracut_shutdown_1.txt in the same directory as the file above. Then you could run

audit2allow -i dmesg_dracut_shutdown_*.txt -M dracut-systemd-shutdown
sudo semodule -i dracut-systemd-shutdown.pp

Lukas Vrabec assigned the priority and severity to medium on 12-18, so I guess that he and maybe the other selinux maintainers are aware of the issue. I'm unsure of whether the dracut shutdown and systemd-shutdown programs should be labelled init_t during shutdown or not, but the denials indicated that they were while dracut was running right before my system shut off. Bug 1665432 appears to involve the same shutdown dmsetup permission denied errors.

commit 18ccb6cef4e6c0b8709a3ccca6999a327fad4b95 (HEAD -> rawhide)
Author: Lukas Vrabec <lvrabec>
Date:   Wed Feb 13 13:35:01 2019 +0100

    Allow nnp transition for domains fsadm_t, lvm_t and mount_t


Added fixes for nosuid_transition. What's in the /.profile file? Have to be this file in / ? 

THanks,
Lukas.

(In reply to Lukas Vrabec from comment #6)
> commit 18ccb6cef4e6c0b8709a3ccca6999a327fad4b95 (HEAD -> rawhide)
> Author: Lukas Vrabec <lvrabec>
> Date:   Wed Feb 13 13:35:01 2019 +0100
> 
>     Allow nnp transition for domains fsadm_t, lvm_t and mount_t
> 
> 
> Added fixes for nosuid_transition. What's in the /.profile file? Have to be
> this file in / ? 
> 
> THanks,
> Lukas.

I searched for /.profile in the initramfs I extracted using dracut for the 4.19.5 kernel around 12-05, and I found that standard output is redirected to /.profile if it doesn't exist in the function _emergency_shell() at at initramfs-4.19.5/usr/lib/dracut-lib.sh:1106
[ -e /.profile ] || >/.profile

I ran the following commands as root (with the 4.20.7 kernel)
mkdir -p /run/initramfs/etc/cmdline.d
echo "rd.debug rd.break=shutdown" > /run/initramfs/etc/cmdline.d/debug.conf
touch /run/initramfs/.need_shutdown
then shutdown. The error /lib/dracut-lib.sh: line 1106: /.profile: Permission denied was shown in the dracut shutdown shell. Allowing dracut-lib.sh to write to /.profile might be needed. /.profile was an empty file using ls -la and less /.profile. /.profile is the only location of that file name I found in the dracut shutdown initramfs. I ran echo "test" > .profile, which wrote test into .profile. I didn't see any further denials in the dmesg output, although auditd might have been stopped by that point. 

Do the rules in your commit allow both nosuid_transition and execute_no_trans for fsadm_t, lvm_t and mount_t?
The denial of systemd-shutdown creating oldroot maybe while pivoting to the dracut initramfs during shutdown might need to be addressed by another rule. I can provide more information. Thanks.

Hi Matt, 

Thanks for investigation, could you please test it again newer version of the policy: 
https://koji.fedoraproject.org/koji/buildinfo?buildID=1208690

Thanks,
Lukas.

(In reply to Lukas Vrabec from comment #8)
> Hi Matt, 
> 
> Thanks for investigation, could you please test it again newer version of
> the policy: 
> https://koji.fedoraproject.org/koji/buildinfo?buildID=1208690
> 
> Thanks,
> Lukas.

I updated to 3.14.2-49 from koji. I removed the policy module I created before using 
sudo semodule -r dracut-systemd-shutdown 
I put printk_devkmsg=on log_buf_len=16M on the 4.20.8 kernel command line when booting because some of the denials were suppressed due to ratelimiting otherwise. In the dracut shutdown shell, dmesg showed the same seven denials I originally reported. The error dracut: /lib/dracut/hooks/shutdown/25-dm-shutdown.sh: line 43: /usr/sbin/dmsetup: Permission denied was near the end of the dmesg output, and that and 20+ similar errors were shown when I shutdown without the dracut shutdown shell being enabled as before. 

Was your commit 18ccb6ce included in 3.14.2-49? I don't see dracut mentioned in 3.14.2-49's changelog. The denials of the dracut shutdown program (with label init_t) running execute_no_trans on processes (labels) /usr/sbin/losetup (fsadm_exec_t), /usr/sbin/dmsetup (lvm_exec_t), /usr/bin/mount (mount_exec_t). I didn't see fsadm_exec_t, lvm_exec_t, mount_exec_t in the commit 18ccb6ce so other rules for those labels might need to be added. The /.profile and /oldroot directory weren't shown in the dracut shutdown shell using ls -la, whereas they were created with the policy module I installed. Thanks.

I updated to 3.14.2-50 from koji which had "Allow nnp transition for domains fsadm_t, lvm_t and mount_t" in its changelog as in comment 6 https://koji.fedoraproject.org/koji/buildinfo?buildID=1216089
I removed the policy module I created as in comment 9. I shutdown the system. The errors like dracut: /lib/dracut/hooks/shutdown/25-dm-shutdown.sh: line 43: /usr/sbin/dmsetup: Permission denied weren't shown. I saw the following errors involving the mounting of /oldroot which I hadn't seen before likely due to them scrolling off screen due to the previous errors:
mount: /oldsys/sys: special device /oldroot/sys does not exist
mount: /oldsys/proc: special device /oldroot/proc does not exist
mount: /oldsys/run: special device /oldroot/run does not exist
mount: /oldsys/dev: special device /oldroot/dev does not exist

An error about the remove ioctl failing on fedora-root followed might be related to the denials involving the dracut shutdown programs executing dmsetup I mentioned previously.
device-mapper: remove ioctl on fedora-root failed: device or remove command failed

/usr/lib/dracut/hooks/shutdown/25-dm-shutdown.sh:21 in _remove_dm() in the initramfs might be where that error was from:
dmsetup -v --noudevsync remove "$devname" || return $? 

My system has root=/dev/mapper/fedora-root on the kernel command line and 
ls -lZ /dev/mapper/fedora-root
lrwxrwxrwx. 1 root root system_u:object_r:device_t:s0 7 Mar  1 16:13 /dev/mapper/fedora-root -> ../dm-0

In the dracut shutdown shell, dmesg showed the denials of systemd-shutdown creating /oldroot and the dracut shutdown program creating /.profile as before. /oldroot and /.profile weren't shown in the dracut shutdown shell using ls -la. The other denials I reported weren't shown. Some shutdown messages were suppressed due to ratelimiting even though I put printk_devkmsg=on log_buf_len=16M on the kernel command line so there might be other denials that were missed. Rules allowing system-shutdown and the dracut shutdown programs (both labelled init_t) to create, open, and probably write to /oldroot and /.profile (labelled root_t) might need to be added. Thanks.

Running F30, I still get the errors regarding /oldroot as well... and when using rd.break, also errors regarding /.profile, current policy is:

selinux-policy-3.14.3-46.fc30.noarch

The following module seems to fix the problems, but it'd be nice if it was part of the regular release :)

policy_module(myshutdown,0.1.0)

require {
        type init_t, root_t;
};

# allow dracut to create /oldroot
allow init_t root_t:dir create;

# allow dracut to create /.profile (rd.break)
allow init_t root_t:file create;

commit 19d8e52639687fbf2766d6d4d7e2998911759d4e (HEAD -> rawhide, origin/rawhide)
Author: Lukas Vrabec <lvrabec>
Date:   Wed Oct 30 19:23:03 2019 +0100

    Update files_filetrans_named_content() interface to allow caller domain to create /oldroot /.profile with correct label etc_runtime_t

This message is a reminder that Fedora 29 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 29 on 2019-11-26.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '29'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 29 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

I saw the same /oldroot errors and denials as in comment 10 when shutting down after installing F31 until I added the custom policy module I mentioned in comment 3. I'm changing the version to 31. Thanks for the patch.

FEDORA-2019-aec8f7ab50 has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2019-aec8f7ab50

selinux-policy-3.14.4-40.fc31 has been pushed to the Fedora 31 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-aec8f7ab50

I updated to selinux-policy-3.14.4-40.fc31. I removed the module with the rules I mentioned in comment 3. I've seen the same /oldroot error messages as in comment 10 when shutting down. The errors happened while the initramfs root was mounted right before the system powered off. The label of /oldroot might've been different in the normal root and the initramfs root.

Hi Matt, 

When are these file and dir available when you're reproducing the issue? If yes please make sure they are removed. 

Thanks,
Lukas.

(In reply to Lukas Vrabec from comment #18)
> Hi Matt, 
> 
> When are these file and dir available when you're reproducing the issue? If
> yes please make sure they are removed. 
> 
> Thanks,
> Lukas.

Lukas, I reproduced the errors involving /oldroot like in comment 10 by shutting down the system, pressing Escape, and looking at the shutdown messages right before the system powered off. The /oldroot directory doesn't exist in my normal F31 root directory or the initramfs root directory when I was in the dracut shutdown shell. /oldsys in the errors in comment 10 was present in the initramfs root directory in the dracut shutdown shell. I didn't see any denials in dmesg in the shutdown shell which was different from before. /.profile was present as an empty file in the dracut shutdown shell, but it wasn't when I reported these denials initially. I meant that the labels for the processes and directories involved might be different between the normal root and the initramfs not just for /oldroot. I think the remaining issue is that /oldroot isn't being created in the shutdown initramfs. The errors in comment 10 weren't shown when the module in comment 3 was installed. I think the rule allow init_t root_t:dir create; allowed /oldroot to be created in the shutdown initramfs. Thanks.

I added printk_devkmsg=on log_buf_len=16M to the kernel command line when booting because over 70 of the shutdown messages were suppressed due to ratelimiting as in comment 9. In the dracut shutdown shell, dmesg showed the denial of systemd-shutdown creating /oldroot labelled etc_runtime_t (instead of root_t as in comment 3).

audit: type=1400 audit(1573270154.803:425): avc:  denied  { create } for  pid=1 comm="systemd-shutdow" name="oldroot" scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:etc_runtime_t:s0 tclass=dir permissive=0

I guess the commit in comment 12 explains the change of /oldroot being labelled etc_runtime_t. sudo sesearch -s init_t -t etc_runtime_t -A had the following line showing that create wasn't allowed for init_t acting on etc_runtime_t directories.
allow init_t etc_runtime_t:dir { add_name getattr ioctl lock open read remove_name search write };

I created a new policy module with audit2allow -i dmesg-dracut-shutdown-2019-10-8.txt -M dracut-shutdown-2 containing the rule 
allow init_t etc_runtime_t:dir create;

I installed the module with sudo semodule -i dracut-shutdown-2.pp When I shut down the system with that module installed, I didn't see the /oldroot errors like in comment 10. In the dracut shutdown shell, ls -laZ showed /oldroot and /.profile labelled etc_runtime_t.

Fixed in Fedora: 

commit f4ed95d2d3c8d4152dccfdf3e2263f74d99ca4f2 (HEAD -> rawhide)
Author: Lukas Vrabec <lvrabec>
Date:   Mon Nov 11 18:29:36 2019 +0100

    Update files_manage_etc_runtime_files() interface to allow manage also
    dirs
    
    Resolves: rhbz#1656430

selinux-policy-3.14.4-40.fc31 has been pushed to the Fedora 31 stable repository. If problems still persist, please make note of it in this bug report.

Note: Lukas' patch in comment 21 didn't make it into 3.14.4-40 (it appears to be in -41+ though, but nothing's released yet).  So /oldroot still can't be created on shutdown...

(In reply to Scott Shambarger from comment #23)
> Note: Lukas' patch in comment 21 didn't make it into 3.14.4-40 (it appears
> to be in -41+ though, but nothing's released yet).  So /oldroot still can't
> be created on shutdown...

I updated to selinux-policy-3.14.4-42.fc31 from updates-testing about 10 days ago, and I removed the module I mentioned in comment 20. I haven't seen the /oldroot errors in comment 10 since then. selinux-policy-3.14.4-42.fc31 had various problems and was obsoleted due to negative karma https://bodhi.fedoraproject.org/updates/FEDORA-2019-fefda9dd5e I haven't seen the problems mentioned there, but I wasn't using the packages that the issues. selinux-policy-3.14.4-43.fc31 was built today at https://koji.fedoraproject.org/koji/buildinfo?buildID=1418803

Hi All, 

Sorry for confusion and delay with package update release. We're working on another issues and I hope we'll do -43.fc31 update in these days. 

THanks,
Lukas

This package has changed maintainer in the Fedora.
Reassigning to the new maintainer of this component.

This message is a reminder that Fedora 31 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 31 on 2020-11-24.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '31'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 31 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

This bug appears fixed (as of F33).

The last of these denials were fixed in selinux-policy-3.14.4-41.fc31 according to the comments above and https://koji.fedoraproject.org/koji/buildinfo?buildID=1418803 and then pushed to stable in selinux-policy-3.14.4-43.fc31 https://bodhi.fedoraproject.org/updates/FEDORA-2019-fefda9dd5e so I'm closing the report. Thanks.