Description of problem: To use machinectl, SELinux must first be set to permissive mode. Some important functions like the login into a container are not usable in enforcing mode even after correcting variuos permission issues. Other functions are partially usable after correction of permissions. Version-Release number of selected component (if applicable): Fedora 33 / systemd 246 & Fedora 32 / systemd 245 How reproducible: always Steps to Reproduce: 1. Create a fresh F33 installation and install additional systemd-container 2. Create a directory /var/lib/machines/test and install a container filesystem in it using dnf (e.g. dnf --releasever=33 --best --setopt=install_weak_deps=False --installroot=/var/lib/machines/test/ install dhcp-client dnf fedora-release glibc glibc-langpack-en glibc-langpack-de iproute iputils less passwd systemd vim-minimal) 3. Start the container as system service via systemctl start systemd-nspawn@test (after After correction of various permission problems, see bug 1900869) 4. Try (a) machienectl list (b) machinectl login test Actual results: (a) The command displays running containers but without network information. SELinux issues the following error messages: (1) SELinux prevents systemd-machine from accessing cap_user's unknown with sys_ptrace access. type=AVC msg=audit(1606055949.332:783): avc: denied { sys_ptrace } for pid=5657 comm="systemd-machine" capability=19 scontext=system_u:system_r:systemd_machined_t:s0 tcontext=system_u:system_r:systemd_machined_t:s0 tclass=cap_userns permissive=0 SELinux prevents (sd-osrelns) from accessing cap_users unknown with sys_admin access. type=AVC msg=audit(1606056106.582:792): avc: denied { sys_admin } for pid=8655 comm="(sd-addrns)" capability=21 scontext=system_u:system_r:systemd_machined_t:s0 tcontext=system_u:system_r:systemd_machined_t:s0 tclass=cap_userns permissive=0 After executing the suggested corrections the command prints a complete list of the running nspawn containers. (b) The Program fails with: Failed to get login PTY: Input/output error SELinux issues the following error messages: (1) SELinux: SELinux prevents (sd-openptns) from accessing cap_user's unknown with setgid access. type=AVC msg=audit(1606056331.257:801): avc: denied { setgid } for pid=8830 comm="(sd-openptns)" capability=6 scontext=system_u:system_r:systemd_machined_t:s0 tcontext=system_u:system_r:systemd_machined_t:s0 tclass=cap_userns permissive=0 (2) SELinux: SELinux prevents (sd-openptns) from accessing cap_users unknown with setuid access. type=AVC msg=audit(1606056459.235:809): avc: denied { setuid } for pid=8954 comm="(sd-openptns)" capability=7 scontext=system_u:system_r:systemd_machined_t:s0 tcontext=system_u:system_r:systemd_machined_t:s0 tclass=cap_userns permissive=0 Despite all proposed corrections the error remains and the system must be put into permissive mode. Expected results: Correct working out of the box Additional info:
I think these permissions can be addressed.
This message is a reminder that Fedora 33 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora 33 on 2021-11-30. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '33'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 33 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
Just checked with a brand new installation of Fedora 35 and systemd 249 & selinux-policy 35.5. One change is to be noted: machinectl works now without "PTY: Input/output error" if all all proposed selinux modification are done.
I've tested with Fedora 34 (systemd-248.9-1.fc34.x86_64 and selinux-policy-34.22-1.fc34.noarch) and machinectl import-tar will not work unless I disable selinux, though the error message is strange: $ sudo machinectl import-tar stage3-amd64-systemd-20211226T170559Z.tar.xz gentoo Failed to transfer image: Remote peer disconnected If I disable selinux the import will proceed as expected. However, if I re-enable selinux after the import, the machine will not start with some denials in the logs: systemd[1]: Starting Container gentoo... audit[619803]: AVC avc: denied { search } for pid=619803 comm="systemd-machine" name="621541" dev="proc" ino=31430177 scontext=system_u:system_r:systemd_machined_t:s0 tcontext=system_u:system_r:unconfined_service_t:s0 tclass=dir permissive=0 systemd-udevd[621543]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable. systemd-udevd[621543]: Using default interface naming scheme 'v247'. systemd-nspawn[621538]: Failed to register machine: Access denied systemd-nspawn[621541]: Parent died too early systemd[1]: systemd-nspawn: Main process exited, code=exited, status=1/FAILURE systemd[1]: systemd-nspawn: Failed with result 'exit-code'. systemd[1]: Failed to start Container gentoo. If I disable SELinux, I am able to start the container and use the shell command to get into it as well.
I forgot to include the log denials in my comment above for the import-tar command. It seems that the denial may have been because the file I was importing was in my home folder, which I would think would be a pretty normal place for such a thing to be: systemd[1]: Starting Virtual Machine and Container Download Service... systemd[1]: Started Virtual Machine and Container Download Service. audit[620333]: AVC avc: denied { read } for pid=620333 comm="systemd-importd" path="/home/bowlofeggs/stage3-amd64-systemd-20211226T170559Z.tar.xz" dev="dm-0" ino=33751693 scontext=system_u:system_r:systemd_importd_t:s0 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=file permissive=0 audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-importd comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' systemd[1]: systemd-importd.service: Deactivated successfully. However, I have some maybe good news! If I import the image from not-my-home-folder (I copied it to /var/lib/machines in the hope that this would give it a favorable context), then it succeeds, and I also seem to be able to start the resulting imported image too. So perhaps it's just funky to try to interact with a user's home folder at all. Does it seem reasonable that importing from a user's home folder should work? I expected it to work, but I could see an argument against it too.
Unfortunately I am not able to reproduce the "maybe good news" from my last comment. Perhaps I had SELinux permissive when I did that, but I am unsure how I got it to work. I've not tried to reproduce the success on that same system and also an F35 system and it fails in both places, unfortunately.
Just checked with a brand new installation of Fedora 36 and systemd 250 & selinux-policy 36.10-1. The bug still exists without perceptible (for me) improvement.
Does your scenario work after loading this policy module? # cat mypolicy.cil ( allow systemd_machined_t unconfined_service_t ( dir ( search ))) ( allow systemd_machined_t unconfined_service_t ( file ( getattr open read ioctl ))) ( allow systemd_machined_t unconfined_service_t ( lnk_file ( getattr read ))) ( allow systemd_machined_t systemd_machined_t ( cap_userns ( sys_ptrace sys_admin setgid setuid ))) ( allow systemd_machined_t tmpfs_t ( lnk_file ( getattr read ))) ( allow systemd_machined_t devpts_t ( chr_file ( open read write ioctl ))) ( allow systemd_machined_t tmpfs_t ( sock_file ( write ))) ( allow system_dbusd_t devpts_t ( chr_file ( read write ))) ( allow systemd_machined_t unconfined_service_t ( unix_stream_socket ( connectto ))) # semodule -i mypolicy.cil # I successfully tested it on RHEL-8.7 and RHEL-9.1.
To find all the missing rules, I had to remove the dontaudit rules from active SELinux policy: # semodule -DB But after several rinse and repeat steps in enforcing mode, the endeavor was successful: # sestatus SELinux status: enabled SELinuxfs mount: /sys/fs/selinux SELinux root directory: /etc/selinux Loaded policy name: targeted Current mode: enforcing Mode from config file: enforcing Policy MLS status: enabled Policy deny_unknown status: allowed Memory protection checking: actual (secure) Max kernel policy version: 33 # machinectl list MACHINE CLASS SERVICE OS VERSION ADDRESSES test container systemd-nspawn fedora 36 - 1 machines listed. # machinectl login test Connected to machine test. Press ^] three times within 1s to exit session. Fedora Linux 36 (Thirty Six) Kernel 5.19.14-200.fc36.x86_64 on an x86_64 (pts/1) test login: Connection to machine test terminated. #
Test coverage for this BZ exists in a form of PR: * https://src.fedoraproject.org/tests/selinux/pull-request/329 The PR waits for review.
Sorry for the delay. I had to sort out my test equipment after all the F37 testing. I tried the cat mypolicy.cil script on a F37 system and the success has been great! I could start a systemd-container service without at first entering all those policy rules I described in the bug report. And I could use all machinectl functions but 2: copy-to and copy-from For the latter I got an "access denied" message. I had to do a 'setenforce 0' before it worked. After reboot and before I did anything with nspwan (and no autostart of a container) I got a SELinux error message: SELinux is preventing systemd-machine from write access on the directory test (that's the container directory I created for testing). advise : setsebool -P daemons_dump_core 1 I got the same message again after I started a nspawn container as a service. But I couldn't detect any issues with the container yet. When I used copy-to in enforcing mode, I got an "access denied" message as above mentioned, but no SELinux messages. In permissive mode, I could copy the file, but got 12 ACE messages (in Cockpit) SELinux is preventing (sd-copy) from getattr access on the file /root/anaconda-ks.cfg. SELinux is preventing (sd-copy) from read access on the file anaconda-ks.cfg. SELinux is preventing (sd-copy) from open access on the file /root/anaconda-ks.cfg. SELinux is preventing (sd-copy) from write access on the directory root. SELinux is preventing (sd-copy) from add_name access on the directory testing2.txt. SELinux is preventing (sd-copy) from create access on the file testing2.txt. SELinux is preventing (sd-copy) from write access on the file /root/testing2.txt. SELinux is preventing (sd-copy) from ioctl access on the file /root/testing2.txt. SELinux is preventing (sd-copy) from setattr access on the file testing2.txt. SELinux is preventing (sd-copy) from using the chown capability. SELinux is preventing (sd-copy) from using the fowner capability. SELinux is preventing (sd-copy) from using the fsetid capability. I couldn't test machinectl bind yet because of user namespacing applied (in config file, does obviously not work in reality). Great work. Would be nice to get soon. Of course, I can test anything else which might help to make it work flawlessly!
I would like to see the SELinux denials which appeared during your nspawn+machined experiments on your machine: # ausearch -m avc -m user_avc -m selinux_err -i -ts today Please attach the output to this BZ. Based on the collected SELinux denials, I can enhance the mypolicy.cil file and eventually make the scenario work as expected. Thank you.
Created attachment 1918078 [details] All the machineclt / nspawn related denials of today (Oct. 14, 2022
After reading the attached SELinux denials, I added just 1 line (last one) into the mypolicy.cil file: # cat mypolicy.cil ( allow systemd_machined_t unconfined_service_t ( dir ( search ))) ( allow systemd_machined_t unconfined_service_t ( file ( getattr open read ioctl ))) ( allow systemd_machined_t unconfined_service_t ( lnk_file ( getattr read ))) ( allow systemd_machined_t systemd_machined_t ( cap_userns ( sys_ptrace sys_admin setgid setuid ))) ( allow systemd_machined_t tmpfs_t ( lnk_file ( getattr read ))) ( allow systemd_machined_t devpts_t ( chr_file ( open read write ioctl ))) ( allow systemd_machined_t tmpfs_t ( sock_file ( write ))) ( allow system_dbusd_t devpts_t ( chr_file ( read write ))) ( allow systemd_machined_t unconfined_service_t ( unix_stream_socket ( connectto ))) ( allow systemd_machined_t systemd_machined_t ( capability ( chown fowner fsetid ))) # semodule -i mypolicy.cil # setsebool -P daemons_dump_core 1 # I ignored the SELinux denials which are related to systemd-gpt-auto-generator, because I believe they do not interfere with systemd-machined / nspawn. If you experiment with the containers in the /root directory, please do that somewhere else (for example: /mnt or /tmp). We try to keep the number of rules which allow access to the /root directory on minimum. # matchpathcon /root /root system_u:object_r:admin_home_t:s0 # matchpathcon /root/testing2.txt /root/testing2.txt system_u:object_r:admin_home_t:s0 # Let me know if the updated policy file improved the situation.
> I ignored the SELinux denials which are related to systemd-gpt-auto-generator, > because I believe they do not interfere with systemd-machined / nspawn. That's completely independent of nspawn and comes up immediately after first boot after installation of Server Edition and before any additional installation > If you experiment with the containers in the /root directory, please do that > somewhere else (for example: /mnt or /tmp). ... OK, I'll add that to our documentation (https://docs.fedoraproject.org/en-US/fedora-server/container-nspawn-install/ by the way, comments welcome) > Let me know if the updated policy file improved the situation. I would like to test it on a fresh installation. Takes a bit of time (it's late in the evening in Europe, now)
I applied your fix to a fresh installation and tested all Machine Commands (1) [root@vm01 ~]# machinectl reboot test01 Could not kill machine: Access denied SELinux is preventing systemd-machine from kill access on the cap_userns labeled systemd_machined_t. type=AVC msg=audit(1665911720.957:488): avc: denied { kill } for pid=2747 comm="systemd-machine" capability=5 scontext=system_u:system_r:systemd_machined_t:s0 tcontext=system_u:system_r:systemd_machined_t:s0 tclass=cap_userns permissive=0 It works, if container uses shared network with host and no private user space (application container in terms of Server Doku) (2) [root@vm01 ~]# machinectl kill test01 Nothing happens, machine still listet, but neither shell nor login possible (timeout). ‚systemctl stop systemd-nspawn@test01‘ works In case on an application conterainer (shared network, no separate user space) no error message, no AVC event, container still listed, but ‚hangs‘, no access possible, an access try results in a time out. After a while the container is no longer listed and everything in normal again (no time out anymore). (3) [root@vm01 ~]# machinectl poweroff test01 Could not kill machine: Access denied SELinux is preventing systemd-machine from kill access on the cap_userns labeled systemd_machined_t. type=AVC msg=audit(1665914393.174:614): avc: denied { kill } for pid=2747 comm="systemd-machine" capability=5 scontext=system_u:system_r:systemd_machined_t:s0 tcontext=system_u:system_r:systemd_machined_t:s0 tclass=cap_userns permissive=0 It works, if container uses shared network with host and no private user space (application container in terms of Server Doku) (4) [root@vm01 ~]# machinectl copy-to test01 /srv/test2.txt /tmp/test2copied.txt Failed to copy: Access denied SELinux is preventing systemd-machine from read access on the directory srv. type=AVC msg=audit(1665920544.937:686): avc: denied { read } for pid=2747 comm="systemd-machine" name="srv" dev="dm-0" ino=8529506 scontext=system_u:system_r:systemd_machined_t:s0 tcontext=system_u:object_r:var_t:s0 tclass=dir permissive=0 This is also true for a system container. (5) [root@vm01 srv]# machinectl copy-from test01 /srv/test.txt ./test.txt Failed to copy: Access denied SELinux is preventing systemd-machine from read access on the directory srv. type=AVC msg=audit(1665915318.844:663): avc: denied { read } for pid=2747 comm="systemd-machine" name="srv" dev="dm-0" ino=8529506 scontext=system_u:system_r:systemd_machined_t:s0 tcontext=system_u:object_r:var_t:s0 tclass=dir permissive=0 solution a: If you want to allow systemd-machine to have read access on the srv directory You need to change the label on srv semanage fcontext -a -t FILE_TYPE 'srv' where FiILE_TYPE is one of …. solution b: If you believe that systemd-machine should be allowed read access on the srv directory by default. You should report this as a bug. In case of an app container (see above): [root@vm01 machines]# machinectl copy-from test02-app /srv/test-vm2.txt /tmp/test-vm2.txt Failed to copy: Access denied type=AVC msg=audit(1665923859.707:799): avc: denied { read } for pid=9463 comm="(sd-copy)" name="srv" dev="dm-1" ino=2534 scontext=system_u:system_r:systemd_machined_t:s0 tcontext=system_u:object_r:var_t:s0 tclass=dir permissive=0 With a system container (see above) [root@vm01 machines]# machinectl bind test01 /srv /mnt Failed to bind mount: Can't bind mount on container with user namespacing applied. No SELinux complain With a app container (see above) (6) [root@vm01 machines]# machinectl bind test02-app /srv /mnt Failed to bind mount: Access denied SELinux is preventing systemd-machine from create access on the directory propagate.xbATaO. type=AVC msg=audit(1665924076.52:805): avc: denied { create } for pid=2747 comm="systemd-machine" name="propagate.xbATaO" scontext=system_u:system_r:systemd_machined_t:s0 tcontext=system_u:object_r:tmp_t:s0 tclass=dir permissive=0 I could not test any of the Image commands, nor of the image transfer commands, because we dont use images here.
I think I ran the same commands as you, but I used /tmp and /mnt locations instead of /srv. Here is the policy module which I built incrementally during my experiments: # cat mypolicy.cil ( allow systemd_machined_t unconfined_service_t ( dir ( search ))) ( allow systemd_machined_t unconfined_service_t ( file ( getattr open read ioctl ))) ( allow systemd_machined_t unconfined_service_t ( lnk_file ( getattr read ))) ( allow systemd_machined_t systemd_machined_t ( cap_userns ( sys_ptrace sys_admin setgid setuid kill ))) ( allow systemd_machined_t tmpfs_t ( lnk_file ( getattr read ))) ( allow systemd_machined_t devpts_t ( chr_file ( open read write ioctl ))) ( allow systemd_machined_t tmpfs_t ( sock_file ( write ))) ( allow system_dbusd_t devpts_t ( chr_file ( read write ))) ( allow systemd_machined_t unconfined_service_t ( unix_stream_socket ( connectto ))) ( allow systemd_machined_t systemd_machined_t ( capability ( chown fowner fsetid ))) ( allow systemd_machined_t mnt_t ( dir ( read write add_name ))) ( allow systemd_machined_t mnt_t ( file ( create write open getattr setattr ioctl ))) ( allow systemd_machined_t tmp_t ( file ( create write open getattr setattr ioctl ))) ( allow systemd_machined_t user_tmp_t ( dir ( read write ))) ( allow systemd_machined_t user_tmp_t ( file ( getattr open read ))) ( allow systemd_machined_t tmpfs_t ( file ( create write open ))) # semodule -i mypolicy.cil # setsebool -P daemons_dump_core 1 # Unfortunately, the bind sub-command always ended up with the following error message: # machinectl bind test /tmp /mnt Failed to bind mount: Can't bind mount on container with user namespacing applied. #
This message is a reminder that Fedora Linux 35 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora Linux 35 on 2022-12-13. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a 'version' of '35'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, change the 'version' to a later Fedora Linux version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora Linux 35 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora Linux, you are encouraged to change the 'version' to a later version prior to this bug being closed.
Based on the above conversation, it sounds like this still affects Fedora 37.
It still affects Fedora 37. My Fedora : Fedora Workstation 37 Selinux : Statistics for policy file: /sys/fs/selinux/policy Policy Version: 33 (MLS enabled) Target Policy: selinux Systemd : 251.10-588 Custom sysctl : kernel.yama.ptrace_scope=1 If I can add my 2 cents. Here are the modules I had to create to get machinectl to work. module machines-policy 1.0; require { type init_var_run_t; type unconfined_service_t; type tmp_t; type systemd_machined_t; type root_t; type container_file_t; class dir { search write }; class file { create getattr ioctl open read write }; class lnk_file read; class cap_userns { kill sys_ptrace }; } #============= systemd_machined_t ============== allow systemd_machined_t container_file_t:dir write; allow systemd_machined_t init_var_run_t:file create; #!!!! This avc can be allowed using the boolean 'daemons_dump_core' allow systemd_machined_t root_t:dir write; allow systemd_machined_t self:cap_userns { kill sys_ptrace }; allow systemd_machined_t tmp_t:file { create open write }; allow systemd_machined_t unconfined_service_t:dir search; allow systemd_machined_t unconfined_service_t:file { getattr ioctl open read }; allow systemd_machined_t unconfined_service_t:lnk_file read; module machines-login 1.0; require { type systemd_machined_t; type unconfined_service_t; type system_dbusd_t; type devpts_t; type tmpfs_t; class cap_userns { setgid setuid sys_admin }; class lnk_file read; class sock_file write; class chr_file { open read write }; class unix_stream_socket connectto; } #============= system_dbusd_t ============== allow system_dbusd_t devpts_t:chr_file { read write }; #============= systemd_machined_t ============== #!!!! This avc can be allowed using the boolean 'daemons_use_tty' allow systemd_machined_t devpts_t:chr_file open; allow systemd_machined_t self:cap_userns { setgid setuid sys_admin }; allow systemd_machined_t tmpfs_t:lnk_file read; allow systemd_machined_t tmpfs_t:sock_file write; allow systemd_machined_t unconfined_service_t:unix_stream_socket connectto; besides I had to enable daemons_use_tty to be able to login with `machinectl login`. But I didn't need to enable daemons_dump_core.
Yes this is still a problem unfortunately. Even run as root which is fun. Just when you find a feature of systemd that seems helpful it doesn't work.
I can confirm (In reply to Aurélien Rouëné from comment #22) > It still affects Fedora 37. > > My Fedora : Fedora Workstation 37 > > Selinux : > Statistics for policy file: /sys/fs/selinux/policy > Policy Version: 33 (MLS enabled) > Target Policy: selinux > > Systemd : 251.10-588 > > Custom sysctl : kernel.yama.ptrace_scope=1 > > > If I can add my 2 cents. > > Here are the modules I had to create to get machinectl to work. > > > module machines-policy 1.0; > > require { > type init_var_run_t; > type unconfined_service_t; > type tmp_t; > type systemd_machined_t; > type root_t; > type container_file_t; > class dir { search write }; > class file { create getattr ioctl open read write }; > class lnk_file read; > class cap_userns { kill sys_ptrace }; > } > > #============= systemd_machined_t ============== > allow systemd_machined_t container_file_t:dir write; > allow systemd_machined_t init_var_run_t:file create; > > #!!!! This avc can be allowed using the boolean 'daemons_dump_core' > allow systemd_machined_t root_t:dir write; > allow systemd_machined_t self:cap_userns { kill sys_ptrace }; > allow systemd_machined_t tmp_t:file { create open write }; > allow systemd_machined_t unconfined_service_t:dir search; > allow systemd_machined_t unconfined_service_t:file { getattr ioctl open read > }; > allow systemd_machined_t unconfined_service_t:lnk_file read; > > > > module machines-login 1.0; > > require { > type systemd_machined_t; > type unconfined_service_t; > type system_dbusd_t; > type devpts_t; > type tmpfs_t; > class cap_userns { setgid setuid sys_admin }; > class lnk_file read; > class sock_file write; > class chr_file { open read write }; > class unix_stream_socket connectto; > } > > #============= system_dbusd_t ============== > allow system_dbusd_t devpts_t:chr_file { read write }; > > #============= systemd_machined_t ============== > > #!!!! This avc can be allowed using the boolean 'daemons_use_tty' > allow systemd_machined_t devpts_t:chr_file open; > allow systemd_machined_t self:cap_userns { setgid setuid sys_admin }; > allow systemd_machined_t tmpfs_t:lnk_file read; > allow systemd_machined_t tmpfs_t:sock_file write; > allow systemd_machined_t unconfined_service_t:unix_stream_socket connectto; > > > besides I had to enable daemons_use_tty to be able to login with `machinectl > login`. But I didn't need to enable daemons_dump_core. I can confirm this solution works for me. Thanks Aurélien!
Closing as a duplicate so additional information are gathered at the same place. *** This bug has been marked as a duplicate of bug 1900869 ***