Bug 2188304
Summary: | 'systemd-tmpfiles --create' in Toolbx container gives lots of errors/warnings | ||
---|---|---|---|
Product: | [Fedora] Fedora | Reporter: | Jens Petersen <petersen> |
Component: | toolbox | Assignee: | Debarshi Ray <debarshir> |
Status: | ASSIGNED --- | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
Severity: | medium | Docs Contact: | |
Priority: | unspecified | ||
Version: | rawhide | CC: | debarshir, fedoraproject, filbranden, go-sig, harrymichal, lnykryn, msekleta, ogutierr, patrick, ryncsn, sumukher, systemd-maint, yuwatana, zbyszek |
Target Milestone: | --- | ||
Target Release: | --- | ||
Hardware: | x86_64 | ||
OS: | Linux | ||
Whiteboard: | |||
Fixed In Version: | Doc Type: | If docs needed, set a value | |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | Type: | --- | |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: |
Description
Jens Petersen
2023-04-20 12:12:28 UTC
I think this can also be reproduced directly with `podman run -it --rm fedora-toolbox dnf install systemd`. (Also I think originally mentioned Silverblue on #fedora-devel, but the above dnf results are from a F38 Workstation VM.) (In reply to Jens Petersen from comment #1) > I think this can also be reproduced directly with `podman run -it --rm > fedora-toolbox dnf install systemd`. This doesn't reproduce here. But it works reproduces fine with the toolbox. Those error messages are from 'systemd-tmpfiles --create 'which is executed by the %transfiletriggerin scriptlet of systemd. This is easy enough to reproduce manually: $ sudo systemd-tmpfiles --create fchownat() of /run/systemd/sessions failed: Operation not permitted fchownat() of /run/systemd/users failed: Operation not permitted fchownat() of /var/lib/systemd/coredump failed: Read-only file system fchownat() of /tmp failed: Operation not permitted Setting access ACL "u::rwx,g::r-x,g:adm:r-x,g:wheel:r-x,g:4294967295:r-x,g:4294967295:r-x,m::r-x,o::r-x" on /var/log/journal failed: Read-only file system Failed to re-open '/var/log/journal': Operation not permitted fchownat() of /var/log/journal failed: Read-only file system Setting access ACL "u::rwx,g::r-x,g:adm:r-x,g:wheel:r-x,g:4294967295:r-x,g:4294967295:r-x,m::r-x,o::r-x" on /var/log/journal/3a9d668b4db749398a4a5e78a03bffa5 failed: Read-only file system Failed to re-open '/var/log/journal/3a9d668b4db749398a4a5e78a03bffa5': Operation not permitted fchownat() of /var/log/journal/3a9d668b4db749398a4a5e78a03bffa5 failed: Read-only file system Failed to re-open '/var/log/journal/remote': Operation not permitted fchownat() of /dev/snd/seq failed: Operation not permitted fchownat() of /dev/snd/timer failed: Operation not permitted fchownat() of /dev/loop-control failed: Operation not permitted fchownat() of /dev/kvm failed: Operation not permitted fchownat() of /dev/vhost-net failed: Operation not permitted fchownat() of /dev/vhost-vsock failed: Operation not permitted Setting access ACL "u::rw-,g::r-x,g:adm:r--,g:wheel:r--,g:4294967295:r--,g:4294967295:r--,m::r--,o::---" on /var/log/journal/3a9d668b4db749398a4a5e78a03bffa5/system.journal failed: Read-only file system fchownat() of /var/log/journal/3a9d668b4db749398a4a5e78a03bffa5/system.journal failed: Read-only file system fchownat() of /sys/kernel/security/tpm0/binary_bios_measurements failed: Operation not permitted fchownat() of /sys/kernel/security/ima/binary_runtime_measurements failed: Operation not permitted In the container: $ ls -ld /run/systemd/sessions drwxr-xr-x 2 nobody nobody 80 Apr 20 16:38 /run/systemd/sessions $ ls -l /sys/kernel/security/ima/binary_runtime_measurements -r--r----- 1 nobody nobody 0 Apr 18 15:01 /sys/kernel/security/ima/binary_runtime_measurements So this is a result of confusion about what the container is. It has parts of the host filesystem mounted into the namespace, so it's like a chroot, but it based on a real container and systemd detects it as such. $ systemd-detect-virt podman $ strace -e newfstatat systemd-detect-virt --chroot newfstatat(AT_FDCWD, "/proc/1/root", 0x7ffeca036830, 0) = -1 EACCES (Permission denied) Failed to check for chroot() environment: Permission denied I'm not sure what to do here and how this hybrid design is supposed to work. I don't think there's anything wrong with what systemd is doing: we're told to execute some standard configuration by the scriptlet and config files in /etc, and we cannot do it because some file have unexpected ownership and permissions are denied. And in general, there are countless scriptlets that touch paths in the fs and don't expect them to be owned by nobody. One option would be for toolbox to inject an override to for systemd-tmpfiles to not touch certain paths. This override would have to be match the paths that are mounted in from the host. I'll reassign this to toolbox for comments. We had hit something similar, but not exactly the same, before with paths marked with %attr(...) in RPMs. eg., the filesystem RPM has `ghost %attr(555,root,root) /proc`. See: https://github.com/containers/toolbox/issues/643 https://bugzilla.redhat.com/show_bug.cgi?id=1830830 That was handled by placing a /usr/lib/rpm/macros.d/macros.toolbox with `%_netsharedpath` containing all such paths inside the Toolbx containers: https://github.com/containers/toolbox/commit/7542f5fc867b57bf3dc67bbae02cc09ccc0b5df2 That won't help in this case. Thanks for digging into this, Zbigniew! (In reply to Zbigniew Jędrzejewski-Szmek from comment #3) > > One option would be for toolbox to inject an override to for > systemd-tmpfiles to not touch certain paths. This override would have to be > match the paths that are mounted in from the host. This sounds reasonable to me. Since Toolbx does the bind mounts, it should also handle the overrides, so that everything can be tested and updated without having to co-ordinate it across multiple Git repositorie and downstream packages. However, I don't know how the overrides will exactly work? Toolbx will place some empty files in /run/tmpfiles.d that match the names of the files in /usr/lib/tmpfiles.d ? Yes: /run/tmpfiles.d/systemd.conf /run/tmpfiles.d/static-nodes-permissions.conf /run/tmpfiles.d/static-nodes.conf (In reply to Zbigniew Jędrzejewski-Szmek from comment #6) > Yes: > /run/tmpfiles.d/systemd.conf > /run/tmpfiles.d/static-nodes-permissions.conf > /run/tmpfiles.d/static-nodes.conf Thanks for that list, Zbigniew! I see that tmpfiles.d/systemd.conf.in contains a lot of things. As far as systemd is concerned, Toolbx only bind mounts these locations from the host: * /dev * /run/systemd/journal * /run/systemd/resolve * /run/systemd/sessions * /run/systemd/system * /run/systemd/users * /var/lib/systemd/coredump * /var/log/journal ... to have working coredumpctl, journalctl, logging, DNS resolution, and running non-nested Wayland display servers. I am wondering if it will be a good idea to override the whole of systemd.conf. Maybe, Toolbx should bind mount all those other locations too? Or maybe, Toolbx should filter out the bind mounted locations from the host's systemd.conf at run-time and install this filtered version as an override inside the container? Everything in tmpfiles.d/static-nodes-permissions.conf.in comes from /dev which is bind mounted from the host. So that's fine. I see that static-nodes.conf is supposed to be generated by kmod-static-nodes.service but I don't have it on my Fedora host. So, I will try to assume that it's safe to suppress it. :) (In reply to Zbigniew Jędrzejewski-Szmek from comment #6) > Yes: > /run/tmpfiles.d/systemd.conf > /run/tmpfiles.d/static-nodes-permissions.conf > /run/tmpfiles.d/static-nodes.conf Also, since these filenames seem to be defined in the upstream systemd sources, I suppose I can rely on their names on other distributions, even those outside the Fedora universe, right? > since these filenames seem to be defined in the upstream systemd sources, I suppose I can rely on their names on other distributions, even those outside the Fedora universe, right? Yes, those names are defined upstream and something that we very much try not to change, because that breaks overrides. Unfortunately, there's a strong force in the other direction too: people ask for files to be broken up to make it easier to override pieces. So occasionally the names (or at least the placement of some lines in the config) do change. Any such change happens upstream at a release boundary, with notices in NEWS. > I see that static-nodes.conf is supposed to be generated by kmod-static-nodes.service but I don't have it on my Fedora host. It is in /run/tmpfiles.d/static-nodes.conf. But all those files are in /dev, so it would have to be suppressed too. > I am wondering if it will be a good idea to override the whole of systemd.conf. Maybe, Toolbx should bind mount all those other locations too? Or maybe, Toolbx should filter out the bind mounted locations from the host's systemd.conf at run-time and install this filtered version as an override inside the container? All the locations defined in systemd.conf are closely tied to the functioning of the host's pid1, so I don't think the container should ever create those files. Toolbx should either bind-mount the subset that makes sense for the container, and suppress the file as a whole. Okay, understood. Thanks for taking the time to explain all that in detail. This message is a reminder that Fedora Linux 38 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora Linux 38 on 2024-05-21. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a 'version' of '38'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, change the 'version' to a later Fedora Linux version. Note that the version field may be hidden. Click the "Show advanced fields" button if you do not see it. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora Linux 38 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora Linux, you are encouraged to change the 'version' to a later version prior to this bug being closed. |