I'm not sure if this is a bug or intentional behavior, so I'm looking for some advice.
$ uname -a
Linux worklaptop 5.3.0-1.fc31.x86_64 #1 SMP Mon Sep 16 12:34:42 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
$ rpm -q systemd
$ mount | grep cgroup
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime,seclabel,nsdelegate)
$ cat /proc/self/cgroup
After I run a mock build (via fedpkg mockbuild), the last bit changes
$ cat /proc/self/cgroup
This broke libvirt VM startup: https://bugzilla.redhat.com/show_bug.cgi?id=1751120
podman container startup using crun also seems broken:
$ podman run -it alpine sh
Error: creating cgroup directory '/firstname.lastname@example.org/user.slice/libpod-69952a2d396138de9ffa0d1a5e096ae2d6382c6e17c93a0baf53b762a8f83ff9.scope': Permission denied: OCI runtime error
So is this a systemd bug, or do both those apps need to adjust their expectations? Also is their some way to 'undo' this situation without rebooting the host, as a temporary workaround?
mock doesn't matter here, plain systemd-nspawn invocation is enough to cause this
(with the right image).
systemd-nspawn has code to detect if the image it is about to start has an init that
has support for the unified cgroup hierarchy. Specifically, it checks if systemd>230
is installed. If it detects no support, if falls back to mounting legacy hierarchy.
This is a bit ugly, but in the beginning almost nothing had support for the unified
hierarchy, and we didn't want to break spawning of existing images. Maybe this should
be revisited, but I don't think there's any easy solution if we want to keep compatibility
with older images.
With recent buildroot dependency cleanups, systemd may no longer be installed by mock,
so suddenly systemd-nspawn detects no support and falls back to legacy in the booted
image. Once that happens, I don't think it is possible to undo without rebooting.
It is possible to override the detection by setting UNIFIED_CGROUP_HIERARCHY=yes.
You can use that as a temporary work-around.
That said, mock actually calls systemd-nspawn with --as-pid2/-a. In this mode, nspawn
"knows" that a compatible init will be running (because it is it), so we should
skip the detection based on the image. I'll prep a patch.
FEDORA-2019-5cb48e5111 has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2019-5cb48e5111
systemd-243-3.gitef67743.fc31 has been pushed to the Fedora 31 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-5cb48e5111
RHBZ#1756143 not fixed. I did ~40 builds and now it hangs again.
This bug is about a very simple thing: whether 'systemd-nspawn --as-pid2' mounts
v1 or v2 hierarchy in the container. It is fully deterministic, without any race conditions,
so if you're seeing hangs when doing a large number of builds, then it's most likely
some different thing.
Oh, sorry. Maybe we should reopen #1754807 then?
@Vascom wrote that it still happens as well.
#1754807 still persist.
systemd-243-3.gitef67743.fc31 has been pushed to the Fedora 31 stable repository. If problems still persist, please make note of it in this bug report.