Bug 1158154
Summary: | systemd-machined sigterm, cgroups of active VM's trimmed | ||
---|---|---|---|
Product: | [Fedora] Fedora | Reporter: | Major Hayden 🤠<mhayden> |
Component: | systemd | Assignee: | systemd-maint |
Status: | CLOSED WORKSFORME | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
Severity: | unspecified | Docs Contact: | |
Priority: | unspecified | ||
Version: | 21 | CC: | agedosier, antony, berrange, clalancette, itamar, jforbes, johannbg, jsynacek, kchamart, laine, libvirt-maint, lnykryn, msekleta, s, systemd-maint, toni.peltonen, veillard, virt-maint, vpavlin, zbyszek |
Target Milestone: | --- | ||
Target Release: | --- | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | Doc Type: | Bug Fix | |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2015-06-11 14:53:25 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: |
Description
Major Hayden ðŸ¤
2014-10-28 18:18:07 UTC
Or perhaps this is a systemd bug. Should systemd be working with the cgroups under /sys/fs/cgroup/devices for qemu instances? Additional context: When everything works, I find the machine-qemu cgroups in the usual locations (/sys/fs/cgroup/{devices,perf_event,...}) as well as the systemd location (/sys/fs/cgroup/systemd/). Problems occur with libvirt when the cgroups for the qemu instance ONLY appear in the systemd cgroup location. Restarting libvirtd cleans up the bad situation and puts all of the cgroups in the right place. This happening with ~ 5% of builds. I'm working to reproduce it on a node with libvirtd debugging logging enabled. Debug logs: https://gist.github.com/major/ba5191d7e995bfdbd188 My system is running in permissive mode, but here's the AVC logged: type=AVC msg=audit(1414678555.120:169): avc: denied { read } for pid=4511 comm="qemu-system-x86" name="nova" dev="dm-0" ino=36334 scontext=system_u:system_r:svirt_t:s0:c279,c465 tcontext=system_u:object_r:nova_var_lib_t:s0 tclass=lnk_file permissive=1 And the audit2allow output: #============= svirt_t ============== allow svirt_t nova_var_lib_t:lnk_file read; The SELinux AVC looks unrelated, but I'm not 100% sure. Finally caught some activity via inotify events: https://gist.github.com/major/f5fb72aa09030ba68ea7 It looks like something is checking to ensure that /sys/fs/cgroup/devices/machine.slice exists and then it looks for the machine-qemu#####.scope directory a few times. Once all that's done, there's an inotify delete event for the scope directory: <Event dir=True mask=0x40000200 maskname=IN_DELETE|IN_ISDIR name=machine-qemu\x2dinstance\x2d5d8aae0e\x2db66b\x2d4690\x2d8a42\x2def79160e5487.scope path=/sys/fs/cgroup/devices/machine.slice pathname=/sys/fs/cgroup/devices/machine.slice/machine-qemu\x2dinsta nce\x2d5d8aae0e\x2db66b\x2d4690\x2d8a42\x2def79160e5487.scope wd=3 > I haven't been able to figure out if this activity is coming from systemd or libvirtd. Libvirtd claims that it is making the cgroups successfully: https://gist.github.com/major/ba5191d7e995bfdbd188 Changing component to systemd. It appears that systemd is aggressively removing the /sys/fs/cgroup/devices entries while the VM is running. This is very similar to the issue brought up in bug 1139223. Talked to folks in #systemd on freenode and an email to the systemd-devel ML was recommended: http://lists.freedesktop.org/archives/systemd-devel/2014-November/024886.html Fixed in git. (0cd385d31814c8c1bc0c81d11ef321036b8b0921) |