Created attachment 1625761 [details] Shows shmem definition and error message Description of problem: When creating an ivshmem device in the domain XML for a QEMU guest, starting the VM with default AppArmor profiles does not allow access to the socket and thus the VM does not start. Version-Release number of selected component (if applicable): 5.6.0 How reproducible: 100% of the time Steps to Reproduce: 1. libvirt AppArmor profiles are enforce 2. Create shmem definition in your domain[1] 3. Start domain Actual results: AppArmor rejects "wr" file open operation for ivshmem socket from qemu Domain doesn't start with Permission denied error message Expected results: libvirt adds ivshmem socket to libvirt-[domain-uuid].files with read and write permissions Domain starts successfully Additional info: While I describe AppArmor in this report, I assume the issue is also present on SELinux, since libvirt abstracts AppArmor and SELinux away behind the virSecurityManager. I have personally worked around this by adding "/{dev,run}/shm/my-ivshmem-socket rw," to /etc/apparamor.d/abstractions/libvirt-qemu. On its own it caused another deny to reading /dev/shm so I also added "/{dev,run}/shm/ r,". This did not prevent VM from booting but should probably not need to read /dev/shm. [1]: https://libvirt.org/formatdomain.html#elementsShmem
> On its own it caused another deny to reading /dev/shm so I also added "/{dev,run}/shm/ r,". This did not prevent VM from booting but should probably not need to read /dev/shm. Please ignore this comment, it is the result of me providing pulseaudio args to QEMU and is not handled by libvirt, so it is my responsibility to manage the AppArmor profile for it. Thus it is unrelated to this bug.
CCing some libvirt apparmor maintainers
Confirmed, neither hotplug nor static definitions work atm: ## Hotplug example: ## <shmem name='my_shmem0'> <model type='ivshmem-plain'/> <size unit='M'>4</size> </shmem> virsh attach-device shmem shmem-plain.xml error: internal error: unable to execute QEMU command 'object-add': can't open backing store /dev/shm/my_shmem0 for guest RAM: Permission denied And along that the denial: apparmor="DENIED" operation="mknod" namespace="root//lxd-e_<var-snap-lxd-common-lxd>" profile="libvirt-1c727397-6e4c-483e-aef3-88da04ffaed4" name="/dev/shm/my_shmem0" pid=13949 comm="qemu-system-x86" requested_mask="c" denied_mask="c" fsuid=64055 ouid=64055 For a doorbell you could like: $ ivshmem-server -S /var/lib/libvirt/ivshmem_socket $ chown libvirt-qemu:kvm /var/lib/libvirt/ivshmem_socket <shmem name='shmem_server'> <model type='ivshmem-doorbell'/> <server path='/var/lib/libvirt/ivshmem_socket'/> <msi vectors='32' ioeventfd='on'/> </shmem> Which will cause a similar error: $ virsh attach-device eoan-shmem shmem-db.xml error: internal error: unable to execute QEMU command 'chardev-add': Failed to connect socket /var/lib/libvirt/ivshmem_socket: Permission denied And also has a deny: apparmor="DENIED" operation="connect" namespace="root//lxd-e_<var-snap-lxd-common-lxd>" profile="libvirt-1c727397-6e4c-483e-aef3-88da04ffaed4" name="/var/lib/libvirt/ivshmem_socket" pid=13949 comm="qemu-system-x86" requested_mask="wr" denied_mask="wr" fsuid=64055 ouid=64055 ## Static example ## Same XML content as above: doorbell: error: internal error: process exited while connecting to monitor: 2019-10-16T08:56:14.819083Z qemu-system-x86_64: -chardev socket,id=charshmem0,path=/var/lib/libvirt/ivshmem_socket: Failed to connect socket /var/lib/libvirt/ivshmem_socket: Permission denied apparmor="DENIED" operation="connect" namespace="root//lxd-e_<var-snap-lxd-common-lxd>" profile="libvirt-1c727397-6e4c-483e-aef3-88da04ffaed4" name="/var/lib/libvirt/ivshmem_socket" pid=24560 comm="qemu-system-x86" requested_mask="wr" denied_mask="wr" fsuid=64055 ouid=64055 plain: error: internal error: process exited while connecting to monitor: 2019-10-16T09:00:52.298546Z qemu-system-x86_64: -object memory-backend-file,id=shmmem-shmem0,mem-path=/dev/shm/my_shmem0,size=4194304,share=yes: can't open backing store /dev/shm/my_shmem0 for guest RAM: Permission denied
For hotplug we usually need a backend to the security labeling calls (hopefully there are any already for shmem). For static definitions we usually need virt-aa-helper to render that from the XML which in these cases also needs it to be able to call the logic creating the path from a plain shmem defintion.
TODOs: Static definition: - src/qemu/qemu_command.c: refactor qemuBuildShmemBackendMemProps to be able to reuse the path creation - virt-aa-helper: - iterate nshmems to access shmems[i] - for model VIR_DOMAIN_SHMEM_MODEL_IVSHMEM_PLAIN use refactored shmem path creation - for model VIR_DOMAIN_SHMEM_MODEL_IVSHMEM_DOORBELL check - ->server - ->server.enabled - Use path of ->server.chr.data.nix.path - model VIR_DOMAIN_SHMEM_MODEL_IVSHMEM can have either kind of attributes - but deprecated so could be skipped if ti is too much of a hazzle Hotplug: I haven't seen a security labeling call qemuSecurity*Label of any kind near qemuDomainAttachShmemDevice yet. Therefore Hotplug might need a deeper check and if in fact there are no calls yet those need to be added first. Those would be called for the substring in props (see above refactoring) and the path that qemuMonitorAttachCharDev uses in there.
FYI: sent upstream https://www.redhat.com/archives/libvir-list/2019-October/msg01461.html
FYI - changes in upstream git now commit 36afd1a78ed8e13e33cdf954c6618c178ef777a1 Author: Christian Ehrhardt <christian.ehrhardt> Date: Thu Oct 17 12:48:10 2019 +0200 virt-aa-helper: add rules for shmem devices