Bug 1761645

Summary: QEMU shmem socket not flagged in AppArmor
Product: [Community] Virtualization Tools Reporter: Jonathan Rubenstein <jrubcop>
Component: libvirtAssignee: Libvirt Maintainers <libvirt-maint>
Status: CLOSED NEXTRELEASE QA Contact:
Severity: low Docs Contact:
Priority: unspecified    
Version: unspecifiedCC: crobinso, jamie, jdenemar, jfehlig, libvirt-maint, paelzer, tburke
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-5.10.0 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-11-21 08:11:20 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Shows shmem definition and error message none

Description Jonathan Rubenstein 2019-10-15 01:48:03 UTC
Created attachment 1625761 [details]
Shows shmem definition and error message

Description of problem:
When creating an ivshmem device in the domain XML for a QEMU guest, starting the VM with default AppArmor profiles does not allow access to the socket and thus the VM does not start.


Version-Release number of selected component (if applicable):
5.6.0


How reproducible:
100% of the time

Steps to Reproduce:
1. libvirt AppArmor profiles are enforce
2. Create shmem definition in your domain[1] 
3. Start domain

Actual results:
AppArmor rejects "wr" file open operation for ivshmem socket from qemu
Domain doesn't start with Permission denied error message

Expected results:
libvirt adds ivshmem socket to libvirt-[domain-uuid].files with read and write permissions
Domain starts successfully

Additional info:
While I describe AppArmor in this report, I assume the issue is also present on SELinux, since libvirt abstracts AppArmor and SELinux away behind the virSecurityManager.

I have personally worked around this by adding "/{dev,run}/shm/my-ivshmem-socket rw," to /etc/apparamor.d/abstractions/libvirt-qemu.

On its own it caused another deny to reading /dev/shm so I also added "/{dev,run}/shm/ r,". This did not prevent VM from booting but should probably not need to read /dev/shm.

[1]: https://libvirt.org/formatdomain.html#elementsShmem

Comment 1 Jonathan Rubenstein 2019-10-15 02:51:23 UTC
> On its own it caused another deny to reading /dev/shm so I also added "/{dev,run}/shm/ r,". This did not prevent VM from booting but should probably not need to read /dev/shm.

Please ignore this comment, it is the result of me providing pulseaudio args to QEMU and is not handled by libvirt, so it is my responsibility to manage the AppArmor profile for it. Thus it is unrelated to this bug.

Comment 2 Cole Robinson 2019-10-15 13:41:55 UTC
CCing some libvirt apparmor maintainers

Comment 3 Christian Ehrhardt 2019-10-16 09:01:45 UTC
Confirmed, neither hotplug nor static definitions work atm:

## Hotplug example: ##
  <shmem name='my_shmem0'>
    <model type='ivshmem-plain'/>
    <size unit='M'>4</size>
  </shmem>

virsh attach-device shmem shmem-plain.xml
error: internal error: unable to execute QEMU command 'object-add': can't open backing store /dev/shm/my_shmem0 for guest RAM: Permission denied

And along that the denial:
apparmor="DENIED" operation="mknod" namespace="root//lxd-e_<var-snap-lxd-common-lxd>" profile="libvirt-1c727397-6e4c-483e-aef3-88da04ffaed4" name="/dev/shm/my_shmem0" pid=13949 comm="qemu-system-x86" requested_mask="c" denied_mask="c" fsuid=64055 ouid=64055

For a doorbell you could like:
$ ivshmem-server -S /var/lib/libvirt/ivshmem_socket
$ chown libvirt-qemu:kvm /var/lib/libvirt/ivshmem_socket
  <shmem name='shmem_server'>
    <model type='ivshmem-doorbell'/>
    <server path='/var/lib/libvirt/ivshmem_socket'/>
    <msi vectors='32' ioeventfd='on'/>
  </shmem>

Which will cause a similar error:
$ virsh attach-device eoan-shmem shmem-db.xml
error: internal error: unable to execute QEMU command 'chardev-add': Failed to connect socket /var/lib/libvirt/ivshmem_socket: Permission denied

And also has a deny:
apparmor="DENIED" operation="connect" namespace="root//lxd-e_<var-snap-lxd-common-lxd>" profile="libvirt-1c727397-6e4c-483e-aef3-88da04ffaed4" name="/var/lib/libvirt/ivshmem_socket" pid=13949 comm="qemu-system-x86" requested_mask="wr" denied_mask="wr" fsuid=64055 ouid=64055


## Static example ##
Same XML content as above:
doorbell:
error: internal error: process exited while connecting to monitor: 2019-10-16T08:56:14.819083Z qemu-system-x86_64: -chardev socket,id=charshmem0,path=/var/lib/libvirt/ivshmem_socket: Failed to connect socket /var/lib/libvirt/ivshmem_socket: Permission denied
apparmor="DENIED" operation="connect" namespace="root//lxd-e_<var-snap-lxd-common-lxd>" profile="libvirt-1c727397-6e4c-483e-aef3-88da04ffaed4" name="/var/lib/libvirt/ivshmem_socket" pid=24560 comm="qemu-system-x86" requested_mask="wr" denied_mask="wr" fsuid=64055 ouid=64055

plain:
error: internal error: process exited while connecting to monitor: 2019-10-16T09:00:52.298546Z qemu-system-x86_64: -object memory-backend-file,id=shmmem-shmem0,mem-path=/dev/shm/my_shmem0,size=4194304,share=yes: can't open backing store /dev/shm/my_shmem0 for guest RAM: Permission denied

Comment 4 Christian Ehrhardt 2019-10-16 09:04:46 UTC
For hotplug we usually need a backend to the security labeling calls (hopefully there are any already for shmem).
For static definitions we usually need virt-aa-helper to render that from the XML which in these cases also needs it to be able to call the logic creating the path from a plain shmem defintion.

Comment 5 Christian Ehrhardt 2019-10-16 13:07:22 UTC
TODOs:
Static definition:
- src/qemu/qemu_command.c: refactor qemuBuildShmemBackendMemProps to be able to reuse the path creation
- virt-aa-helper:
  - iterate nshmems to access shmems[i]
  - for model VIR_DOMAIN_SHMEM_MODEL_IVSHMEM_PLAIN use refactored shmem path creation
  - for model VIR_DOMAIN_SHMEM_MODEL_IVSHMEM_DOORBELL check
    - ->server
    - ->server.enabled
    - Use path of ->server.chr.data.nix.path
  - model VIR_DOMAIN_SHMEM_MODEL_IVSHMEM can have either kind of attributes
    - but deprecated so could be skipped if ti is too much of a hazzle

Hotplug:
I haven't seen a security labeling call qemuSecurity*Label of any kind near qemuDomainAttachShmemDevice yet.
Therefore Hotplug might need a deeper check and if in fact there are no calls yet those need to be added first.
Those would be called for the substring in props (see above refactoring) and the path that qemuMonitorAttachCharDev uses in there.

Comment 6 Christian Ehrhardt 2019-10-22 12:20:38 UTC
FYI: sent upstream https://www.redhat.com/archives/libvir-list/2019-October/msg01461.html

Comment 7 Christian Ehrhardt 2019-11-21 07:38:23 UTC
FYI - changes in upstream git now

commit 36afd1a78ed8e13e33cdf954c6618c178ef777a1
Author: Christian Ehrhardt <christian.ehrhardt>
Date:   Thu Oct 17 12:48:10 2019 +0200

    virt-aa-helper: add rules for shmem devices