+++ This bug was initially created as a clone of Bug #1665400 +++ Description of problem: Libvirt reports <sev supported='no'/> even though SEV is enabled on the platform, i.e. both the cpuflags contain 'sev' and /dev/sev exists on the filesystem. Version-Release number of selected component (if applicable): 4.5.0-17.module+el8+2625+db702f9d How reproducible: Steps to Reproduce: 1. enable SEV on the platform so that /dev/sev appears on the filesystem 2. install libvirt, launch libvirtd 3. run virsh domcapabilities, the output contains: ... <sev supported='no'/> ... Actual results: Libvirt is unable to launch a SEV VM because it reports the feature as unsupported Expected results: Libvirt is able to launch a SEV VM Additional info: 1) Going through libvirt debug logs, one can also see that the error is coming from QEMU when probing for capabilities: debug : qemuMonitorJSONIOProcessLine:197 : Line [{"id": "libvirt-53", "error": {"class": "GenericError", "desc": "SEV feature is not available"}}] 2) Even though 1) suggests this might be a bug in QEMU, the problem are the default permissions on /dev/sev: # ls -l /dev/sev crw-------. 1 root root which is a problem because when libvirt probes QEMU for capabilities, the process runs as qemu:qemu by default. --- Additional comment from Erik Skultety on 2019-02-01 12:51:25 CET --- Fixed upstream by: commit a2d3dea9d41dba313d9566120a8ec9d358567bd0 Refs: v5.0.0-198-ga2d3dea9d4 Author: Erik Skultety <eskultet> AuthorDate: Thu Jan 24 10:33:01 2019 +0100 Commit: Erik Skultety <eskultet> CommitDate: Fri Feb 1 12:44:28 2019 +0100 qemu: caps: Use CAP_DAC_OVERRIDE for probing to avoid permission issues This is mainly about /dev/sev and its default permissions 0600. Of course, rule of 'tinfoil' would be that we can't trust anything, but the probing code in QEMU is considered safe from security's perspective + we can't create an udev rule for this at the moment, because ioctls and file system permissions aren't cross-checked in kernel and therefore a user with read permissions could issue a 'privileged' operation on SEV which is currently only limited to root. https://bugzilla.redhat.com/show_bug.cgi?id=1665400 Signed-off-by: Erik Skultety <eskultet> Reviewed-by: Daniel P. Berrangé <berrange>
Test with upstream libvirt build on RHEL8: 1. libvirt can probe sev supported status without change the /dev/sev file permissions: # virsh domcapabilities <sev supported='yes'> <cbitpos>47</cbitpos> <reducedPhysBits>1</reducedPhysBits> </sev> # ll -Z /dev/sev crw-------. 1 root root system_u:object_r:device_t:s0 10, 58 Feb 1 03:13 /dev/sev 2. still require user manually change /dev/sev selinux label before start guest: # virsh start ovmf.rhel7.sev.q35 error: Failed to start domain ovmf.rhel7.sev.q35 error: internal error: process exited while connecting to monitor: 2019-02-02T07:28:18.990508Z qemu-kvm: sev_guest_init: Failed to open /dev/sev 'Permission denied' 2019-02-02T07:28:18.993993Z qemu-kvm: failed to initialize KVM: Operation not permitted AVC error: type=AVC msg=audit(1549079965.416:866): avc: denied { read write } for pid=39969 comm="qemu-kvm" name="sev" dev="tmpfs" ino=302314 scontext=system_u:system_r:svirt_t:s0:c227,c466 tcontext=system_u:object_r:device_t:s0 tclass=chr_file permissive=0 3. /dev/sev DAC label only changed in guest namesapce: # nsenter --target 46747 -m ls -lZ /dev/sev crw-------. 1 qemu qemu system_u:object_r:device_t:s0 10, 58 Feb 2 02:32 /dev/sev # ll -Z /dev/sev crw-------. 1 root root system_u:object_r:device_t:s0 10, 58 Feb 1 03:13 /dev/sev 4. libvirt won't relabel /dev/sev if forbid libvirt run qemu in mount namespace (set namespaces = [ ] in qemu.conf)
(In reply to Luyao Huang from comment #4) > Test with upstream libvirt build on RHEL8: > > 1. libvirt can probe sev supported status without change the /dev/sev file > permissions: > > > # virsh domcapabilities > <sev supported='yes'> > <cbitpos>47</cbitpos> > <reducedPhysBits>1</reducedPhysBits> > </sev> > > # ll -Z /dev/sev > crw-------. 1 root root system_u:object_r:device_t:s0 10, 58 Feb 1 03:13 > /dev/sev > > 2. still require user manually change /dev/sev selinux label before start > guest: > > # virsh start ovmf.rhel7.sev.q35 > error: Failed to start domain ovmf.rhel7.sev.q35 > error: internal error: process exited while connecting to monitor: > 2019-02-02T07:28:18.990508Z qemu-kvm: sev_guest_init: Failed to open > /dev/sev 'Permission denied' > 2019-02-02T07:28:18.993993Z qemu-kvm: failed to initialize KVM: Operation > not permitted > > AVC error: > > type=AVC msg=audit(1549079965.416:866): avc: denied { read write } for > pid=39969 comm="qemu-kvm" name="sev" dev="tmpfs" ino=302314 > scontext=system_u:system_r:svirt_t:s0:c227,c466 > tcontext=system_u:object_r:device_t:s0 tclass=chr_file permissive=0 This is not libvirt's fault, I'm planning on filing a BZ on selinux-policy to add a rule for /dev/sev, so that svirt_t can access it > > 3. /dev/sev DAC label only changed in guest namesapce: > > # nsenter --target 46747 -m ls -lZ /dev/sev > crw-------. 1 qemu qemu system_u:object_r:device_t:s0 10, 58 Feb 2 02:32 > /dev/sev > > # ll -Z /dev/sev > crw-------. 1 root root system_u:object_r:device_t:s0 10, 58 Feb 1 03:13 > /dev/sev > > 4. libvirt won't relabel /dev/sev if forbid libvirt run qemu in mount > namespace (set namespaces = [ ] in qemu.conf) ^This is by design, we don't want to expose /dev/sev to all qemu processes, only to those that need SEV and since libvirt runs QEMU in a namespace by default, I explicitly forbade relabeling without namespaces, leaving the decision to the platform admin.
(In reply to Erik Skultety from comment #5) > (In reply to Luyao Huang from comment #4) > > Test with upstream libvirt build on RHEL8: > > > > This is not libvirt's fault, I'm planning on filing a BZ on selinux-policy > to add a rule for /dev/sev, so that svirt_t can access it > > > > > 3. /dev/sev DAC label only changed in guest namesapce: > > > > # nsenter --target 46747 -m ls -lZ /dev/sev > > crw-------. 1 qemu qemu system_u:object_r:device_t:s0 10, 58 Feb 2 02:32 > > /dev/sev > > > > # ll -Z /dev/sev > > crw-------. 1 root root system_u:object_r:device_t:s0 10, 58 Feb 1 03:13 > > /dev/sev > > > > 4. libvirt won't relabel /dev/sev if forbid libvirt run qemu in mount > > namespace (set namespaces = [ ] in qemu.conf) > > ^This is by design, we don't want to expose /dev/sev to all qemu processes, > only to those that need SEV and since libvirt runs QEMU in a namespace by > default, I explicitly forbade relabeling without namespaces, leaving the > decision to the platform admin. I see, thanks a lot for your clearly explanation !
Verify this bug with libvirt-5.0.0-2.module+el8+2779+3106feb7.x86_64: 1. check /dev/sev device permission: # ll -Z /dev/sev crw-------. 1 root root system_u:object_r:sev_device_t:s0 10, 58 Feb 9 13:14 /dev/sev 2. install libvirt and check domcapabilities output: # virsh domcapabilities <sev supported='yes'> <cbitpos>47</cbitpos> <reducedPhysBits>1</reducedPhysBits> </sev> 3. start a guest with sev launchSecurity: # virsh dumpxml sev-q35 <launchSecurity type='sev'> <cbitpos>47</cbitpos> <reducedPhysBits>1</reducedPhysBits> <policy>0x0001</policy> </launchSecurity> # virsh start sev-q35 Domain sev-q35 started 4. check /dev/sev device label: # ll -Z /dev/sev crw-------. 1 root root system_u:object_r:sev_device_t:s0 10, 58 Feb 9 13:14 /dev/sev 5. set namespaces = [ ] in qemu.conf to disable guest mount namespace 6. restart libvirtd and start sev guest: # virsh start sev-q35 error: Failed to start domain sev-q35 error: internal error: process exited while connecting to monitor: 2019-02-11T03:17:25.192428Z qemu-kvm: sev_guest_init: Failed to open /dev/sev 'Permission denied' 2019-02-11T03:17:25.196308Z qemu-kvm: failed to initialize KVM: Operation not permitted 7. check /dev/sev device label: # ll -Z /dev/sev crw-------. 1 root root system_u:object_r:sev_device_t:s0 10, 58 Feb 9 13:14 /dev/sev 8. check domcapabilities output in non-root user: $ virsh domcapabilities <sev supported='no'/>
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:1293