Bug 1671791 - Libvirt reports SEV feature as unsupported even though the platform provides the feature
Summary: Libvirt reports SEV feature as unsupported even though the platform provides ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux Advanced Virtualization
Classification: Red Hat
Component: libvirt
Version: 8.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: rc
: 8.0
Assignee: Erik Skultety
QA Contact: Luyao Huang
URL:
Whiteboard:
Depends On: 1665400 1665469 1672188
Blocks: 1654309 1501607
TreeView+ depends on / blocked
 
Reported: 2019-02-01 16:39 UTC by Erik Skultety
Modified: 2019-05-29 16:05 UTC (History)
3 users (show)

Fixed In Version: libvirt-5.0.0-2.el8
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1665400
Environment:
Last Closed: 2019-05-29 16:05:30 UTC
Type: Bug
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:1293 None None None 2019-05-29 16:05:55 UTC

Description Erik Skultety 2019-02-01 16:39:05 UTC
+++ This bug was initially created as a clone of Bug #1665400 +++

Description of problem:
Libvirt reports <sev supported='no'/> even though SEV is enabled on the platform, i.e. both the cpuflags contain 'sev' and /dev/sev exists on the filesystem. 

Version-Release number of selected component (if applicable):
4.5.0-17.module+el8+2625+db702f9d

How reproducible:


Steps to Reproduce:
1. enable SEV on the platform so that /dev/sev appears on the filesystem
2. install libvirt, launch libvirtd
3. run virsh domcapabilities, the output contains:
...
<sev supported='no'/>
...

Actual results:
Libvirt is unable to launch a SEV VM because it reports the feature as unsupported

Expected results:
Libvirt is able to launch a SEV VM

Additional info:
1) Going through libvirt debug logs, one can also see that the error is coming from QEMU when probing for capabilities:

debug : qemuMonitorJSONIOProcessLine:197 : Line [{"id": "libvirt-53", "error": {"class": "GenericError", "desc": "SEV feature is not available"}}]

2) Even though 1) suggests this might be a bug in QEMU, the problem are the default permissions on /dev/sev:

# ls -l /dev/sev
crw-------. 1 root root

which is a problem because when libvirt probes QEMU for capabilities, the process runs as qemu:qemu by default.

--- Additional comment from Erik Skultety on 2019-02-01 12:51:25 CET ---

Fixed upstream by:

commit a2d3dea9d41dba313d9566120a8ec9d358567bd0                                                                                                                               
Refs: v5.0.0-198-ga2d3dea9d4                                                                                             
Author:     Erik Skultety <eskultet@redhat.com>                                                                                                                               
AuthorDate: Thu Jan 24 10:33:01 2019 +0100                                                                                                                                    
Commit:     Erik Skultety <eskultet@redhat.com>                                                                                                                               
CommitDate: Fri Feb 1 12:44:28 2019 +0100                                                                                                                                     

    qemu: caps: Use CAP_DAC_OVERRIDE for probing to avoid permission issues

    This is mainly about /dev/sev and its default permissions 0600. Of
    course, rule of 'tinfoil' would be that we can't trust anything, but the
    probing code in QEMU is considered safe from security's perspective + we
    can't create an udev rule for this at the moment, because ioctls and
    file system permissions aren't cross-checked in kernel and therefore a
    user with read permissions could issue a 'privileged' operation on SEV
    which is currently only limited to root.

    https://bugzilla.redhat.com/show_bug.cgi?id=1665400

    Signed-off-by: Erik Skultety <eskultet@redhat.com>                                                                                                                        
    Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>

Comment 4 Luyao Huang 2019-02-02 07:42:50 UTC
Test with upstream libvirt build on RHEL8:

1. libvirt can probe sev supported status without change the /dev/sev file permissions:


# virsh domcapabilities
    <sev supported='yes'>
      <cbitpos>47</cbitpos>
      <reducedPhysBits>1</reducedPhysBits>
    </sev>

# ll -Z /dev/sev 
crw-------. 1 root root system_u:object_r:device_t:s0 10, 58 Feb  1 03:13 /dev/sev

2. still require user manually change /dev/sev selinux label before start guest:

# virsh start ovmf.rhel7.sev.q35
error: Failed to start domain ovmf.rhel7.sev.q35
error: internal error: process exited while connecting to monitor: 2019-02-02T07:28:18.990508Z qemu-kvm: sev_guest_init: Failed to open /dev/sev 'Permission denied'
2019-02-02T07:28:18.993993Z qemu-kvm: failed to initialize KVM: Operation not permitted

AVC error:

type=AVC msg=audit(1549079965.416:866): avc:  denied  { read write } for  pid=39969 comm="qemu-kvm" name="sev" dev="tmpfs" ino=302314 scontext=system_u:system_r:svirt_t:s0:c227,c466 tcontext=system_u:object_r:device_t:s0 tclass=chr_file permissive=0

3. /dev/sev DAC label only changed in guest namesapce:

# nsenter --target 46747 -m ls -lZ /dev/sev
crw-------. 1 qemu qemu system_u:object_r:device_t:s0 10, 58 Feb  2 02:32 /dev/sev

# ll -Z /dev/sev
crw-------. 1 root root system_u:object_r:device_t:s0 10, 58 Feb  1 03:13 /dev/sev

4. libvirt won't relabel /dev/sev if forbid libvirt run qemu in mount namespace (set namespaces = [ ] in qemu.conf)

Comment 5 Erik Skultety 2019-02-04 07:51:00 UTC
(In reply to Luyao Huang from comment #4)
> Test with upstream libvirt build on RHEL8:
> 
> 1. libvirt can probe sev supported status without change the /dev/sev file
> permissions:
> 
> 
> # virsh domcapabilities
>     <sev supported='yes'>
>       <cbitpos>47</cbitpos>
>       <reducedPhysBits>1</reducedPhysBits>
>     </sev>
> 
> # ll -Z /dev/sev 
> crw-------. 1 root root system_u:object_r:device_t:s0 10, 58 Feb  1 03:13
> /dev/sev
> 
> 2. still require user manually change /dev/sev selinux label before start
> guest:
> 
> # virsh start ovmf.rhel7.sev.q35
> error: Failed to start domain ovmf.rhel7.sev.q35
> error: internal error: process exited while connecting to monitor:
> 2019-02-02T07:28:18.990508Z qemu-kvm: sev_guest_init: Failed to open
> /dev/sev 'Permission denied'
> 2019-02-02T07:28:18.993993Z qemu-kvm: failed to initialize KVM: Operation
> not permitted
> 
> AVC error:
> 
> type=AVC msg=audit(1549079965.416:866): avc:  denied  { read write } for 
> pid=39969 comm="qemu-kvm" name="sev" dev="tmpfs" ino=302314
> scontext=system_u:system_r:svirt_t:s0:c227,c466
> tcontext=system_u:object_r:device_t:s0 tclass=chr_file permissive=0

This is not libvirt's fault, I'm planning on filing a BZ on selinux-policy to add a rule for /dev/sev, so that svirt_t can access it

> 
> 3. /dev/sev DAC label only changed in guest namesapce:
> 
> # nsenter --target 46747 -m ls -lZ /dev/sev
> crw-------. 1 qemu qemu system_u:object_r:device_t:s0 10, 58 Feb  2 02:32
> /dev/sev
> 
> # ll -Z /dev/sev
> crw-------. 1 root root system_u:object_r:device_t:s0 10, 58 Feb  1 03:13
> /dev/sev
> 
> 4. libvirt won't relabel /dev/sev if forbid libvirt run qemu in mount
> namespace (set namespaces = [ ] in qemu.conf)

^This is by design, we don't want to expose /dev/sev to all qemu processes, only to those that need SEV and since libvirt runs QEMU in a namespace by default, I explicitly forbade relabeling without namespaces, leaving the decision to the platform admin.

Comment 7 Luyao Huang 2019-02-11 01:59:10 UTC
(In reply to Erik Skultety from comment #5)
> (In reply to Luyao Huang from comment #4)
> > Test with upstream libvirt build on RHEL8:
> > 

> 
> This is not libvirt's fault, I'm planning on filing a BZ on selinux-policy
> to add a rule for /dev/sev, so that svirt_t can access it
> 
> > 
> > 3. /dev/sev DAC label only changed in guest namesapce:
> > 
> > # nsenter --target 46747 -m ls -lZ /dev/sev
> > crw-------. 1 qemu qemu system_u:object_r:device_t:s0 10, 58 Feb  2 02:32
> > /dev/sev
> > 
> > # ll -Z /dev/sev
> > crw-------. 1 root root system_u:object_r:device_t:s0 10, 58 Feb  1 03:13
> > /dev/sev
> > 
> > 4. libvirt won't relabel /dev/sev if forbid libvirt run qemu in mount
> > namespace (set namespaces = [ ] in qemu.conf)
> 
> ^This is by design, we don't want to expose /dev/sev to all qemu processes,
> only to those that need SEV and since libvirt runs QEMU in a namespace by
> default, I explicitly forbade relabeling without namespaces, leaving the
> decision to the platform admin.

I see, thanks a lot for your clearly explanation !

Comment 8 Luyao Huang 2019-02-11 03:18:49 UTC
Verify this bug with libvirt-5.0.0-2.module+el8+2779+3106feb7.x86_64:

1. check /dev/sev device permission:

# ll -Z /dev/sev 
crw-------. 1 root root system_u:object_r:sev_device_t:s0 10, 58 Feb  9 13:14 /dev/sev

2. install libvirt and check domcapabilities output:

# virsh domcapabilities

    <sev supported='yes'>
      <cbitpos>47</cbitpos>
      <reducedPhysBits>1</reducedPhysBits>
    </sev>

3. start a guest with sev launchSecurity:

# virsh dumpxml sev-q35

  <launchSecurity type='sev'>
    <cbitpos>47</cbitpos>
    <reducedPhysBits>1</reducedPhysBits>
    <policy>0x0001</policy>
  </launchSecurity>

# virsh start sev-q35
Domain sev-q35 started

4. check /dev/sev device label:

# ll -Z /dev/sev 
crw-------. 1 root root system_u:object_r:sev_device_t:s0 10, 58 Feb  9 13:14 /dev/sev

5. set namespaces = [ ] in qemu.conf to disable guest mount namespace

6. restart libvirtd and start sev guest:

# virsh start sev-q35
error: Failed to start domain sev-q35
error: internal error: process exited while connecting to monitor: 2019-02-11T03:17:25.192428Z qemu-kvm: sev_guest_init: Failed to open /dev/sev 'Permission denied'
2019-02-11T03:17:25.196308Z qemu-kvm: failed to initialize KVM: Operation not permitted

7. check /dev/sev device label:

# ll -Z /dev/sev 
crw-------. 1 root root system_u:object_r:sev_device_t:s0 10, 58 Feb  9 13:14 /dev/sev

8. check domcapabilities output in non-root user:

$ virsh domcapabilities

<sev supported='no'/>

Comment 10 errata-xmlrpc 2019-05-29 16:05:30 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:1293


Note You need to log in before you can comment on or make changes to this bug.