Bug 2032406

Summary:	Support blob for virtio-vga/virtio-gpu device -- libvirt
Product:	Red Hat Enterprise Linux 9	Reporter:	Guo, Zhiyi <zhguo>
Component:	libvirt	Assignee:	Jonathon Jongsma <jjongsma>
libvirt sub component:	General	QA Contact:	zhentang <zhetang>
Status:	CLOSED ERRATA	Docs Contact:	Jiri Herrmann <jherrman>
Severity:	medium
Priority:	medium	CC:	berrange, coli, gfialova, jdenemar, jherrman, jjongsma, jsuchane, kraxel, lmen, smitterl, virt-maint, xuzhang, yafu, ymankad, zhetang
Version:	9.1	Keywords:	AutomationTriaged, FutureFeature, Triaged
Target Milestone:	rc	Flags:	zhguo: needinfo- zhguo: needinfo- lmen: needinfo-
Target Release:	---
Hardware:	All
OS:	All
Whiteboard:
Fixed In Version:	libvirt-9.2.0-1.el9	Doc Type:	Enhancement
Doc Text:	.`virtio-vga` and `virtio-gpu` devices now support `blob` resources It is now possible for `virtio-vga` and `virtio-gpu` devices to use `blob` memory resources, which improves their performance in certain scenarios. To attach a `blob` resource to a `virtio` graphics device, add a `blob="on"` option to the corresponding `<video>` section in the virtual machine's XML configuration. For example: ---- <video> <model type="virtio" heads="1" primary="yes" blob="on"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x0"/> </video> ---- Note, however, that this feature currently does not work on IBM Z hosts.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2023-11-07 08:30:47 UTC	Type:	Feature Request
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:	libvirt-9.2.0
Embargoed:
Bug Depends On:	2032399, 2175785, 2223727
Bug Blocks:

Description Guo, Zhiyi 2021-12-14 12:49:23 UTC

Description of problem:
Support blob for virtio-vga/virtio-gpu device -- libvirt

Start from qemu 6.1, virtio-vga and virtio-gpu has a new option blob enabled:
# /usr/libexec/qemu-kvm -device virtio-vga,?
...
  blob=<bool>            - on/off (default: false)
...

# /usr/libexec/qemu-kvm -device virtio-gpu-pci,?
...
  blob=<bool>            - on/off (default: false)
...

Per discussion with Gerd(kraxel), this feature is nice to be enabled and can bring performance improvement in certain virtualization scenarios.

We also think it makes sense to enable this option from libvirt layer
So open this bug to track the discussion might be needed and development/validation progress.

Comment 1 Jonathon Jongsma 2022-11-04 16:38:24 UTC

patches have been posted upstream. waiting for review.

Comment 2 Jonathon Jongsma 2023-03-08 19:55:45 UTC

This is now pushed upstream and should be included in upstream libvirt 9.2.0

Comment 4 Jonathon Jongsma 2023-05-16 20:10:57 UTC

This should be fixed by the rebase in rhel 9.3

Comment 5 zhentang 2023-07-17 03:05:50 UTC

Tested on 
libvirt-9.3.0-2.el9.x86_64
qemu-kvm-8.0.0-6.el9.x86_64


Vm could not start with "blob = 'on'", due to cannot open /dev/udmabuf: Permission denied


[root@zhetang-functiontest ~]# virsh dumpxml rhel9123 --xpath //video
<video>
  <model type="virtio" heads="1" primary="yes" blob="on"/>
  <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x0"/>
</video>


[root@zhetang-functiontest ~]# virsh start rhel9123 
error: Failed to start domain 'rhel9123'
error: internal error: qemu unexpectedly closed the monitor: 2023-07-17T02:59:26.065049Z qemu-kvm: -device {"driver":"virtio-vga","id":"video0","max_outputs":1,"blob":true,"bus":"pcie.0","addr":"0x1"}: warning: open /dev/udmabuf: Permission denied
2023-07-17T02:59:26.065277Z qemu-kvm: -device {"driver":"virtio-vga","id":"video0","max_outputs":1,"blob":true,"bus":"pcie.0","addr":"0x1"}: cannot enable blob resources without udmabuf


[root@zhetang-functiontest ~]# ll /dev/udmabuf
crw-rw----. 1 root kvm 10, 125 Jul 11 04:42 /dev/udmabuf

Comment 6 Jonathon Jongsma 2023-07-17 20:56:53 UTC

hmm, it looks like perhaps selinux is preventing this access. Do you see any selinux AVC messages? Does it work when selinux is in permissive mode?

Comment 7 zhentang 2023-07-18 02:41:52 UTC




(In reply to Jonathon Jongsma from comment #6)
> hmm, it looks like perhaps selinux is preventing this access. Do you see any
> selinux AVC messages? Does it work when selinux is in permissive mode?

Yes, there are some selinux messages
[root@zhetang-functiontest ~]# grep "SELinux is preventing" /var/log/messages
Jul 16 22:59:27 zhetang-functiontest setroubleshoot[9433]: SELinux is preventing /usr/libexec/qemu-kvm from 'read, write' accesses on the chr_file udmabuf. For complete SELinux messages run: sealert -l 3aacbdaf-2056-4048-83eb-ae858a3ec554
Jul 16 22:59:27 zhetang-functiontest setroubleshoot[9433]: SELinux is preventing /usr/libexec/qemu-kvm from 'read, write' accesses on the chr_file udmabuf.#012#012*****  Plugin catchall (100. confidence) suggests   **************************#012#012If you believe that qemu-kvm should be allowed read write access on the udmabuf chr_file by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'qemu-kvm' --raw | audit2allow -M my-qemukvm#012# semodule -X 300 -i my-qemukvm.pp#012


----------------------------------------------------------------------------------------
Tried permissive mode, the guest could not start.

[root@zhetang-functiontest ~]# setenforce 0
[root@zhetang-functiontest ~]# getenforce
Permissive
[root@zhetang-functiontest ~]# virsh start rhel9123 
error: Failed to start domain 'rhel9123'
error: internal error: qemu unexpectedly closed the monitor: 2023-07-18T02:33:57.083178Z qemu-kvm: -device {"driver":"virtio-vga","id":"video0","max_outputs":1,"blob":true,"bus":"pcie.0","addr":"0x1"}: cannot enable blob resources without udmabuf

[root@zhetang-functiontest ~]# grep "SELinux is preventing" /var/log/messages
Jul 16 22:59:27 zhetang-functiontest setroubleshoot[9433]: SELinux is preventing /usr/libexec/qemu-kvm from 'read, write' accesses on the chr_file udmabuf. For complete SELinux messages run: sealert -l 3aacbdaf-2056-4048-83eb-ae858a3ec554
Jul 16 22:59:27 zhetang-functiontest setroubleshoot[9433]: SELinux is preventing /usr/libexec/qemu-kvm from 'read, write' accesses on the chr_file udmabuf.#012#012*****  Plugin catchall (100. confidence) suggests   **************************#012#012If you believe that qemu-kvm should be allowed read write access on the udmabuf chr_file by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'qemu-kvm' --raw | audit2allow -M my-qemukvm#012# semodule -X 300 -i my-qemukvm.pp#012
Jul 17 22:33:58 zhetang-functiontest setroubleshoot[10202]: SELinux is preventing /usr/libexec/qemu-kvm from 'read, write' accesses on the chr_file udmabuf. For complete SELinux messages run: sealert -l 66b7bf1e-303c-4e4e-b2b8-4cca415b0dbe
Jul 17 22:33:58 zhetang-functiontest setroubleshoot[10202]: SELinux is preventing /usr/libexec/qemu-kvm from 'read, write' accesses on the chr_file udmabuf.#012#012*****  Plugin catchall (100. confidence) suggests   **************************#012#012If you believe that qemu-kvm should be allowed read write access on the udmabuf chr_file by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'qemu-kvm' --raw | audit2allow -M my-qemukvm#012# semodule -X 300 -i my-qemukvm.pp#012
Jul 17 22:33:58 zhetang-functiontest setroubleshoot[10202]: SELinux is preventing /usr/libexec/qemu-kvm from open access on the chr_file /dev/udmabuf. For complete SELinux messages run: sealert -l 29d95512-14e8-425e-9df0-18cbe82a654e
Jul 17 22:33:58 zhetang-functiontest setroubleshoot[10202]: SELinux is preventing /usr/libexec/qemu-kvm from open access on the chr_file /dev/udmabuf.#012#012*****  Plugin catchall (100. confidence) suggests   **************************#012#012If you believe that qemu-kvm should be allowed open access on the udmabuf chr_file by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'qemu-kvm' --raw | audit2allow -M my-qemukvm#012# semodule -X 300 -i my-qemukvm.pp#012

Comment 8 Jonathon Jongsma 2023-07-18 18:51:10 UTC

(In reply to zhentang from comment #7)
> Tried permissive mode, the guest could not start.

For this to work, the guest must be configured with a memfd memory backing.

<memoryBacking>
  <source type='memfd'/>
</memoryBacking>


You should be able to add that to your guest xml config and hopefully it will work. Unfortunately the error isn't very helpful. I have posted a patch for review upstream to report a friendlier error in this case: https://listman.redhat.com/archives/libvir-list/2023-July/240788.html

For the selinux issue, I opened bug #2223727

Comment 9 zhentang 2023-07-21 11:30:14 UTC

Verified on 
libvirt-9.3.0-2.el9.x86_64
qemu-kvm-8.0.0-6.el9.x86_64


1. start a vm with blob='on', vm can start successfully 

[root@zhetang-functiontest images]# virsh start rhel-blob 
Domain 'rhel-blob' started


[root@zhetang-functiontest images]# virsh dumpxml rhel-blob --xpath //video
<video>
  <model type="virtio" heads="1" primary="yes" blob="on"/>
  <alias name="video0"/>
  <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x0"/>
</video>


[root@zhetang-functiontest images]# ps aux | grep rhel-blob 
...-device {"driver":"virtio-vga","id":"video0","max_outputs":1,"blob":true,"bus":"pcie.0","addr":"0x1"} ...

(in guest)
Last login: Fri Jul 21 18:02:02 on tty1
[root@localhost ~]# dmesg | grep blob
[   11.429381] [drm] features: -virgl +edid +resource_blob -host_visible


2. start vm with blob = 'off', guest can start successfully

[root@zhetang-functiontest images]# virsh dumpxml rhel-blob --xpath //video
<video>
  <model type="virtio" heads="1" primary="yes" blob="off"/>
  <alias name="video0"/>
  <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x0"/>
</video>

[root@zhetang-functiontest images]# ps aux | grep rhel| grep blob
...... -device {"driver":"virtio-vga","id":"video0","max_outputs":1,"blob":false,"bus":"pcie.0","addr":"0x1"} ......


[root@localhost ~]# dmesg | grep blob
[    8.282991] [drm] features: -virgl +edid -resource_blob -host_visible

3.start vm with no blob, guest can start successfully

[root@zhetang-functiontest images]# virsh dumpxml rhel-blob --xpath //video
<video>
  <model type="virtio" heads="1" primary="yes"/>
  <alias name="video0"/>
  <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x0"/>
</video>


[root@zhetang-functiontest images]# ps aux | grep rhel | grep blob
....... -device {"driver":"virtio-vga","id":"video0","max_outputs":1,"bus":"pcie.0","addr":"0x1"} ......


[root@localhost ~]# dmesg | grep blob
[    7.914464] [drm] features: -virgl +edid -resource_blob -host_visible

Comment 31 errata-xmlrpc 2023-11-07 08:30:47 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: libvirt security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:6409

Comment 32 Red Hat Bugzilla 2024-03-07 04:25:07 UTC

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days