Bug 2230469

Summary: [s390x] virtio-gpu blob option not functional
Product: Red Hat Enterprise Linux 9 Reporter: smitterl
Component: qemu-kvmAssignee: Thomas Huth <thuth>
qemu-kvm sub component: Graphics QA Contact: virt-qe-z
Status: ASSIGNED --- Docs Contact:
Severity: medium    
Priority: medium CC: bmarcynk, clegoate, jinzhao, juzhang, kraxel, lijin, marcandre.lureau, mdeng, thuth, virt-maint, zhguo
Version: 9.3Keywords: Triaged
Target Milestone: rcFlags: smitterl: needinfo? (bmarcynk)
smitterl: needinfo? (mdeng)
zhguo: needinfo? (kraxel)
Target Release: 9.4   
Hardware: s390x   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description smitterl 2023-08-09 14:05:51 UTC
Description of problem:
Follow up on verified https://bugzilla.redhat.com/show_bug.cgi?id=2032406

Using blob option for virtio-gpu doesn't work (no graphical output).

Version-Release number of selected component (if applicable):
Host kernel 5.14.0-350.el9.s390x
Guest kernel kernel-5.14.0-348.el9.s390x
libvirt-9.5.0-5.el9.s390x
qemu-kvm-8.0.0-10.el9.s390x

How reproducible:
100%

Steps to Reproduce:
1. setenforce 0 (workaround for missing selinux label)
2. confirm /dev/udmabuf is available
3. start vm with graphics and memory backing, e.g. from libvirt
...
  <memoryBacking>
    <source type='memfd'/>
  </memoryBacking>
...
    <video>
      <model type='virtio' heads='1' primary='yes' blob='on'/>
      <alias name='video0'/>
      <address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0002'/>
    </video>
...
4. Connect vnc client

Actual results:
No graphical output

Expected results:
Same as without blob=on, it works

Additional info:
It works when I don't use blob=on

Comment 2 smitterl 2023-08-11 14:56:29 UTC
/usr/libexec/qemu-kvm
...
 -machine s390-ccw-virtio-rhel9.2.0,usb=off,dump-guest-core=off,memory-backend=s390.ram
...
 -object {"qom-type":"memory-backend-memfd","id":"s390.ram","x-use-canonical-path-for-ramblock-id":false,"size":1073741824}
 -overcommit mem-lock=off
....
 -device {"driver":"virtio-gpu-ccw","id":"video0","max_outputs":1,"blob":true,"devno":"fe.0.0002"}
...

Comment 3 smitterl 2023-08-11 14:58:27 UTC
(In reply to smitterl from comment #2)
> /usr/libexec/qemu-kvm
> ...
>  -machine
> s390-ccw-virtio-rhel9.2.0,usb=off,dump-guest-core=off,memory-backend=s390.ram
> ...
>  -object
> {"qom-type":"memory-backend-memfd","id":"s390.ram","x-use-canonical-path-for-
> ramblock-id":false,"size":1073741824}
>  -overcommit mem-lock=off
> ....
>  -device
> {"driver":"virtio-gpu-ccw","id":"video0","max_outputs":1,"blob":true,"devno":
> "fe.0.0002"}
> ...

 -device {"driver":"virtio-keyboard-ccw","id":"input0","devno":"fe.0.0005"}
 -device {"driver":"virtio-mouse-ccw","id":"input1","devno":"fe.0.0006"}
 -audiodev {"id":"audio1","driver":"none"}
 -vnc 127.0.0.1:0,audiodev=audio1

Comment 4 smitterl 2023-08-11 15:03:47 UTC
I tried to find a shorter qemu-cmdline but failed. The full line from libvirt is:

qemu-kvm\ 
 -name guest=avocado-vt-vm1,debug-threads=on\ 
 -S\ 
 -object {"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain-2-avocado-vt-vm1/master-key.aes"}\ 
 -machine s390-ccw-virtio-rhel9.2.0,usb=off,dump-guest-core=off,memory-backend=s390.ram\ 
 -accel kvm\ 
 -cpu gen15a-base,aen=on,cmmnt=on,vxpdeh=on,aefsi=on,diag318=on,csske=on,mepoch=on,msa9=on,msa8=on,msa7=on,msa6=on,msa5=on,msa4=on,msa3=on,msa2=on,msa1=on,sthyi=on,edat=on,ri=on,deflate=on,edat2=on,unpack=on,etoken=on,vx=on,ipter=on,mepochptff=on,ap=on,vxeh=on,vxpd=on,esop=on,msa9_pckmo=on,vxeh2=on,esort=on,apqi=on,apft=on,els=on,iep=on,apqci=on,cte=on,ais=on,bpb=on,gs=on,ppa15=on,zpci=on,sea_esop2=on,te=on,cmm=on\ 
 -m size=1048576k\ 
 -object {"qom-type":"memory-backend-memfd","id":"s390.ram","x-use-canonical-path-for-ramblock-id":false,"size":1073741824}\ 
 -overcommit mem-lock=off\ 
 -smp 2,sockets=2,cores=1,threads=1\ 
 -uuid 880b9873-aa33-4445-9e02-3a69021d530f\ 
 -no-user-config\ 
 -nodefaults\ 
 -chardev socket,id=charmonitor,fd=24,server=on,wait=off\ 
 -mon chardev=charmonitor,id=monitor,mode=control\ 
 -rtc base=utc\ 
 -no-shutdown\ 
 -boot strict=on\ 
 -device {"driver":"virtio-scsi-ccw","id":"scsi0","devno":"fe.0.0003"}\ 
 -device {"driver":"virtio-serial-ccw","id":"virtio-serial0","devno":"fe.0.0004"}\ 
 -blockdev {"driver":"file","filename":"/var/lib/avocado/data/avocado-vt/images/jeos-27-s390x.qcow2","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}\ 
 -blockdev {"node-name":"libvirt-1-format","read-only":false,"driver":"qcow2","file":"libvirt-1-storage","backing":null}\ 
 -device {"driver":"virtio-blk-ccw","devno":"fe.0.0000","drive":"libvirt-1-format","id":"virtio-disk0","bootindex":1}\ 
 -netdev {"type":"tap","fd":"25","vhost":true,"vhostfd":"27","id":"hostnet0"}\ 
 -device {"driver":"virtio-net-ccw","netdev":"hostnet0","id":"net0","mac":"52:54:00:90:b1:d2","devno":"fe.0.0001"}\ 
 -chardev pty,id=charserial0\ 
 -device {"driver":"sclpconsole","chardev":"charserial0","id":"serial0"}\ 
 -chardev socket,id=charchannel0,fd=23,server=on,wait=off\ 
 -device {"driver":"virtserialport","bus":"virtio-serial0.0","nr":1,"chardev":"charchannel0","id":"channel0","name":"org.qemu.guest_agent.0"}\ 
 -chardev pty,id=charconsole1\ 
 -device {"driver":"virtconsole","chardev":"charconsole1","id":"console1"}\ 
 -device {"driver":"virtio-keyboard-ccw","id":"input0","devno":"fe.0.0005"}\ 
 -device {"driver":"virtio-mouse-ccw","id":"input1","devno":"fe.0.0006"}\ 
 -audiodev {"id":"audio1","driver":"none"}\ 
 -vnc 127.0.0.1:0,audiodev=audio1\ 
 -device {"driver":"virtio-gpu-ccw","id":"video0","max_outputs":1,"blob":true,"devno":"fe.0.0002"}\ 
 -device {"driver":"virtio-balloon-ccw","id":"balloon0","devno":"fe.0.0007"}\ 
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny\ 
 -device {"driver":"vhost-vsock-ccw","id":"vsock0","guest-cid":3,"vhostfd":"20","devno":"fe.0.0008"}\ 
 -msg timestamp=on

Comment 7 Thomas Huth 2023-08-15 09:17:06 UTC
(In reply to smitterl from comment #4)
> I tried to find a shorter qemu-cmdline but failed.

Thanks for the full command line, I condensed it to the following which seems to be enough to reproduce the problem:

/usr/libexec/qemu-kvm \
 -machine s390-ccw-virtio,memory-backend=s390.ram \
 -accel kvm -cpu host -smp 2 -m size=1G \
 -object memory-backend-memfd,id="s390.ram",size=1G \
 -hda "guest.qcow2" \
 -device virtio-keyboard-ccw \
 -device virtio-mouse-ccw \
 -device virtio-gpu-ccw,id="video0",max_outputs=1,blob=true \
 -serial mon:stdio -vnc :0

Comment 8 Thomas Huth 2023-08-15 11:45:57 UTC
Seems to be an endianness problem ... I can make it work with this patch in QEMU:

diff a/include/hw/virtio/virtio-gpu-bswap.h b/include/hw/virtio/virtio-gpu-bswap.h
--- a/include/hw/virtio/virtio-gpu-bswap.h
+++ b/include/hw/virtio/virtio-gpu-bswap.h
@@ -63,7 +63,10 @@ virtio_gpu_create_blob_bswap(struct virtio_gpu_resource_create_blob *cblob)
 {
     virtio_gpu_ctrl_hdr_bswap(&cblob->hdr);
     le32_to_cpus(&cblob->resource_id);
+    le32_to_cpus(&cblob->blob_mem);
     le32_to_cpus(&cblob->blob_flags);
+    le32_to_cpus(&cblob->nr_entries);
+    le64_to_cpus(&cblob->blob_id);
     le64_to_cpus(&cblob->size);
 }

Comment 9 Thomas Huth 2023-08-15 12:22:48 UTC
Suggested patch upstream:

https://lore.kernel.org/qemu-devel/20230815122007.928049-1-thuth@redhat.com/