Bug 2003679

Summary: qemu-kvm crashes after I use virt-viewer/virt-manager connect to a vnc vm which listening on unix socket
Product: Red Hat Enterprise Linux 8 Reporter: John Ferlan <jferlan>
Component: qemu-kvmAssignee: Marc-Andre Lureau <marcandre.lureau>
qemu-kvm sub component: Graphics QA Contact: zhoujunqin <juzhou>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: unspecified CC: chhu, coli, fjin, hongzliu, jmaloy, juzhou, marcandre.lureau, mrezanin, tyan, tzheng, virt-maint, xiaodwan, yafu, yicui, zhetang
Version: 8.2Keywords: Triaged
Target Milestone: rcFlags: pm-rhel: mirror+
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: qemu-kvm-6.2.0-1.module+el8.6.0+13725+61ae1949 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 2000814 Environment:
Last Closed: 2022-05-10 13:21:40 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 2000814, 2027716    
Bug Blocks:    

Description John Ferlan 2021-09-13 12:24:27 UTC
+++ This bug was initially created as a clone of Bug #2000814 +++

Description of problem:
qemu-kvm crashes after I use virt-viewer/virt-manager connect to a vnc vm which listening on unix socket

Version-Release number of selected component (if applicable):
libvirt-7.6.0-2.el9.x86_64
qemu-kvm-6.1.0-1.el9.x86_64
virt-viewer-10.0-3.el9.x86_64


How reproducible:
100%

Steps to Reproduce:
1. Edit the xml of the vnc guest and set listen to type to 'socket'
...
    <graphics type='vnc' port='-1' autoport='yes'>
      <listen type='socket'/>
    </graphics>
...

2.  Start the guest.
# virsh start $vncguest

3. Check the xml of the guest, verify a socket file is generated for the vnc guest.
...
    <graphics type='vnc' socket='/var/lib/libvirt/qemu/domain-4-7vnc1/vnc.sock'>
      <listen type='socket' socket='/var/lib/libvirt/qemu/domain-4-7vnc1/vnc.sock'/>
    </graphics>
...

4. Run virt-viewer to connect the guest by root user.
# virt-viewer $vncguest

Error message prompts by virt-viewer:
"""
Unable to connect to the graphics server localhost:/var/lib/libvirt/qemu/domain-4-7vnc1/vnc.sock

Server closed the connection.
"""

Test result: Failed to connect to the VM's console, and the running VM will be shut down immediately.

# virsh domstate 7vnc1
shut off

Actual results:
As the description.

Expected results:
Fix it.

Additional info:
# coredumpctl  debug
           PID: 5522 (qemu-kvm)
           UID: 107 (qemu)
           GID: 107 (qemu)
        Signal: 6 (ABRT)
     Timestamp: Fri 2021-09-03 00:36:30 EDT (2min 25s ago)
  Command Line: /usr/libexec/qemu-kvm -name guest=7vnc1,debug-threads=on -S -object $'{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain-4-7vnc1/master-key.aes"}' -machine pc-q35-rhel8.5.0,accel=kvm,usb=off,dump-guest-core=off,memory-backend=pc.ram -cpu Skylake-Client-IBRS,ss=on,vmx=on,pdcm=on,hypervisor=on,tsc-adjust=on,clflushopt=on,umip=on,md-clear=on,stibp=on,arch-capabilities=on,ssbd=on,xsaves=on,pdpe1gb=on,ibpb=on,ibrs=on,amd-stibp=on,amd-ssbd=on,skip-l1dfl-vmentry=on,pschange-mc-no=on,hle=off,rtm=off -m 1024 -object $'{"qom-type":"memory-backend-ram","id":"pc.ram","size":1073741824}' -overcommit mem-lock=off -smp 2,sockets=2,cores=1,threads=1 -uuid 51b3dcb2-b0ec-46ae-a76f-e402b3bce38b -no-user-config -nodefaults -chardev socket,id=charmonitor,fd=32,server=on,wait=off -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -global ICH9-LPC.disable_s3=1 -global ICH9-LPC.disable_s4=1 -boot strict=on -device pcie-root-port,port=0x10,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x2 -device pcie-root-port,port=0x11,chassis=2,id=pci.2,bus=pcie.0,addr=0x2.0x1 -device pcie-root-port,port=0x12,chassis=3,id=pci.3,bus=pcie.0,addr=0x2.0x2 -device pcie-root-port,port=0x13,chassis=4,id=pci.4,bus=pcie.0,addr=0x2.0x3 -device pcie-root-port,port=0x14,chassis=5,id=pci.5,bus=pcie.0,addr=0x2.0x4 -device pcie-root-port,port=0x15,chassis=6,id=pci.6,bus=pcie.0,addr=0x2.0x5 -device pcie-root-port,port=0x16,chassis=7,id=pci.7,bus=pcie.0,addr=0x2.0x6 -device qemu-xhci,p2=15,p3=15,id=usb,bus=pci.2,addr=0x0 -device virtio-serial-pci,id=virtio-serial0,bus=pci.3,addr=0x0 -device virtio-serial-pci,id=virtio-serial1,bus=pci.7,addr=0x0 -blockdev $'{"driver":"file","filename":"/var/lib/libvirt/images/7vnc1.qcow2","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' -blockdev $'{"node-name":"libvirt-1-format","read-only":false,"driver":"qcow2","file":"libvirt-1-storage","backing":null}' -device virtio-blk-pci,bus=pci.4,addr=0x0,drive=libvirt-1-format,id=virtio-disk0,bootindex=1 -netdev tap,fd=34,id=hostnet0,vhost=on,vhostfd=35 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:93:c0:a6,bus=pci.1,addr=0x0 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,fd=36,server=on,wait=off -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -device usb-tablet,id=input0,bus=usb.0,port=1 -audiodev id=audio1,driver=none -vnc vnc=unix:/var/lib/libvirt/qemu/domain-4-7vnc1/vnc.sock,audiodev=audio1 -device VGA,id=video0,vgamem_mb=16,bus=pcie.0,addr=0x1 -device virtio-balloon-pci,id=balloon0,bus=pci.5,addr=0x0 -object $'{"qom-type":"rng-random","id":"objrng0","filename":"/dev/urandom"}' -device virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.6,addr=0x0 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on
    Executable: /usr/libexec/qemu-kvm
 Control Group: /machine.slice/machine-qemu\x2d4\x2d7vnc1.scope/libvirt/emulator
          Unit: machine-qemu\x2d4\x2d7vnc1.scope
         Slice: machine.slice
       Boot ID: a4d2138faa5441ee82c162125dff01f6
    Machine ID: 6e3ff601a3c94dd3913478317e81b21f
      Hostname: juzhou-rhel9
       Storage: none
       Message: Process 5522 (qemu-kvm) of user 107 dumped core.

Coredump entry has no core attached (neither internally in the journal nor externally on disk).

--- Additional comment from zhoujunqin on 2021-09-03 05:11:35 UTC ---



--- Additional comment from Guo, Zhiyi on 2021-09-03 06:40:30 UTC ---

Junqing will help to cover local display scenario and verify bug

--- Additional comment from Marc-Andre Lureau on 2021-09-03 16:02:28 UTC ---

This is fixed by "[PATCH v3] qemu-sockets: fix unix socket path copy (again)" upstream, pending merge.
We will have to backport it ince it is merged.

please qa ack

--- Additional comment from Marc-Andre Lureau on 2021-09-07 16:05:06 UTC ---

https://gitlab.com/redhat/centos-stream/src/qemu-kvm/-/merge_requests/42

waiting for CI & acks

--- Additional comment from  on 2021-09-13 07:41:31 UTC ---

Hi,
This issue can be reproduced in rhel 8.6 with qemu-kvm 6.1.0-1.
Could you please help check if we need to clone this bug to rhel 8.6? 


Thanks,
Yingshun

--- Additional comment from  on 2021-09-13 07:42:34 UTC ---



--- Additional comment from Marc-Andre Lureau on 2021-09-13 08:03:10 UTC ---

(In reply to yicui from comment #5)
> Hi,
> This issue can be reproduced in rhel 8.6 with qemu-kvm 6.1.0-1.
> Could you please help check if we need to clone this bug to rhel 8.6? 

yes, thanks

--- Additional comment from zhoujunqin on 2021-09-13 09:48:34 UTC ---

Add additional info:
output for command "# ps -ef |grep 7vnc1"


qemu       86718       1  0 Sep07 ?        00:14:28 /usr/libexec/qemu-kvm -name guest=7vnc1,debug-threads=on -S -object {"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain-2-7vnc1/master-key.aes"} -machine pc-q35-rhel8.5.0,accel=kvm,usb=off,dump-guest-core=off,memory-backend=pc.ram -cpu Skylake-Client-IBRS,ss=on,vmx=on,pdcm=on,hypervisor=on,tsc-adjust=on,clflushopt=on,umip=on,md-clear=on,stibp=on,arch-capabilities=on,ssbd=on,xsaves=on,pdpe1gb=on,ibpb=on,ibrs=on,amd-stibp=on,amd-ssbd=on,skip-l1dfl-vmentry=on,pschange-mc-no=on,hle=off,rtm=off -m 1024 -object {"qom-type":"memory-backend-ram","id":"pc.ram","size":1073741824} -overcommit mem-lock=off -smp 2,sockets=2,cores=1,threads=1 -uuid 51b3dcb2-b0ec-46ae-a76f-e402b3bce38b -no-user-config -nodefaults -chardev socket,id=charmonitor,fd=37,server=on,wait=off -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -global ICH9-LPC.disable_s3=1 -global ICH9-LPC.disable_s4=1 -boot strict=on -device pcie-root-port,port=0x10,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x2 -device pcie-root-port,port=0x11,chassis=2,id=pci.2,bus=pcie.0,addr=0x2.0x1 -device pcie-root-port,port=0x12,chassis=3,id=pci.3,bus=pcie.0,addr=0x2.0x2 -device pcie-root-port,port=0x13,chassis=4,id=pci.4,bus=pcie.0,addr=0x2.0x3 -device pcie-root-port,port=0x14,chassis=5,id=pci.5,bus=pcie.0,addr=0x2.0x4 -device pcie-root-port,port=0x15,chassis=6,id=pci.6,bus=pcie.0,addr=0x2.0x5 -device pcie-root-port,port=0x16,chassis=7,id=pci.7,bus=pcie.0,addr=0x2.0x6 -device qemu-xhci,p2=15,p3=15,id=usb,bus=pci.2,addr=0x0 -device virtio-serial-pci,id=virtio-serial0,bus=pci.3,addr=0x0 -device virtio-serial-pci,id=virtio-serial1,bus=pci.7,addr=0x0 -blockdev {"driver":"file","filename":"/var/lib/libvirt/images/7vnc1.qcow2","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"} -blockdev {"node-name":"libvirt-1-format","read-only":false,"driver":"qcow2","file":"libvirt-1-storage","backing":null} -device virtio-blk-pci,bus=pci.4,addr=0x0,drive=libvirt-1-format,id=virtio-disk0,bootindex=1 -netdev tap,fd=39,id=hostnet0,vhost=on,vhostfd=40 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:93:c0:a6,bus=pci.1,addr=0x0 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,fd=41,server=on,wait=off -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -device usb-tablet,id=input0,bus=usb.0,port=1 -audiodev id=audio1,driver=none -vnc vnc=unix:/var/lib/libvirt/qemu/domain-2-7vnc1/vnc.sock,audiodev=audio1 -device VGA,id=video0,vgamem_mb=16,bus=pcie.0,addr=0x1 -device virtio-balloon-pci,id=balloon0,bus=pci.5,addr=0x0 -object {"qom-type":"rng-random","id":"objrng0","filename":"/dev/urandom"} -device virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.6,addr=0x0 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on

--- Additional comment from John Ferlan on 2021-09-13 12:22:39 UTC ---

Update to be included in the qemu-6.2 rebase planned for Nov/Dec

Comment 3 John Ferlan 2021-11-18 12:44:18 UTC
Still need a qa_ack+ please! Some day soon I hope they remove the need to adjust both.

Comment 5 John Ferlan 2021-12-22 18:01:48 UTC
Mass update of DTM/ITM to +3 values since the rebase of qemu-6.2 into RHEL 8.6 has been delayed or slowed due to process roadblocks (authentication changes, gating issues). This avoids the DevMissed bot and worse the bot that could come along and strip release+. The +3 was chosen mainly to give a cushion. 

Also added the qemu-6.2 rebase bug 2027716 as a dependent.

Comment 8 Yanan Fu 2021-12-24 02:48:41 UTC
QE bot(pre verify): Set 'Verified:Tested,SanityOnly' as gating/tier1 test pass.

Comment 9 Hongzhou Liu 2022-01-05 07:39:16 UTC
Verify this bug on rhel8.6

packages:
qemu-kvm-6.2.0-2.module+el8.6.0+13738+17338784.x86_64
virt-viewer-9.0-12.el8.x86_64
libvirt-7.10.0-1.module+el8.6.0+13502+4f24a11d.x86_64


1. prepare a vm, use virsh edit edit the xml of the vnc guest and set listen to type to 'socket'

<graphics type='vnc'>
      <listen type='socket'/>
    </graphics>

2. start the guest and check the xml
# virsh domstate rhel9.0 
>
running
# virsh dumpxml rhel9.0  | grep grap -C2
> 
    <graphics type='vnc' socket='/var/lib/libvirt/qemu/domain-13-rhel9.0/vnc.sock'>
      <listen type='socket' socket='/var/lib/libvirt/qemu/domain-13-rhel9.0/vnc.sock'/>
    </graphics>
3. connect the guest via virt-viewer
# virt-viewer rhel9.0

result: virt-viewer can connect the guest correctly, the result is as expected so I change the status to verified. Thanks!

Comment 11 errata-xmlrpc 2022-05-10 13:21:40 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: virt:rhel and virt-devel:rhel security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:1759