Bug 2144436

Summary: usb device cannot be found in VM when starting VM with a usb-redir device
Product: Red Hat Enterprise Linux 9 Reporter: Han Han <hhan>
Component: qemu-kvmAssignee: Gerd Hoffmann <kraxel>
qemu-kvm sub component: USB QA Contact: yduan
Status: CLOSED ERRATA Docs Contact:
Severity: unspecified    
Priority: unspecified CC: chwen, dzheng, hhan, jinzhao, jtomko, juzhang, kraxel, mrezanin, vgoyal, victortoso, virt-maint, xuwei, zhguo
Version: 9.2Keywords: Automation, Regression, Triaged
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: qemu-kvm-7.2.0-1.el9 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-05-09 07:20:51 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 2135806    
Bug Blocks: 2135760    
Attachments:
Description Flags
the VM XML, logs of usbredirserver, virtqemud and qemu none

Description Han Han 2022-11-21 09:17:15 UTC
Created attachment 1926063 [details]
the VM XML, logs of usbredirserver, virtqemud and qemu

Description of problem:
As subject

Version-Release number of selected component (if applicable):
libvirt-8.9.0-2.el9.x86_64
qemu-kvm-7.1.0-5.el9.x86_64
usbredir-0.12.0-3.el9.x86_64
libusbx-1.0.26-1.el9.x86_64
guest kernel: kernel-5.14.0-197.el9.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Prepare a host with a USB mass storage:
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=ehci-pci/2p, 480M
    ID 1d6b:0002 Linux Foundation 2.0 root hub
    /sys/bus/usb/devices/usb2  /dev/bus/usb/002/001
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/8p, 480M
        ID 8087:8002 Intel Corp. 8 channel internal hub
        /sys/bus/usb/devices/2-1  /dev/bus/usb/002/002
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=ehci-pci/2p, 480M
    ID 1d6b:0002 Linux Foundation 2.0 root hub
    /sys/bus/usb/devices/usb1  /dev/bus/usb/001/001
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/6p, 480M
        ID 8087:800a Intel Corp. Hub
        /sys/bus/usb/devices/1-1  /dev/bus/usb/001/002
        |__ Port 1: Dev 3, If 0, Class=Mass Storage, Driver=usb-storage, 480M
            ID 0781:5591 SanDisk Corp. Ultra Flair
            /sys/bus/usb/devices/1-1.1  /dev/bus/usb/001/003
        |__ Port 6: Dev 4, If 0, Class=Hub, Driver=hub/6p, 480M
            ID 413c:a001 Dell Computer Corp. Hub
            /sys/bus/usb/devices/1-1.6  /dev/bus/usb/001/004

2. Start the usbredirserver to share the mass storage:
# LIBUSB_DEBUG=4 usbredirserver -v5 -p 6000 1-3

3. Start a VM with a USB redirdev connected. VM XML:
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
...
<redirdev bus="usb" type="tcp">
  <source mode="connect" host="localhost4" service="6000" tls="no"/>
  <protocol type="raw"/>
  <alias name="ua-REDIR"/>
  <address type="usb" bus="0" port="1"/>
</redirdev>
...
  <qemu:commandline>
    <qemu:env name='LIBUSB_DEBUG' value='4'/>
  </qemu:commandline>
</domain>

# virsh create vm.xml 
Domain 'avocado-vt-vm1' created from vm.xml

4. Login to the VM, check the usb mass storage:
# lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

No such a device.

Actual results:
As above

Expected results:
The usbredir device should be in the VM.

Additional info:
I.
The hmp query for usb and chardev look well:
# virsh qemu-monitor-command avocado-vt-vm1 --hmp info usb
  Device 0.0, Port 1, Speed 1.5 Mb/s, Product USB Redirection Device, ID: ua-REDIR

# virsh qemu-monitor-command avocado-vt-vm1 --hmp info chardev
charua-REDIR: filename=tcp:127.0.0.1:47770 <-> 127.0.0.1:6000
charserial0: filename=pty:/dev/pts/8
charmonitor: filename=unix:/var/lib/libvirt/qemu/domain-1-avocado-vt-vm1/monitor.sock,server=on

II.
And it works for hot-plug:
1. Start a VM without redirdev
2. Hot-plug the redirdev after guest is booted:
# virsh attach-device avocado-vt-vm1 redirdev.xml 
Device attached successfully

Check usb disk in guest:
(guest)# lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 002: ID 0781:5591 SanDisk Corp. Ultra Flair
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

III.
Reproduced with the manual set qemu cmdline with debugging options **debug=4**:
  <qemu:commandline>
    <qemu:env name='LIBUSB_DEBUG' value='4'/>
    <qemu:arg value='-chardev'/>
    <qemu:arg value='socket,id=charua-REDIR,host=localhost4,port=6000,debug=4'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='{"driver":"usb-redir","chardev":"charua-REDIR","id":"ua-REDIR","bus":"usb.0","port":"1","debug":4}'/>
  </qemu:commandline>

IV.
The attachment:
redir-libvirt/ : the VM XML, logs of usbredirserver, virtqemud and qemu from the steps of bugs
redir-hotplug: the VM XML, redirdev XML, logs of usbredirserver, virtqemud and qemu from II
redir-qemu/ : the VM XML, logs of usbredirserver, virtqemud and qemu from III

V.
It works on:
libvirt	libvirt-8.9.0-2.el9.x86_64
qemu-kvm	qemu-kvm-7.1.0-4.el9.x86_64
kernel	kernel-5.14.0-188.el9.x86_64
qemu-kvm-rhev	not installed
selinux-policy	selinux-policy-34.1.46-1.el9.noarch
spice-gtk	not installed
spice	not installed
libcgroup	not installed
systemd	systemd-250-12.el9_1.x86_64
netcf	not installed
dnsmasq	dnsmasq-2.85-5.el9.x86_64
virtio-win	not installed

Covered by tp-libvirt usb_device.redirdev.tcp.usb2_0.companion.pcie-root.pcie-root-port.pcie-to-pci-bridge

Comment 1 Han Han 2022-11-21 09:20:07 UTC
Ján,
Please help to check if it is a issue of libvirt.

Comment 2 Ján Tomko 2022-11-21 12:25:56 UTC
I have bisected this to QEMU commit:

commit 0631d4b448454ae8a1ab091c447e3f71ab6e088a
    usbredir: avoid queuing hello packet on snapshot restore
git describe: v7.0.0-1841-g0631d4b448 contains: v7.1.0-rc0~63^2~4

A revert has been proposed by the author but was not merged:
https://lists.gnu.org/archive/html/qemu-devel/2022-08/msg02068.html

Comment 3 Victor Toso 2022-11-21 14:30:19 UTC
Hi, just for more context, the original patch [0] was fixing a behavior in usbredir that affected QEMU even when no usbredir was being used. In usbredir, this behavior was reverted with [1].

[0] https://gitlab.com/qemu-project/qemu/-/commit/0631d4b448454ae8a1ab091c447e3f71ab6e088a
[1] https://gitlab.freedesktop.org/spice/usbredir/-/merge_requests/61

Comment 4 John Ferlan 2022-11-21 23:39:29 UTC
Gerd - should/can the patch from above that Jano reviewed get merged before qemu-7.2 so it can be included into RHEL 9.2?

Comment 5 Gerd Hoffmann 2022-11-22 10:10:40 UTC
(In reply to John Ferlan from comment #4)
> Gerd - should/can the patch from above that Jano reviewed get merged before
> qemu-7.2 so it can be included into RHEL 9.2?

Looking ....

Hmm, it's actually part of a three patch series, where patch #2 introduces
a new runstate.  I don't think that kind of change is a good idea during
7.2 release freeze.

Comment 6 Ján Tomko 2022-11-22 12:41:07 UTC
The rest of the series is irrelevant. It implements a workaround that is no longer needed.
Patch 1/3 reverts the original workaround, which broke usbredir with libvirt.

Would it be helpful if I resent the patch separately?

Comment 7 John Ferlan 2022-11-22 16:05:18 UTC
Adding needinfo just to be sure Gerd sees the question in time

Comment 8 Ján Tomko 2022-11-23 08:55:35 UTC
I resent the patch, just in case:
https://lists.gnu.org/archive/html/qemu-devel/2022-11/msg03769.html

Comment 9 yduan 2022-11-25 01:49:52 UTC
Reproduced with following components:
5.14.0-200.el9.x86_64
qemu-kvm-7.1.0-5.el9.x86_64
usbredir-0.12.0-3.el9.x86_64
usbredir-server-0.12.0-3.el9.x86_64
libusbx-1.0.26-1.el9.x86_64

usb.usb_redir.via_tcp

Comment 10 John Ferlan 2022-11-27 13:11:56 UTC
The upstream patch is merged and will be present in qemu-7.2, adjusting other fields (ITR, DTM, depends on bug, State, DevelWhiteboard, devel_ack)... Dropping the needinfo - thanks Gerd for getting this in!

Comment 13 Han Han 2022-12-19 08:22:39 UTC
Test case usb_device.redirdev.tcp.usb2_0.companion.pcie-root.pcie-root-port.pcie-to-pci-bridge passed on qemu-kvm-7.2.0-1.el9

Comment 14 Yanan Fu 2022-12-20 09:19:05 UTC
QE bot(pre verify): Set 'Verified:Tested,SanityOnly' as gating/tier1 test pass.

Comment 20 errata-xmlrpc 2023-05-09 07:20:51 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: qemu-kvm security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:2162