Bug 976685

AFAIK USB-3 is not supported in RHEL-6 qemu, so we will likely never support USB-3 devices with RHEL-6 based hosts. However upstream qemu has some patches to allow passing through simple USB-3 devices (such as bulk-only-transport using USB 3 memory sticks) as USB-2, even if they are plugged into a USB-3 port on the client. We could try to backport that to RHEL-6.

moving to RHEL-7.

it depend on QEMU bug (824659)

I've been working on getting full USB-3 support (including usb 3 bulk streams) into upstream qemu, libusb, usbredir and the kernel for the last few months. I've just finished this work, I'll file seprate bugs for getting all this into 7.x, and make them block this bug.

I've filed bugs for getting all the bits necessary for usb-3 bulk stream support into 7.x in place:
kernel bug 1033088
libusbx bug 1033092
qemu bug 1033098
libusbredir bug 1033101

I cannot forward to VM next device:

Bus 002 Device 003: ID 0b95:1790 ASIX Electronics Corp. AX88179 Gigabit Ethernet

Client:
virt-viewer-2.0-7.el7.x86_64

Guest:
spice-vdagent-0.14.0-10.el7.x86_64 + kernel-3.10.0-369.el7.x86_64

Host:
kernel-3.10.0-327.10.1.el7.x86_64 + qemu-kvm-rhev-2.3.0-31.el7_2.10.x86_64

At the post time those are the latest client/vm/host software versions.

At VM dmesg shows:

[  598.308574] usb 1-5: new high-speed USB device number 8 using ehci-pci
[  598.415559] usb 1-5: device descriptor read/64, error 18
[  598.623584] usb 1-5: device descriptor read/64, error 18
[  598.826600] usb 1-5: new high-speed USB device number 9 using ehci-pci
[  598.934560] usb 1-5: device descriptor read/64, error 18
[  599.142555] usb 1-5: device descriptor read/64, error 18
[  599.350679] usb 1-5: new high-speed USB device number 10 using ehci-pci
[  599.367172] usb 1-5: Invalid ep0 maxpacket: 9
[  599.485550] usb 1-5: new high-speed USB device number 11 using ehci-pci
[  599.555353] usb 1-5: Invalid ep0 maxpacket: 9
[  599.556566] usb usb1-port5: unable to enumerate USB device
[  599.780500] usb 4-1: new full-speed USB device number 6 using uhci_hcd
[  599.916510] usb 4-1: device descriptor read/64, error 18
[  600.130497] usb 4-1: device descriptor read/64, error 18
[  600.333497] usb 4-1: new full-speed USB device number 7 using uhci_hcd
[  600.446495] usb 4-1: device descriptor read/64, error 18
[  600.661503] usb 4-1: device descriptor read/64, error 18
[  600.864450] usb 4-1: new full-speed USB device number 8 using uhci_hcd
[  600.880762] usb 4-1: Invalid ep0 maxpacket: 9
[  600.983486] usb 4-1: new full-speed USB device number 9 using uhci_hcd
[  601.004615] usb 4-1: Invalid ep0 maxpacket: 9
[  601.005637] usb usb4-port1: unable to enumerate USB device

qemu-kvm-rhev got the necessary support, I think the 7.3 kernel got whatever was needed too, so hopefully this will all work in 7.3. This will need to be tested though to make sure of it ;)

Andrei, not sure if there is anything else pending on devel side, could you test this once more as all dependencies are solved?

I tested using current RHEL7-nightly as a client and guest.

Client : virt-viewer-2.0-11.el7.x86_64
Client/Guest: kernel-3.10.0-505.el7.x86_64

Server side, however, is current RHV4.x:
qemu-kvm-rhev-2.3.0-31.el7_2.21.x86_64
spice-server-0.12.4-15.el7_2.1.x86_64

However the server packages are quite fresh: 
* Tue Aug 02 2016 Miroslav Rezanina <mrezanin> - rhev-2.3.0-31.el7_2.21
* Mon Apr 25 2016 Christophe Fergeau <cfergeau> - 0.12.4-15.1


On the client I can see that USB3 device:

$ lsusb -t
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/10p, 5000M
    |__ Port 3: Dev 2, If 0, Class=Mass Storage, Driver=usb-storage, 5000M
    |__ Port 6: Dev 3, If 0, Class=Vendor Specific Class, Driver=ax88179_178a, 5000M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/16p, 480M
    |__ Port 6: Dev 2, If 0, Class=Human Interface Device, Driver=usbhid, 1.5M
    |__ Port 6: Dev 2, If 1, Class=Human Interface Device, Driver=usbhid, 1.5M
    |__ Port 7: Dev 3, If 0, Class=Human Interface Device, Driver=usbhid, 1.5M
    |__ Port 10: Dev 4, If 0, Class=Mass Storage, Driver=usb-storage, 480M

As you can see that there are: USB3 hub + USB3 device:

Port 6: Dev 3, If 0, Class=Vendor Specific Class, Driver=ax88179_178a, 5000M


When I connect to the guest + forward the USB3 device:

There are no USB3 hub:

# lsusb -t
/:  Bus 04.Port 1: Dev 1, Class=root_hub, Driver=uhci_hcd/2p, 12M
/:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=uhci_hcd/2p, 12M
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=uhci_hcd/2p, 12M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=ehci-pci/6p, 480M

dmesg prints:

[ 1207.689246] usb 1-4: new high-speed USB device number 4 using ehci-pci
[ 1207.798239] usb 1-4: device descriptor read/64, error 18
[ 1208.011222] usb 1-4: device descriptor read/64, error 18
[ 1208.214254] usb 1-4: new high-speed USB device number 5 using ehci-pci
[ 1208.323210] usb 1-4: device descriptor read/64, error 18
[ 1208.533200] usb 1-4: device descriptor read/64, error 18
[ 1208.736209] usb 1-4: new high-speed USB device number 6 using ehci-pci
[ 1208.754188] usb 1-4: Invalid ep0 maxpacket: 9
[ 1208.858148] usb 1-4: new high-speed USB device number 7 using ehci-pci
[ 1208.938405] usb 1-4: Invalid ep0 maxpacket: 9
[ 1208.940178] usb usb1-port4: unable to enumerate USB device
[ 1209.164174] usb 3-2: new full-speed USB device number 2 using uhci_hcd
[ 1209.307185] usb 3-2: device descriptor read/64, error 18
[ 1209.525151] usb 3-2: device descriptor read/64, error 18
[ 1209.746191] usb 3-2: new full-speed USB device number 3 using uhci_hcd
[ 1209.863152] usb 3-2: device descriptor read/64, error 18
[ 1210.080123] usb 3-2: device descriptor read/64, error 18
[ 1210.283136] usb 3-2: new full-speed USB device number 4 using uhci_hcd
[ 1210.301102] usb 3-2: Invalid ep0 maxpacket: 9
[ 1210.404124] usb 3-2: new full-speed USB device number 5 using uhci_hcd
[ 1210.434265] usb 3-2: Invalid ep0 maxpacket: 9
[ 1210.435314] usb usb3-port2: unable to enumerate USB device



/usr/libexec/qemu-kvm -name RHEL7NIGHTLY -S -machine pc-i440fx-rhel7.2.0,accel=kvm,usb=off -cpu SandyBridge -m size=2072576k,slots=16,maxmem=4294967296k -realtime mlock=off -smp 1,maxcpus=16,sockets=16,cores=1,threads=1 -numa node,nodeid=0,cpus=0,mem=2024 -uuid cedd4d91-81d1-41fc-90ca-fae7692bd210 -smbios type=1,manufacturer=Red Hat,product=RHEV Hypervisor,version=7.2-9.el7,serial=4C4C4544-0054-4710-8059-B1C04F37354A,uuid=cedd4d91-81d1-41fc-90ca-fae7692bd210 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-RHEL7NIGHTLY/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=2016-09-13T15:18:35,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -boot menu=on,splash-time=10000,strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x5.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x5 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x5.0x2 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x5.0x1 -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x4 -device virtio-serial-pci,id=virtio-serial0,max_ports=16,bus=pci.0,addr=0x6 -drive if=none,id=drive-ide0-1-0,readonly=on,format=raw -device ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -drive file=/rhev/data-center/00000001-0001-0001-0001-0000000000f5/ae0bc21f-bbac-4641-b518-b77053b8dc70/images/3bf07142-fa6f-40af-8dc9-cb6ab3089fca/10bc1ea8-003a-43b8-a793-6747cdc32cad,if=none,id=drive-virtio-disk0,format=raw,serial=3bf07142-fa6f-40af-8dc9-cb6ab3089fca,cache=none,werror=stop,rerror=stop,aio=native -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x7,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=41,id=hostnet0,vhost=on,vhostfd=42 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:1a:4a:16:01:6b,bus=pci.0,addr=0x3,bootindex=2 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/channels/cedd4d91-81d1-41fc-90ca-fae7692bd210.com.redhat.rhevm.vdsm,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.rhevm.vdsm -chardev socket,id=charchannel1,path=/var/lib/libvirt/qemu/channels/cedd4d91-81d1-41fc-90ca-fae7692bd210.org.qemu.guest_agent.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0 -chardev spicevmc,id=charchannel2,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=3,chardev=charchannel2,id=channel2,name=com.redhat.spice.0 -spice tls-port=5907,addr=0,x509-dir=/etc/pki/vdsm/libvirt-spice,tls-channel=default,tls-channel=main,tls-channel=display,tls-channel=inputs,tls-channel=cursor,tls-channel=playback,tls-channel=record,tls-channel=smartcard,tls-channel=usbredir,seamless-migration=on -device qxl-vga,id=video0,ram_size=268435456,vram_size=8388608,vgamem_mb=64,bus=pci.0,addr=0x2 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0 -chardev spicevmc,id=charredir1,name=usbredir -device usb-redir,chardev=charredir1,id=redir1 -chardev spicevmc,id=charredir2,name=usbredir -device usb-redir,chardev=charredir2,id=redir2 -chardev spicevmc,id=charredir3,name=usbredir -device usb-redir,chardev=charredir3,id=redir3 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -msg timestamp=on

FWIW it doesn't work for me even with components that should work:

kernel-3.10.0-505.el7.x86_64
usbredir-0.7.1-1.el7.x86_64
libusbx-1.0.20-1.el7.x86_64
qemu-kvm-rhev-2.6.0-23.el7.x86_64
spice-server-0.12.4-19.el7.x86_64
spice-gtk3-0.31-6.el7.x86_64
virt-viewer-2.0-11.el7.x86_64

My USB3 device is USB flash, remote-viewer indicates it's redirected but lsusb in the guest won't show it

We will need to investigate what is missing. Moving to 7.4.

(In reply to David Jaša from comment #15)
> FWIW it doesn't work for me even with components that should work:
> 
> kernel-3.10.0-505.el7.x86_64
> usbredir-0.7.1-1.el7.x86_64
> libusbx-1.0.20-1.el7.x86_64
> qemu-kvm-rhev-2.6.0-23.el7.x86_64
> spice-server-0.12.4-19.el7.x86_64
> spice-gtk3-0.31-6.el7.x86_64
> virt-viewer-2.0-11.el7.x86_64
> 
> My USB3 device is USB flash, remote-viewer indicates it's redirected but
> lsusb in the guest won't show it

To test usb-3 support you need to tell qemu to emulate a xhci controller, not an ehci/uhci pair.

(In reply to Hans de Goede from comment #18)
> (In reply to David Jaša from comment #15)
> > FWIW it doesn't work for me even with components that should work:
> > 
> > kernel-3.10.0-505.el7.x86_64
> > usbredir-0.7.1-1.el7.x86_64
> > libusbx-1.0.20-1.el7.x86_64
> > qemu-kvm-rhev-2.6.0-23.el7.x86_64
> > spice-server-0.12.4-19.el7.x86_64
> > spice-gtk3-0.31-6.el7.x86_64
> > virt-viewer-2.0-11.el7.x86_64
> > 
> > My USB3 device is USB flash, remote-viewer indicates it's redirected but
> > lsusb in the guest won't show it
> 
> To test usb-3 support you need to tell qemu to emulate a xhci controller,
> not an ehci/uhci pair.

Sure, I do emulate nec-xhci.

I tested also with standard ehci+uhci emulated hubs with an expectation that the device would fall back to USB2 mode, but that's not the case. The behaviour is actually the same with both hubs: client reports device as redirected but it isn't visible in the guest.

I have to add more information: after nec-xhci --> ehci+uhci --> nec-xhci changes, USB3 device (a different one than in earlier testing) now gets successfully redirected. Logs were silent at even most verbose level.

(In reply to David Jaša from comment #20)
> I have to add more information: after nec-xhci --> ehci+uhci --> nec-xhci
> changes, USB3 device (a different one than in earlier testing) now gets
> successfully redirected. Logs were silent at even most verbose level.

Ok, good that means that it looks like at least all the spice + spice-client bits are in place (although you would need to test with an uas device to make sure).

Assuming all the spice bits are ok, then this really becomes a qemu controller emulation issue, at which point I suggest talking to Gerd Hoffmann about this.

One thing to check is in which order things were build, usbredir needs to be built with a new enough libusb in its buildroot. If the latest usbredir build was done before the libusb rebase then that at least is going to be part of the problem.

Likewise for qemu when using qemu's build-in redirection on the host machine. Also spice-gtk must be build against the latest usbredir, so if usbredir was rebased after spice-gtk, spice-gtk is going to need a rebuild too.

(In reply to Hans de Goede from comment #21)
> One thing to check is in which order things were build, usbredir needs to be
> built with a new enough libusb in its buildroot. If the latest usbredir
> build was done before the libusb rebase then that at least is going to be
> part of the problem.
> 

Ah, usbredir was indeed built with an older libusbx in its chroot:
I assume libusbx-1.0.20-1.el7 got the needed bits. Latest usbredir is usbredir-0.7.1-1.el7 which was built against 1.0.15-4.el7

> Also spice-gtk must be build against the latest usbredir, so if
> usbredir was rebased after spice-gtk, spice-gtk is going to need a rebuild
> too.

Latest spice-gtk (spice-gtk-0.31-6.el7) was built against libusbx-1.0.20-1.el7/usbredir-0.7.1-1.el7. Dunno if it matters that the usbredir it was built against was not built with a new enough libusbx.

(In reply to Christophe Fergeau from comment #22)
> (In reply to Hans de Goede from comment #21)
> > One thing to check is in which order things were build, usbredir needs to be
> > built with a new enough libusb in its buildroot. If the latest usbredir
> > build was done before the libusb rebase then that at least is going to be
> > part of the problem.
> > 
> 
> Ah, usbredir was indeed built with an older libusbx in its chroot:
> I assume libusbx-1.0.20-1.el7 got the needed bits.

Yes 1.0.20 has all the needed bits.

> > Also spice-gtk must be build against the latest usbredir, so if
> > usbredir was rebased after spice-gtk, spice-gtk is going to need a rebuild
> > too.
> 
> Latest spice-gtk (spice-gtk-0.31-6.el7) was built against
> libusbx-1.0.20-1.el7/usbredir-0.7.1-1.el7. Dunno if it matters that the
> usbredir it was built against was not built with a new enough libusbx.

No that does not matter, libusbredirhost disables usb-3 bulk streams functionality at compile time when build against a too old libusb, but this is an internal thing, the API it exposes does not change because of this. So I believe you only need to rebuild usbredir.

Created attachment 1203392 [details]
60_usb3

a hook for USB 3 testing on RHV hosts. Add 'usb3' custom property (value setting not necessary)[1] to VM properties and launch the VM.

[1] "usb3=^$", https://access.redhat.com/documentation/en/red-hat-virtualization/4.0/paged/administration-guide/a7-defining-custom-properties

We only need to rebuild usbredir with newer version of libusbx to have this.

Martin / David, shouldn't we change the target release as well here, just to make the bug more clear?

(In reply to Marina from comment #30)
> Martin / David, shouldn't we change the target release as well here, just to
> make the bug more clear?

Thanks, make sense, done.

*** Bug 1402821 has been marked as a duplicate of this bug. ***

*** Bug 1426537 has been marked as a duplicate of this bug. ***

*** Bug 1425961 has been marked as a duplicate of this bug. ***

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:1849