Bug 1410188
Summary: | Virtual machines will not boot with more than 6 USB devices being passed through | ||
---|---|---|---|
Product: | Red Hat Enterprise Linux 7 | Reporter: | emahoney |
Component: | libvirt | Assignee: | Ján Tomko <jtomko> |
Status: | CLOSED ERRATA | QA Contact: | Pei Zhang <pzhang> |
Severity: | medium | Docs Contact: | |
Priority: | unspecified | ||
Version: | 7.3 | CC: | chayang, coli, dyuan, emahoney, hachen, jtomko, juzhang, knoel, kraxel, lmen, michen, qzhang, rbalakri, virt-maint, xuma, xutian, xuzhang |
Target Milestone: | rc | ||
Target Release: | --- | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | libvirt-3.1.0-1.el7 | Doc Type: | If docs needed, set a value |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2017-08-01 17:21:45 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: |
Description
emahoney
2017-01-04 17:26:06 UTC
Please attach libvirt config for both 7.2 and 7.3 Gerd, this is the error being received on the guest: ################################# internalError: No free USB ports file "/usr/lib64/python2.7/site-packages/libvirt.py", line 536, in attachDevice if ret == -1: raise libvirtError ('virDomainAttachDevice() failed, dom=self') libvirtError: internal error: No free USB ports ################################## I cannot provide the libvirt config, but all options in the files are commented out. So it is a default config. -mahoney > ################################# > internalError: No free USB ports > > file "/usr/lib64/python2.7/site-packages/libvirt.py", line 536, in > attachDevice > if ret == -1: raise libvirtError ('virDomainAttachDevice() failed, dom=self') > > libvirtError: internal error: No free USB ports > ################################## That looks like libvirt not being able to handle the domain configuration after upgrading. > I cannot provide the libvirt config, but all options in the files are > commented out. So it is a default config. Oh, I mean the libvirt xml config (i.e. the output of "virsh dumpxml $domain"). I suspect what causes problems here is that libvirt starts managing usb ports in recent versions. Old libvirt versions leave that to qemu. qemu will then automatically add a usb hub to avoid running out of usb ports. You can see that with "lsusb -t" (in the guest). Problem with that approach is that it can break live migration in case usb devices end up in different ports on source and target host (can happen if you hotplug usb devices). New libvirt versions manage the usb ports, and in that case usb hubs must be explicitly added to the libvirt domain config. Appearently libvirt doesn't handle that case automatically after upgrading. Try removing usb devices until you have 5 only, then add a usb hub, then re-add the devices. That should get things going how they did on 7.2 libvirt xml syntax looks like this for a hub: <hub type='usb'> <address type='usb' bus='0' port='6'/> </hub> plug usb devices into the hub this way: <input type='tablet' bus='usb'> <address type='usb' bus='0' port='6.1'/> </input> port='6.1' specifies the usb port, "6" is the root port (where the hub is plugged in) and "1" means the first port of the hub. Alternative approach #1: Add a second usb2 host controller, so you have 12 root ports. Alternative approach #2: Use a xhci (usb3 host controller) instead, which allows to configure the number of root ports (4 by default, up to 15 are possible): <controller type='usb' index='0' model='nec-xhci' ports='8'/> Moving bug to libvirt. Pushed upstream as: commit 3ac97c2dedd2a4715d06cf506be3727df32e79eb Author: Ján Tomko <jtomko> CommitDate: 2017-01-31 13:09:08 +0100 qemu: Add enough USB hubs to accomodate all devices Commit 815d98a started auto-adding one hub if there are more USB devices than available USB ports. This was a strange choice, since there might be even more devices. Before USB address allocation was implemented in libvirt, QEMU automatically added a new USB hub if the old one was full. Adjust the logic to try adding as many hubs as will be needed to plug in all the specified devices. https://bugzilla.redhat.com/show_bug.cgi?id=1410188 git describe: v3.0.0-95-g3ac97c2 Version : libvirt-3.2.0-9.el7.x86_64 qemu-kvm-rhev-2.9.0-9.el7.x86_64 Steps : Define a vm with multiple usb devices (more than 6) 1. Prepare vm1 xml as following : ...... <disk type='file' device='disk'> <driver name='qemu' type='qcow2'/> <source file='/var/lib/libvirt/images/r74.qcow2'/> <target dev='vda' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> </disk> <disk type='file' device='disk'> <driver name='qemu' type='raw'/> <source file='/var/lib/libvirt/images/usb/usba.img'/> <target dev='sda' bus='usb'/> </disk> ...... <disk type='file' device='disk'> <driver name='qemu' type='raw'/> <source file='/var/lib/libvirt/images/usb/usbe.img'/> <target dev='sde' bus='usb'/> </disk> ...... <redirdev bus='usb' type='spicevmc'> </redirdev> <redirdev bus='usb' type='spicevmc'> </redirdev> <redirdev bus='usb' type='spicevmc'> </redirdev> <redirdev bus='usb' type='spicevmc'> </redirdev> 2. Define and start vm1 # virsh define vm1.xml Domain vm1 defined from vm1.xml # virsh start vm1 Domain vm1 started 3. Check domain xml, it will add a hub automatically. ...... <hub type='usb'> <address type='usb' bus='0' port='1'/> </hub> ...... Also check using virsh edit to add multiple usb devices without address, save and start vm successfully. Hi, I want to cold plug multiple USB disks then start guest. It seems that usb hub will not be added automatically for cold plug. I wonder if adding usb hubs automatically only works well for define or edit VMs. 1. Define a guest without USB disks. # virsh list --all Id Name State ---------------------------------------------------- - vm1 shut off # virsh domblklist vm1 Target Source ------------------------------------------------ vda /var/lib/libvirt/images/r74.qcow2 # virsh dumpxml vm1 | grep usb -A 3 ...... <controller type='usb' index='0' model='ich9-ehci1'> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x7'/> </controller> <controller type='usb' index='0' model='ich9-uhci1'> <master startport='0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0' multifunction='on'/> </controller> <controller type='usb' index='0' model='ich9-uhci2'> <master startport='2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x1'/> </controller> <controller type='usb' index='0' model='ich9-uhci3'> <master startport='4'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x2'/> </controller> -- <redirdev bus='usb' type='spicevmc'> <address type='usb' bus='0' port='1'/> </redirdev> <redirdev bus='usb' type='spicevmc'> <address type='usb' bus='0' port='2'/> </redirdev> 2. Cold plug multiple USB disks # for i in {a..h}; do virsh attach-disk vm1 /var/lib/libvirt/images/usb/usb$i.img --targetbus usb sd$i --config; done Disk attached successfully Disk attached successfully Disk attached successfully Disk attached successfully error: Failed to attach disk error: internal error: No free USB ports error: Failed to attach disk error: internal error: No free USB ports ...... The code does not plug in another hub when you fill up the last free port with a non-hub device. This also happens on hotplug. And I believe this deserves a separate bug. Thanks a lot for your info. I filed another Bug 1461633 to track issue in comment 10. And move this bug to verified. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:1846 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:1846 |