Bug 843560
Summary: | Add live migration support for USB | |||
---|---|---|---|---|
Product: | Red Hat Enterprise Linux 6 | Reporter: | Bhavna Sarathy <bsarathy> | |
Component: | libvirt | Assignee: | Jiri Denemark <jdenemar> | |
Status: | CLOSED ERRATA | QA Contact: | Virtualization Bugs <virt-bugs> | |
Severity: | high | Docs Contact: | ||
Priority: | high | |||
Version: | 6.4 | CC: | acathrow, ajia, areis, bsarathy, dallan, dyasny, dyuan, hdegoede, jdenemar, juzhang, lnovich, michen, mkenneth, mzhan, qzhou, rwu, shu, tburke, virt-maint, weizhan, zhpeng | |
Target Milestone: | rc | |||
Target Release: | --- | |||
Hardware: | All | |||
OS: | Linux | |||
Whiteboard: | ||||
Fixed In Version: | libvirt-0.10.2-5.el6 | Doc Type: | Bug Fix | |
Doc Text: | Story Points: | --- | ||
Clone Of: | 805172 | |||
: | 852624 (view as bug list) | Environment: | ||
Last Closed: | 2013-02-21 07:20:06 UTC | Type: | --- | |
Regression: | --- | Mount Type: | --- | |
Documentation: | --- | CRM: | ||
Verified Versions: | Category: | --- | ||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
Cloudforms Team: | --- | Target Upstream Version: | ||
Embargoed: | ||||
Bug Depends On: | 805172 | |||
Bug Blocks: | 832177, 798682, 804821, 838170, 840804, 852624 |
Comment 2
Dave Allan
2012-09-13 17:45:57 UTC
At least from the spice usb-redir migration pov, after adding seamless migration support, no additional changes are needed for USB migration, so I believe that this bug can be closed. qemuMigrationIsAllowed is not called during save/restore, which means everything will work just fine when saving/restoring a domain with USB passed through devices as long as the devices remain attach at the same address. In other words, if the host is not rebooted and the device is not detached from the host in the meantime. To make it work even if any of this happens, we need to make some changes to libvirt. First, we may add support for startupPolicy to USB devices which (similarly to disk devices) would tell if the device is really required to be present when starting or migrating a domain. With this, we could support device unplug between save and restore since qemu should support -device usb-host,id=... with no address and thus no real USB device. To support change in USB address, we would need to (partially) ignore the USB address in domain XML (since libvirt is able to find the right device using vendor/product), however, this would not work that well in case several identical devices are attached to the host and we need to make sure we use the right one. I'll try to think more about how to handle this. Both issues are fixed with the series I sent upstream yesterday: https://www.redhat.com/archives/libvir-list/2012-October/msg00335.html This is now implemented upstream by commits v0.10.2-129-ge658dae through v0.10.2-134-gbd1282d and v0.10.2-139-gf95560b: commit e658daeb58ec88deec5dd60fd3279f4feb4eac5b Author: Jiri Denemark <jdenemar> Date: Tue Oct 2 15:14:02 2012 +0200 conf: Add support for startupPolicy for USB devices USB devices can disappear without OS being mad about it, which makes them ideal for startupPolicy. With this attribute, USB devices can be configured to be mandatory (the default), requisite (will disappear during migration if they cannot be found), or completely optional. commit 7bcc7278bf145e2628831823f77192307f09bfa1 Author: Jiri Denemark <jdenemar> Date: Wed Oct 3 13:55:35 2012 +0200 qemu: Introduce qemuFindHostdevUSBDevice The code which looks up a USB device specified by hostdev is duplicated in two places. This patch creates a dedicated function that can be called in both places. commit 059aff6b9860b18634709bc76d64ea78f7e609db Author: Jiri Denemark <jdenemar> Date: Wed Oct 3 16:57:28 2012 +0200 qemu: Add option to treat missing USB devices as success All USB device lookup functions emit an error when they cannot find the requested device. With this patch, their caller can choose if a missing device is an error or normal condition. commit edc9269a2a80082a6898e14b1c10b4a94197b731 Author: Jiri Denemark <jdenemar> Date: Thu Oct 4 16:18:16 2012 +0200 qemu: Implement startupPolicy for USB passed through devices commit 28f8dfdcccd4c0f69063ef741545b37d8a7f7935 Author: Jiri Denemark <jdenemar> Date: Mon Oct 8 11:58:05 2012 +0200 Add MIGRATABLE flag for virDomainGetXMLDesc Using VIR_DOMAIN_XML_MIGRATABLE flag, one can request domain's XML configuration that is suitable for migration or save/restore. Such XML may contain extra run-time stuff internal to libvirt and some default configuration may be removed for better compatibility of the XML with older libvirt releases. This flag may serve as an easy way to get the XML that can be passed (after desired modifications) to APIs that accept custom XMLs, such as virDomainMigrate{,ToURI}2 or virDomainSaveFlags. commit bd1282d6247742d732f0076811521cfdc6b13472 Author: Jiri Denemark <jdenemar> Date: Tue Oct 9 13:15:46 2012 +0200 qemu: Make save/restore with USB devices usable Save/restore with passed through USB devices currently only works if the USB device can be found at the same USB address where it used to be before saving a domain. This makes sense in case a user explicitly configure the USB address in domain XML. However, if the device was found automatically by vendor/product identification, we should try to search for that device when restoring the domain and use any device we find as long as there is only one available. In other words, the USB device can now be removed and plugged again or the host can be rebooted between saving and restoring the domain. commit f95560b3fe5e239ed85ec01885ed5eb872439e98 Author: Jiri Denemark <jdenemar> Date: Thu Oct 11 22:33:46 2012 +0200 conf: Mark missing optional USB devices in domain XML When startupPolicy set for a USB devices allows such device to be missing, there was no way this could be detected from domain XML. With this patch, libvirt emits a new missing='yes' attribute for such devices when active domain XML is generated. Test on qemu-kvm-0.12.1.2-2.327.el6.x86_64 libvirt-0.10.2-3.el6.x86_64 kernel-2.6.32-329.el6.x86_64 Migrate a guest with <hostdev mode='subsystem' type='usb' managed='yes'> <source startupPolicy='requisite'> <address bus='2' device='2'/> </source> <alias name='hostdev0'/> </hostdev> report error error: Requested operation is not valid: Domain with assigned host devices cannot be migrated and with --migratable, it ony remove the <controller type='usb' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/> </controller> device, but if I set <hostdev mode='subsystem' type='usb' managed='yes'> <source> <address bus='2' device='2'/> </source> <alias name='hostdev0'/> </hostdev> there is no change in dumpxml --migratable, is that right? Oops, I forgot to change one check that forbids migration ir there are any hostdev devices configured for the domain. It will work for save/restore/managedsave scenario, though. I'll prepare the patch for fixing the migration check and include it in the next build. BTW, to really test all scenarios (such as usb device replug after managedsaving a domain), you should rather identify the usb device using vendor/product. And the --migratable flag will have greater impact in that case, too. It should remove the automatically detected address of the device. Actually, the automatically detected address should still be there but a new attribute autoAddress='yes' should appeare in the source element. I'm sorry about the confusion. The missing part is fixed upstream by v0.10.2-204-g9fcc543: commit 9fcc5436d3d824ceb82b44852e1a8023c3810655 Author: Jiri Denemark <jdenemar> Date: Fri Oct 19 13:36:33 2012 +0200 qemu: Allow migration with host USB devices Recently, patches were added support for (managed)saving, restoring, and migrating domains with host USB devices. However, qemu driver would still forbid migration of such domains because qemuMigrationIsAllowed was not updated. Verify pass on qemu-kvm-rhev-0.12.1.2-2.317.el6.x86_64 libvirt-0.10.2-5.el6.x86_64 kernel-2.6.32-329.el6.x86_64 Test scenario 1. With startupPolicy='requisite' virsh start vr-rhel6u3-x86_64-kvm error: Failed to start domain vr-rhel6u3-x86_64-kvm error: internal error Did not find USB device 951:1646 Migration, restore succeed 2. With startupPolicy='optional' Start, Migration and restore succeed 3. With startpolicy='mandatory' # virsh start vr-rhel6u3-x86_64-kvm error: Failed to start domain vr-rhel6u3-x86_64-kvm error: internal error Did not find USB device 951:1646 # virsh restore /tmp/vr-rhel6u3-x86_64-kvm.save error: Failed to restore domain from /tmp/vr-rhel6u3-x86_64-kvm.save error: internal error Did not find USB device 951:1646 # virsh migrate --live vr-rhel6u3-x86_64-kvm qemu+ssh://10.66.5.143/system --verbose root.5.143's password: error: internal error Did not find USB device 951:1646 4. Check the --migratable in dumpxml # virsh dumpxml vr-rhel6u3-x86_64-kvm --migratable ... <hostdev mode='subsystem' type='usb' managed='yes'> <source startupPolicy='mandatory' autoAddress='yes'> <vendor id='0x0951'/> <product id='0x1646'/> <address bus='2' device='41'/> </source> <alias name='hostdev0'/> </hostdev> ... and the usb controller section will be removed 5. Migrate to target without host usb or restore guest without host usb, the xml in target changed to <hostdev mode='subsystem' type='usb' managed='yes'> <source startupPolicy='optional' missing='yes'> <vendor id='0x0951'/> <product id='0x1646'/> <address bus='2' device='41'/> </source> <alias name='hostdev0'/> </hostdev> If have the usb device with the same verdor and product id on target, then the address change to new bus and device num and migration/restore can succeed with all 3 policies Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHSA-2013-0276.html |