Bug 856864

Summary: Do live migration from rhel6.1.z release version to rhel6.4 newest version and back will get "error Unknown controller type 'usb'"
Product: Red Hat Enterprise Linux 6 Reporter: EricLee <bili>
Component: libvirtAssignee: Ján Tomko <jtomko>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 6.4CC: acathrow, dallan, dyasny, dyuan, jdenemar, juzhang, laine, lsu, mzhan, rwu, weizhan, whuang, yupzhang, zhpeng
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-0.10.2-5.el6 Doc Type: Bug Fix
Doc Text:
Cause: Libvirt was putting the default USB controller into the XML when doing a v2 migration. Consequence: Migration to RHEL 6.1 failed, because it doesn't understand USB controllers. Fix: Don't include the default USB controller in XML while doing a v2 migration. Result: Migration to RHEL 6.1. works.
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-02-21 07:23:37 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 863059    
Bug Blocks:    
Attachments:
Description Flags
xml on 6.1.z host
none
xml on 6.4 host
none
xml on 6.2.z host
none
xml on 6.4 host of r6u2 guest none

Description EricLee 2012-09-13 02:06:15 UTC
Description of problem:
Do live migration from rhel6.1.z release version to rhel6.4 newest version and back will get "error Unknown controller type 'usb'"

Version-Release number of selected component (if applicable):
6.1.z host:
Kernel :  kernel-2.6.32-131.32.1.el6  
qemu-kvm : qemu-kvm-0.12.1.2-2.160.el6_1.9
libvirt:  libvirt-0.8.7-18.el6_1.5
6.4 host:
libvirt-0.10.1-1.el6.x86_64
qemu-kvm-0.12.1.2-2.308.el6.x86_64
kernel-2.6.32-298.el6.x86_64

How reproducible:
100%

Steps to Reproduce:
Setup:
1. Prepare 2 hosts, host 1 is rhel6.1.z release version, host 2  is newest rhel6.4 version

   Please update kernel , qemu-kvm and libvirt package to z-stream on rhel6.1.z host
   For now ,
   Kernel :  kernel-2.6.32-131.32.1.el6  
   qemu-kvm : qemu-kvm-0.12.1.2-2.160.el6_1.9
   libvirt:  libvirt-0.8.7-18.el6_1.5

  How to find the newest z-stream package:
  Search kernel-2.6.32-131* , qemu-kvm-0.12.1.2-2.160.el6* and  libvirt-0.8.7-18.el6*  separately with Build option in brewweb.

2. Prepare a nfs which is mounted on both hosts, and setting the virt_use_nfs boolean on both sides
  # setsebool virt_use_nfs 1
   and close the iptable on both sides
  # iptables -F

Actions:
1. Prepare a guest locate on host 1(6.1.z) and start, check the qemu-kvm command
the disk xml part should be
...
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='none'/>
      <source dev='/$SHARED_DIR/kvm-rhel6u1-x86_64-new.img'/>
      <target dev='vda' bus='virtio'/>
    </disk>
...

2. Do live migration from host 1(6.1.z) to host 2(6.4)

# virsh migrate --live guest qemu+ssh://{host 2 ip}/system

3. Login guest to check the guest status

4. Do live migration from host 2(6.4) to host 1(6.1.z)

# virsh migrate --live guest qemu+ssh://{host 1 ip}/system
  
Actual results:
Step 4 will get  "error Unknown controller type 'usb'"

Expected results:
Migration should succeed without error

Additional info:
The Bug 815503
migration of rhel6.2.0 guest from RHEL6.3 host to RHEL6.2 host fails "error Unknown controller type 'usb'"  has closed as fixed.

Comment 2 EricLee 2012-09-13 05:55:06 UTC
Since Laine said he will track this bug, and let us to assign the bug to him in email, so I set the Assignee to Laine when I file this bug.

Comment 3 Laine Stump 2012-09-13 20:58:05 UTC
Jirka - this seems to be complaining about the same problem as Bug 815503. Any ideas on why your patch for that doesn't seem to be working in this case?

Comment 4 Jiri Denemark 2012-09-17 12:16:58 UTC
No idea, the fix was to avoid formatting any usb controller if there is just one default controller configured (which should be the case of 6.1.z->6.4->6.1.z migration). I guess someone must have either change that code or added something that effectively disabled the code. Anyway, this looks like a regression... I think 6.2(.z)->6.4->6.2(.z) should not work either, Eric, can you check that?

Comment 5 Laine Stump 2012-09-17 14:14:06 UTC
The odd part is that, in a separate email prior to opening this bug, the reporter says that 6.2.z-->6.4-->6.2.z *does* work (and that 6.1.z-->6.3-->6.1.z also fails):


> | From: "Laine Stump" <laine>
> |
> | Does it work properly when you migrate from
> | RHEL6.2.z->RHEL6.4->RHEL6.2.z? What about
> | RHEL6.1.z->RHEL6.3->RHEL6.1.z?
> | 
>

On 09/10/2012 10:04 PM, Bing Li wrote:
>
> I have tried:
> RHEL6.2.z->RHEL6.4->RHEL6.2.z works well;
> RHEL6.1.z->RHEL6.3->RHEL6.1.z also get the same error.


Of course it would be good to get independent verification of this result, as it seems to defy logic :-)

Comment 7 EricLee 2012-09-18 02:54:11 UTC
(In reply to comment #4)
> No idea, the fix was to avoid formatting any usb controller if there is just
> one default controller configured (which should be the case of
> 6.1.z->6.4->6.1.z migration). I guess someone must have either change that
> code or added something that effectively disabled the code. Anyway, this
> looks like a regression... I think 6.2(.z)->6.4->6.2(.z) should not work
> either, Eric, can you check that?

I have tried that again, and the results are the same with comment #5 which I tested before opening this bug:
RHEL6.2.z->RHEL6.4->RHEL6.2.z works well:
6.2.z host:
libvirt-0.9.4-23.el6_2.9
qemu-kvm-0.12.1.2-2.209.el6_2.5
kernel-2.6.32-220.26.1.el6
6.4 host:
libvirt-0.10.1-1.el6.x86_64
qemu-kvm-0.12.1.2-2.308.el6.x86_64
kernel-2.6.32-298.el6.x86_64

RHEL6.1.z->RHEL6.3->RHEL6.1.z also get the same error: "error: internal error Unknown controller type 'usb'":
6.1.z host:
Kernel :  kernel-2.6.32-131.32.1.el6  
qemu-kvm : qemu-kvm-0.12.1.2-2.160.el6_1.9
libvirt:  libvirt-0.8.7-18.el6_1.5
6.3 host:
kernel-2.6.32-279.el6.x86_64
libvirt-0.9.10-21.el6.x86_64
qemu-kvm-0.12.1.2-2.295.el6.x86_64

That maybe indicate that the patch added for fixing Bug 815503 just works on RHEL6u2 and after 6u2 version, or the package in 6.1 have some different from 6.2 which make your patch do not work.

Comment 10 Jiri Denemark 2012-09-18 07:31:11 UTC
Hmm, this is very puzzling indeed. We'll need to try ourselves while closely watching what the hack is going on with the domain when it is being migrated from pre-6.3 to post-6.3 and back.

BTW, could you attach domain XMLs of the domains before you tried to migrate them, i.e., from 6.[12].z hosts?

Comment 11 EricLee 2012-09-18 08:25:43 UTC
(In reply to comment #10)
> Hmm, this is very puzzling indeed. We'll need to try ourselves while closely
> watching what the hack is going on with the domain when it is being migrated
> from pre-6.3 to post-6.3 and back.
> 
> BTW, could you attach domain XMLs of the domains before you tried to migrate
> them, i.e., from 6.[12].z hosts?

Please see the following comments.

comment #12:
r6u1-before.xml which is used before migrating on 6.1.z host

comment #13:
r6u1-after.xml on 6.4 host which migrated from 6.1.z host

Comment 12 EricLee 2012-09-18 08:27:01 UTC
Created attachment 613897 [details]
xml on 6.1.z host

Comment 13 EricLee 2012-09-18 08:27:35 UTC
Created attachment 613898 [details]
xml on 6.4 host

Comment 14 Laine Stump 2012-09-18 08:35:52 UTC
How about the pre-migration domain xml from the 6.2.z host (where migration does work)?

Comment 15 EricLee 2012-09-18 08:57:51 UTC
(In reply to comment #14)
> How about the pre-migration domain xml from the 6.2.z host (where migration
> does work)?

Please see comment #16 and comment #17.

Comment 16 EricLee 2012-09-18 08:58:28 UTC
Created attachment 613913 [details]
xml on 6.2.z host

Comment 17 EricLee 2012-09-18 08:59:14 UTC
Created attachment 613915 [details]
xml on 6.4 host of r6u2 guest

Comment 18 Ján Tomko 2012-10-11 12:01:45 UTC
RHEL-6.2 doesn't recognize USB controllers, but it supports v3 migration which got fixed by bug 815503.
RHEL-6.1 only supports v2, where the XML is generated by domainGetXMLDesc, which always leaves the USB controller in, because it's the same function used by "virsh dumpxml".

Comment 19 Ján Tomko 2012-10-19 15:01:00 UTC
Upstream patch proposed:
https://www.redhat.com/archives/libvir-list/2012-October/msg01059.html

Comment 22 Luwen Su 2012-10-25 06:30:49 UTC
Clear for steps
1.Two hosts 
host1: 6.1.z 
libvirt-0.8.7-18.el6_1.5.x86_64
kernel-2.6.32-131.32.1.el6  
qemu-kvm-0.12.1.2-2.160.el6_1.9
host2: 6.4 
libvirt-0.10.2-5.el6.x86_64
qemu-kvm-0.12.1.2-2.317.el6.x86_64
kernel-2.6.32-330.el6.x86_64


2.Prepare nfs and configurate selinux rules on both two hosts for migration
# setsebool virt_use_nfs 1
# iptables -F
host1:
$nfs-server on /mnt type nfs (rw,vers=3,addr=$nfsip)
host2:
$nfs-server on /mnt type nfs (rw,vers=3,addr=$nfsip)

3.start a guest in host1 to migrate to host2 and back.
host1:
#virsh dumpxml test | grep  /mnt
<source file='/mnt/tsu-1.img'/>
#virsh migrate --live test qemu+ssh://$host2ip/system
//success
host2:
#virsh migrate --live test qemu+ssh://$host1ip/system
//success

I reproduced the bug with libvirt-0.10.2-4.el6.x86_64 ,migration successed with 
libvirt-0.10.2-5.el6.x86_64.

Comment 26 errata-xmlrpc 2013-02-21 07:23:37 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2013-0276.html