Bug 540715

(In reply to comment #0)
> Description of problem:
> 
> I get an error "Unable to migrate guest" when trying to migrate a Windows XP 32
> bit virtual machine from one machine to another.  Guest then kernel panics
> instead of failing gracefully.

Sigh.  Can you attach the output of "cat /etc/hosts" here?  There's a bug in the anaconda shipped with F-12 that has caused this to happen, so I want to check if you are running into the same problem.

Chris Lalancette

Here is the output of my /etc/hosts file:

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

Reassigning to libvirt, since my guess is this isn't virt-manager specific. Gary, can you reproduce using virsh?

There are known error reporting issues WRT migration: it's possible that a variety of legitimate errors are being lost before returned to the user. You can possible see some interesting info by manually running 'libvirtd' on both the source and destination hosts, and watching for any error console output.

I started a virsh console on both machines, and started the guest Winxp machine on one.  Wasn't sure of the syntax of the migrate command so I may have done this wrong:

migrate --live wintest qemu+ssh://10.100.100.97/system

I received the RSA bit, and entered the password and got this back:

error: Unknown failure

After that, the Winxp guest blue screened.  If I have the command wrong, please post what I should have typed.

I may be running into the same issue.  I've attempted a live migration between two F12 hosts, once using virt-manager and once using virsh (virsh migrate --live example2 qemu+ssh://virt02/system )  Like Mr. Scarborough, the virsh command did go so far as to accept my password, and the destination host does log the vnic coming up on br0, then going back down:

Feb 23 15:59:30 virt02 kernel: device vnic.example2 entered promiscuous mode
Feb 23 15:59:30 virt02 kernel: br0: port 7(vnic.example2) entering learning state
Feb 23 15:59:30 virt02 kernel: br0: port 7(vnic.example2) entering disabled state
Feb 23 15:59:30 virt02 kernel: device vnic.example2 left promiscuous mode
Feb 23 15:59:30 virt02 kernel: br0: port 7(vnic.example2) entering disabled state

I didn't get much info from the virsh command, but virt-manager gave me:

Unable to migrate guest:
Traceback (most recent call last):
 File "/usr/share/virt-manager/virtManager/migrate.py", line 453, in _async_migrate
   vm.migrate(dstconn, migrate_uri, rate, live, secure)
 File "/usr/share/virt-manager/virtManager/domain.py", line 230, in migrate
   self._backend.migrate(destconn.vmm, flags, newname, interface, rate)
 File "/usr/lib64/python2.6/site-packages/libvirt.py", line 384, in migrate
   if ret is None:raise libvirtError('virDomainMigrate() failed', dom=self)
libvirtError: Unknown failure

Here's what my hosts file looks like:
[root@virt05 ~]# cat /etc/hosts
# hostname virt05 added to /etc/hosts by anaconda
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4 virt05
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6 virt05

x.x.x.201 virt01 virt01.example.com
x.x.x.202 virt02 virt02.example.com
x.x.x.224 virt05 virt05.example.com

-Chris

(In reply to comment #5)
> Here's what my hosts file looks like:
> [root@virt05 ~]# cat /etc/hosts
> # hostname virt05 added to /etc/hosts by anaconda
> 127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
> virt05
> ::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
> virt05

Yeah, the above two lines in your /etc/hosts is what's causing the problem.  The good news is that Fedora 13 isn't silly, and doesn't resolve your hostname to localhost.  The other good news is that we now have a patch in libvirt 0.7.7 that gives you a bit more information than "Unknown error".  The bad news is that the installer in Fedora 12 *does* put those entries in place, and will forevermore.  I guess the best solution at present is to just remove the "virt05" entry at the end of those two lines; that should fix up the issue for you.

Chris Lalancette

Mr. Lalancette,
Thanks for the pointer.  I did remove the 'virt05' bit from each line in the hosts file, but alas that had no effect. 

Interestingly, I also checked the hosts file of the target host, virt02 and it was missing the last bit. Perhaps they'd already been modified by another sysamdin...



[root@virt05 ~]# uname -a
Linux virt05 2.6.31.9-174.fc12.x86_64 #1 SMP Mon Dec 21 05:33:33 UTC 2009 x86_64 x86_64 x86_64 GNU/Linux
[root@virt05 ~]# cat /etc/issue
Fedora release 12 (Constantine)
Kernel \r on an \m (\l)

[root@virt02 ~]# uname -a
Linux virt02.reachone.com 2.6.31.5-127.fc12.x86_64 #1 SMP Sat Nov 7 21:11:14 EST 2009 x86_64 x86_64 x86_64 GNU/Linux
[root@virt02 ~]# cat /etc/issue
Fedora release 12 (Constantine)
Kernel \r on an \m (\l) 

  Any other ideas?

does turning off the firewall on both machines make any difference?

unfortunately, i'm not at liberty to do that :-)

Not even for testing purposes? Have you opened the necessary ports in your firewall to allow for migration? With qemu libvirt, this is the port range 49152-49216.

Trying this with virt-manager might help, it provides a nicer interface to the multitude of options, so you can mix and match to see if something gets the job done.

Cole, 
After thinking about it for 2 seconds I realize that of course I can do that.  It worked like a charm!  Thanks for the input.  The "live" migration did take about 2 mins until i could ssh to the guest from outside, but that might be an ARP issue, frankly I'm not clear on that.  I'll do some more research.  

So what's the fix here?  update the default iptables rules?

There's a few things we should fix:

1) Make 'Unknown failure' into an actually useful error (done upstream)
2) Somehow show an explicit error about 'Migration connection to port 1234 refused'. This may need to be done at the qemu level.
3) Document the qemu migration port range somewhere obvious: maybe a libvirt wiki page about migration and how to debug it if it goes wrong.

We shouldn't just open those ports by default, this should really be opt in, so I think just better error messages and documentation is the best way to go.

(In reply to comment #12)
> There's a few things we should fix:
> 
> 1) Make 'Unknown failure' into an actually useful error (done upstream)

Yes.

> 2) Somehow show an explicit error about 'Migration connection to port 1234
> refused'. This may need to be done at the qemu level.

Yes, for the "simple" migration case, this does need to be done at the qemu level, since qemu is the one making the connection.  For the tunnelled migration that libvirt can also do, it *should* report correct errors already.

> 3) Document the qemu migration port range somewhere obvious: maybe a libvirt
> wiki page about migration and how to debug it if it goes wrong.
> 
> We shouldn't just open those ports by default, this should really be opt in, so
> I think just better error messages and documentation is the best way to go.    

Yeah, agreed.

Chris Lalancette

There are quite a few bugs tracking this problem, duping this one to 499750. I'll be pushing an f12 build which fixes the libvirt error reporting, as well as adding an entry to the libvirt FAQ to document common migration issues and error messages.

*** This bug has been marked as a duplicate of bug 499750 ***