Bug 562017 - Virsh migrate fails with "error: Unknown Failure"
Summary: Virsh migrate fails with "error: Unknown Failure"
Keywords:
Status: CLOSED DUPLICATE of bug 499750
Alias: None
Product: Fedora
Classification: Fedora
Component: libvirt
Version: 12
Hardware: x86_64
OS: Linux
low
high
Target Milestone: ---
Assignee: Daniel Veillard
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2010-02-05 00:23 UTC by David
Modified: 2010-05-26 20:37 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2010-05-26 20:37:48 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
/proc/cpuinfo for the client. (2.97 KB, application/octet-stream)
2010-02-05 00:23 UTC, David
no flags Details
client /var/log/libvirt/qemu log info (1.25 KB, application/octet-stream)
2010-02-05 00:26 UTC, David
no flags Details
client lspci info (1.75 KB, application/octet-stream)
2010-02-05 00:26 UTC, David
no flags Details

Description David 2010-02-05 00:23:39 UTC
Created attachment 388932 [details]
/proc/cpuinfo for the client.

Description of problem:
Attempting to migrate a running or stopped VM fails in all cases.

Version-Release number of selected component (if applicable):
virsh # version
Compiled against library: libvir 0.7.1
Using library: libvir 0.7.1
Using API: QEMU 0.7.1
Running hypervisor: QEMU 0.11.0


Linux savant.aml.cs.byu.edu 2.6.31.9-174.fc12.x86_64 #1 SMP Mon Dec 21 05:33:33 UTC 2009 x86_64 x86_64 x86_64 GNU/Linux


How reproducible:
Always


Steps to Reproduce:
1. Restore a VM on the local machine.

virsh # restore /aml/images/pa4.img
Domain restored from /aml/images/pa4.img

virsh # list
 Id Name                 State
----------------------------------
 13 pa4                  running

virsh # uri
qemu:///system

2. Migrate the VM restored to another machine.


virsh # migrate --live pa4 qemu+ssh://3potato/system
error: Unknown failure

virsh # migrate pa4 qemu+ssh://3potato/system
error: Unknown failure

3. Verify that libvirtd is running on the other machine. 

virsh # connect qemu+ssh://3potato/system

virsh # list
 Id Name                 State
----------------------------------
  2 pa8                  running

Actual results:
An "error: Unknown failure" is returned to the console when migrate is invoked.


Expected results:
The VM migration should succeed. The VM should stop on the host and should start on the client.


Additional info:

This is in a lab environment with all firewalls and selinux disabled on all
physical machines. 


package kvm is not installed
python-virtinst-0.500.1-2.fc12.noarch
package virt-viewer is not installed
virt-manager-0.8.2-1.fc12.noarch

Comment 1 David 2010-02-05 00:26:29 UTC
Created attachment 388933 [details]
client /var/log/libvirt/qemu log info

Comment 2 David 2010-02-05 00:26:51 UTC
Created attachment 388934 [details]
client lspci info

Comment 3 David 2010-02-05 00:29:22 UTC
I replicated the error with LIBVIRT_DEBUG=1 /usr/sbin/libvirtd --verbose running on the client, but nothing extra was printed out.

Comment 4 David 2010-02-05 22:49:23 UTC
I updated the host and the client both to version 0.7.6. I still have an error. However, the error message I received from virsh is now: 

error: operation failed: migration to 'tcp:0potato.aml.cs.byu.edu:49152' failed: migration failed

in /var/log/messages, libvirtd created this: Feb  5 15:43:08 savant libvirtd: 15:43:08.196: error : qemuMonitorTextMigrate:1075 : operation failed: migration to 'tcp:0potato.aml.cs.byu.edu:49152' failed: migration failed#015#012


When I ran LIBVIRT_DEBUG=1 /usr/sbin/libvirtd --verbose, the only log message created was: libvir: error : Unknown failure

Comment 5 Christopher Hunt 2010-02-24 00:20:54 UTC
could this be a duplicate of 540715?

-Chris

Comment 6 David 2010-02-24 00:32:50 UTC
I spent about 15 hours stepping through libvirt trying to track down this bug. I'm pretty sure it's a problem in qemu.

The host has a read-only file system. qemu (nor libvirt for that matter) don't add an entry in rwtab.d for read-only file systems.

Comment 7 Chris Lalancette 2010-02-24 14:04:53 UTC
(In reply to comment #6)
> I spent about 15 hours stepping through libvirt trying to track down this bug.
> I'm pretty sure it's a problem in qemu.
> 
> The host has a read-only file system. qemu (nor libvirt for that matter) don't
> add an entry in rwtab.d for read-only file systems.    

Hm, you have multiple interesting things going on here.  The first is that live migration in raw qemu (without KVM) isn't very heavily tested, so you may be running into bugs there.  The second is that a read-only filesystem is also not a heavily tested feature, especially with live migration.  The third is that you may indeed be running into BZ 540715.  Can you give the output of "cat /etc/hosts"?

Chris Lalancette

Comment 8 David 2010-02-24 14:21:34 UTC
Here is the content of /etc/hosts. It is the same on the server migrating from and server migrating to. http://pastebin.com/KjCNkybL

Comment 9 Chris Lalancette 2010-02-24 14:31:40 UTC
(In reply to comment #8)
> Here is the content of /etc/hosts. It is the same on the server migrating from
> and server migrating to. http://pastebin.com/KjCNkybL    

OK, thanks.  You are not running into BZ 540715, then; you have different problems.  At the moment, I don't have any good ideas, although obviously "Unknown error" isn't very helpful.  We should fix that.

Chris Lalancette

Comment 10 Cole Robinson 2010-02-24 15:16:45 UTC
I committed a series of patches upstream a while ago that fixed the 'unknown error' case. Unfortunately the best you will probably get is 'migrate did not successfully complete' which is marginally better. We should find a way to get better migration error reporting out of qemu.

Comment 11 Cole Robinson 2010-05-26 20:37:48 UTC
I've filed a qemu bug asking for better error reporting:

https://bugzilla.redhat.com/show_bug.cgi?id=596506

Either way, let's just use 499750 to track the 'unknown error' issue.

*** This bug has been marked as a duplicate of bug 499750 ***


Note You need to log in before you can comment on or make changes to this bug.