Bug 514902

Summary: Migration with virsh command fail on xen i386
Product: Red Hat Enterprise Linux 5 Reporter: zhanghaiyan <yoyzhang>
Component: libvirtAssignee: Daniel Veillard <veillard>
Status: CLOSED DUPLICATE QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: low    
Version: 5.4CC: berrange, clalance, hbrock, jdenemar, kelvin.zhao, llim, mrezanin, tyan, virt-maint, xen-maint
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-10-19 10:15:51 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
capabilities info
none
guest xml
none
pool xml
none
verison info
none
debug info
none
source xend log
none
target xend log
none
rhel56-source-xend.log
none
rhel56-source-xend-debug.log
none
rhel56-target-xend.log
none
rhel56-target-xend-debug.log none

Description zhanghaiyan 2009-07-31 11:42:50 UTC
Description of problem:
Use virsh command to migrate fail on xen i386 while can migrate success with virt-manager

Version-Release number of selected component (if applicable):
- RHEL5.4-Client-i386
- 2.6.18-158.el5xen 
- xen-3.0.3-92.el5
- libvirt-0.6.3-17.el5


How reproducible:
Always

Steps to Reproduce:
On both machine
1.# virsh pool-define pool // (attach pool)
2.# virsh pool-start pool
3.# iptables -F
4.# setenforce 0
5.Config /etc/xen/xend-config.sxp

On source machine
1.# virsh define guest // (attach guest)
2.# virsh start test1
3.# virsh migrate --live test1 xen+ssh://10.66.70.188

  
Actual results:
[root@dhcp-66-70-82 Desktop]#  virsh migrate --live test1 xen+ssh://10.66.70.188
root.70.188's password:
error: Domain not found: xenUnifiedDomainLookupByName

And in source machine, test1 is reboot
in target machine, cannot find test1


Expected results:
Migrate successfully

Additional info:
Can migrate success in virt-manager
Attach pool xml, guest xml, capabilities info, system version info and debug info.

Comment 1 zhanghaiyan 2009-07-31 11:43:35 UTC
Created attachment 355796 [details]
capabilities info

Comment 2 zhanghaiyan 2009-07-31 11:44:02 UTC
Created attachment 355797 [details]
guest xml

Comment 3 zhanghaiyan 2009-07-31 11:44:20 UTC
Created attachment 355798 [details]
pool xml

Comment 4 zhanghaiyan 2009-07-31 11:44:44 UTC
Created attachment 355799 [details]
verison info

Comment 5 zhanghaiyan 2009-07-31 11:45:04 UTC
Created attachment 355800 [details]
debug info

Comment 6 Daniel Berrangé 2009-07-31 12:18:00 UTC
The debug log here has this line:

18:36:50.765: debug : xenDaemonDomainMigratePerform:4431 : hostname = localhost.localdomain, port = 0
18:36:57.355: debug : xenDaemonDomainMigratePerform:4445 : migration done


which says that XenD told libvirt that the migration operation was complete.

This error

error: Domain not found: xenUnifiedDomainLookupByName

says that the domain does not exist in the target though.

So either migration failed and XenD didn't tell libvirt there was an error.
Or migration succeeeded, but the guest then crashed.


I think we need the XenD logs to diagnose this - it doesn't sound like a libvirt bug currently.

Can you provide the /var/log/xen/xend.log & xend-error.log files, from both source & target hosts.

Comment 7 zhanghaiyan 2009-08-02 01:45:26 UTC
Attach source-xend.log and source-xend-debug.log (whose content is null)

Comment 8 zhanghaiyan 2009-08-02 01:45:52 UTC
Created attachment 355907 [details]
source xend log

Comment 9 zhanghaiyan 2009-08-02 01:48:03 UTC
Attach target-xend.log.
Cannot attach source-xend-debug.log and target-xend-debug.log while both of them are null.

Comment 10 zhanghaiyan 2009-08-02 01:48:25 UTC
Created attachment 355908 [details]
target xend log

Comment 11 zhanghaiyan 2009-08-05 06:40:43 UTC
Can migrate successfully with xm command:
# xm migrate --live test1 10.66.70.188

And I notice that, after run command 
# virsh migrate --live test1 xen+ssh://10.66.70.188
the file /etc/xen/test1 is deleted, so cause an error:
error: Domain not found: xenUnifiedDomainLookupByName


Hope this info can be useful

Comment 13 Andrew Ross 2009-12-04 04:22:43 UTC
> Actual results:
> [root@dhcp-66-70-82 Desktop]#  virsh migrate --live test1
> xen+ssh://10.66.70.188
> root.70.188's password:
> error: Domain not found: xenUnifiedDomainLookupByName
> 
> And in source machine, test1 is reboot
> in target machine, cannot find test1
> 

While verifying bug 449696 I got a different error:

[root@localhost ~]# virsh migrate --live testvm-F10 xen+ssh://192.168.0.11
root.0.11's password:
error: POST operation failed: xend_post: error from xen daemon: (xend.err "can't connect: Name or service not known")

However xm migrate worked fine with the same setup.

Comment 14 Miroslav Rezanina 2010-01-06 10:12:13 UTC
As xen migration itself works, problem is probably in way libvirt handle it.

Comment 15 Daniel Veillard 2010-01-14 15:27:21 UTC
I don't think we can cover this for 5.5 at this point, retargetting it for 5.6

Daniel

Comment 20 zhanghaiyan 2010-09-17 06:17:17 UTC
Reproduced this bug via virsh and virt-manager with the same error, while can successfully migrate via xm.
RHEL5.6-Server-i386
- xen-3.0.3-116.el5
- kernel-xen-2.6.18-219.el5
- libvirt-0.8.2-4.el5

[root@dhcp-93-192 etc]# virsh migrate --live 13 xen+ssh://10.66.93.12
root.93.12's password: 
error: Domain not found: xenUnifiedDomainLookupByName

Still, after executing the command, the file /etc/xen/rhel-5.5-32-hvm is deleted.

Comment 21 zhanghaiyan 2010-09-17 06:19:11 UTC
Created attachment 447907 [details]
rhel56-source-xend.log

Comment 22 zhanghaiyan 2010-09-17 06:19:45 UTC
Created attachment 447908 [details]
rhel56-source-xend-debug.log

Comment 23 zhanghaiyan 2010-09-17 06:20:19 UTC
Created attachment 447909 [details]
rhel56-target-xend.log

Comment 24 zhanghaiyan 2010-09-17 06:20:42 UTC
Created attachment 447910 [details]
rhel56-target-xend-debug.log

Comment 25 Jiri Denemark 2010-10-14 12:40:43 UTC
Migration works just fine for me on xen on both i386 and x86_64. Any chance I could get to the machines where you are seeing the issue?

Comment 27 Jiri Denemark 2010-10-19 09:05:07 UTC
For comment 26: The problem was missing DNS entry for target hostname. After adding the target machine to /etc/hosts on source host, migration finished successfully.

Comment 28 Jiri Denemark 2010-10-19 10:15:51 UTC
We agreed with the reporter who tested this on some additional machines that this was a bug in 0.6.3-based libvirt package but reproducing it with 0.8.2 was most likely caused by some configuration/installation issues on the only two machines where this bug was reproduced. My suspicion is this was fixed by the patches from bug 593120. I'm closing this as a duplicate. Feel free to reopen if you hit this bug again.

*** This bug has been marked as a duplicate of bug 593120 ***