Bug 699953

Summary: Tunnelled migration failed from virt-manager
Product: Red Hat Enterprise Linux 6 Reporter: mliu
Component: virt-managerAssignee: Cole Robinson <crobinso>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 6.1CC: dyuan, hjiang, jwu, mliu, mzhan, zpeng
Target Milestone: rc   
Target Release: 6.2   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: virt-manager initiated tunneled migration had several defects Consequence: tunneled migration failed Fix: fixed defects Result: tunneled migration works
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-12-06 16:09:52 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 748554    
Attachments:
Description Flags
debug log for tunnelled migration
none
Screenshot for tunnelled migration error
none
migration failed on one virt-manager on the source host
none
Debug log of virt-manager for migration failure. none

Description mliu 2011-04-27 02:54:14 UTC
Description of problem:
When do tunnelled migration from virt-manager, popup error message.

Version-Release number of selected component (if applicable):
virt-manager-0.8.6-4.el6.noarch
libvirt-0.8.7-18.el6.x86_64
kernel-2.6.32-131.0.5.el6.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Prepare 2 hosts and prepare a nfs which is mounted on both hosts, and setting the virt_use_nfs boolean on both sides
  # setsebool -P virt_use_nfs 1
   and close the iptable on both sides
  # iptables -F

2. Dispatch ssh public key of source host to target host.
    Creating your local public key pair
    # ssh-keygen -t rsa
    Copying the public key to remote host
    # ssh-copy-id -i ~/.ssh/id_rsa.pub root@{target ip}

3. Run virt-manager

4. Click File->Add Connection, choose QEMU/KVM on Hypervisor, choose SSH on Method, and write down the hostname of target host, then click Connect

5. Right click guest and select migrate, select new host with dest ip, click the "Advance", choose checkbox "Tunnel migration through libvirt's daemon", then click "Migrate"
  
Actual results:
popup error message, migration cannot start.

Expected results:
migration should be successful

Additional info:
Use virsh migrate can be successful.

# virsh migrate --live --p2p --tunnelled migrateVM qemu+ssh://{target ip}/system
# virsh migrate --p2p --tunnelled migrateVM qemu+ssh://{target ip}/system

Comment 2 Cole Robinson 2011-04-27 13:04:11 UTC
What is the exact error you are seeing? Can you provide the output of virt-manager --debug when reproducing? Does tunneled migration work using virsh?

Comment 3 mliu 2011-04-28 02:52:23 UTC
Created attachment 495380 [details]
debug log for tunnelled migration

Comment 4 mliu 2011-04-28 02:53:44 UTC
Created attachment 495381 [details]
Screenshot for tunnelled migration error

Comment 5 mliu 2011-04-28 03:02:10 UTC
(In reply to comment #2)
> What is the exact error you are seeing? Can you provide the output of
> virt-manager --debug when reproducing? Does tunneled migration work using
> virsh?

1) I have created two attachments for error message.

2) Tunneled migration can work well by using virsh.

# virsh migrate --live --p2p --tunnelled migrateVM qemu+ssh://{target
ip}/system

Result: success

# virsh migrate --p2p --tunnelled migrateVM qemu+ssh://{target ip}/system

Result: success

Comment 8 Cole Robinson 2011-07-28 15:29:15 UTC
Fixed in virt-manager-0.9.0-2.el6

Comment 10 Huming Jiang 2011-08-05 10:13:08 UTC
Created attachment 516868 [details]
migration failed on one virt-manager on the source host

Comment 11 Huming Jiang 2011-08-05 10:14:13 UTC
Created attachment 516869 [details]
Debug log of virt-manager for migration failure.

Comment 12 Huming Jiang 2011-08-05 10:17:53 UTC
Test with the following components:
virt-manager-0.9.0-5.el6.x86_64
qemu-kvm-0.12.1.2-2.175.el6.x86_64
libvirt-0.9.4-1.el6.x86_64
python-virtinst-0.600.0-2.el6.noarch
kernel-2.6.32-171.el6.x86_64

Steps:
1. Prepare 2 hosts and prepare a nfs which is mounted on both hosts, and
setting the virt_use_nfs boolean on both sides
  # setsebool -P virt_use_nfs 1
   and close the iptable on both sides
  # iptables -F
Edit /etc/hosts file on both hosts, add the following sentence to the file.
{IP_addr_of_source_hosts}  {HOSTNAME_of_source_hosts}
{IP_addr_of_target_hosts}  {HOSTNAME_of_target_hosts}

2. Dispatch ssh public key of source host to target host, and also from target host to source host.
    Creating your local public key pair
    # ssh-keygen -t rsa
    Copying the public key to remote host
    # ssh-copy-id -i ~/.ssh/id_rsa.pub root@{target ip}

The following steps is done on one virt-manager on the source host.
3. Run virt-manager

4. Click File->Add Connection, choose QEMU/KVM on Hypervisor, choose SSH on
Method, and write down the hostname of target host, then click Connect

5. Right click guest below the localhost connect and select migrate, select new host with dest ip, click
the "Advance", choose checkbox "Tunnel migration through libvirt's daemon",
then click "Migrate", it will succeed.

6. Right click the guest below the remote connection,select new host with dest ip, click
the "Advance", choose checkbox "Tunnel migration through libvirt's daemon",
then click "Migrate", it will failed with error:"unable to migrate guest":operation failed: Failed to connect to remote libvirt URI(null).

Test failed. so set the status to 'ASSIGN'.

Addition info:
Tunneled migration can work well by using virsh.
If do this test on two virt-manager, it will work well.

Comment 13 Cole Robinson 2011-08-24 16:31:34 UTC
Hmm, don't know if that's a virt-manager issue or not, guessing it's libvirt.

Is this reproducible every time?
Does:
virsh --connect qemu:///system migrate rhel61 --p2p --tunnelled qemu+ssh://root.3.130/system

fail in the same way?

Comment 14 Huming Jiang 2011-08-25 04:32:25 UTC
(In reply to comment #13)
> Hmm, don't know if that's a virt-manager issue or not, guessing it's libvirt.
> 
> Is this reproducible every time?
> Does:
> virsh --connect qemu:///system migrate rhel61 --p2p --tunnelled
> qemu+ssh://root.3.130/system
> 
> fail in the same way?

I redo the test as comment 12, it always has this problem when use virt-manager on the source host to do the tunnelled migration.

And i use the command to do the test, and it has no this problem:

On Source host:
1.# virsh --connect qemu:///system migrate test --p2p --tunnelled qemu+ssh://root.3.130/system    
 ------(OK)
2. # virsh migrate --live --p2p --tunnelled test qemu+ssh://10.66.3.130/system
 ------(OK)
3.# virsh migrate --p2p --tunnelled test qemu+ssh://10.66.3.130/system
 ------(OK)

On target host:
1.# virsh --connect qemu:///system migrate test --p2p --tunnelled qemu+ssh://root.3.131/system    
 ------(OK)
2. # virsh migrate --live --p2p --tunnelled test qemu+ssh://10.66.3.131/system
 ------(OK)
3.# virsh migrate --p2p --tunnelled test qemu+ssh://10.66.3.131/system
 ------(OK)

Comment 15 Cole Robinson 2011-10-13 20:14:16 UTC
Okay, I've pushed an upstream fix for making a better attempt at generating a remotely accessible URI for localhost:

http://git.fedorahosted.org/git?p=virt-manager.git;a=commit;h=bca155230609efb01c0d63062036d413fab3a43c

Comment 16 Cole Robinson 2011-10-13 21:49:24 UTC
Fixed in virt-manager-0.9.0-7.el6

Comment 17 Daisy Wu 2011-10-17 03:41:37 UTC
The bug can reproduce on version:
virt-manager-0.9.0-6.el6

Verified on version:
virt-manager-0.9.0-7.el6

Environment:
kernel-2.6.32-206.el6.x86_64
libvirt-0.9.4-17.el6
python-virtinst-0.600.0-3.el6
qemu-kvm-0.12.1.2-2.195.el6.x86_64

Detail Steps:
1. Prepare 2 hosts and prepare a nfs which is mounted on both hosts, and
setting the virt_use_nfs boolean on both sides # setsebool -P virt_use_nfs 1
and close the iptable on both sides # iptables -F
Edit /etc/hosts file on both hosts, add the following sentence to the file. {IP_addr_of_source_hosts} {HOSTNAME_of_source_hosts} {IP_addr_of_target_hosts} {HOSTNAME_of_target_hosts}

2. Dispatch ssh public key of source host to target host, and also from target
host to source host.
Creating your local public key pair # ssh-keygen -t rsa
Copying the public key to remote host # ssh-copy-id -i ~/.ssh/id_rsa.pub root@{target ip}

The following steps is done on one virt-manager on the source host.
3. Run virt-manager

4. Click File->Add Connection, choose QEMU/KVM on Hypervisor, choose SSH on
Method, and write down the hostname of target hostIP, then click Connect

5. Right click guest below the localhost connect and select migrate, select new
host with dest ip, click
the "Advance", choose checkbox "Tunnel migration through libvirt's daemon",
then click "Migrate", it will succeed.

6. Right click the guest below the remote connection,select new host with localhost(QEMU)
, click the "Advance", choose checkbox "Tunnel migration through libvirt's daemon",then click "Migrate", it will succeed.

Comment 18 Cole Robinson 2011-10-18 13:03:06 UTC
Hi Daisy, thanks for confirming, can you set bug status to VERIFIED?

Comment 19 Min Zhan 2011-10-19 01:53:47 UTC
Move to VERIFIED per Comment 17.

Comment 20 Cole Robinson 2011-11-07 22:26:40 UTC
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Cause:
virt-manager initiated tunneled migration had several defects

Consequence:
tunneled migration failed

Fix:
fixed defects

Result:
tunneled migration works

Comment 21 errata-xmlrpc 2011-12-06 16:09:52 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2011-1642.html