Bug 506641 - virsh migrate hangs for KVM guests
Summary: virsh migrate hangs for KVM guests
Keywords:
Status: CLOSED WORKSFORME
Alias: None
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: libvirt
Version: 5.4
Hardware: All
OS: Linux
low
medium
Target Milestone: rc
: ---
Assignee: Daniel Berrangé
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2009-06-18 03:43 UTC by Nan Zhang
Modified: 2009-12-14 21:17 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2009-10-29 16:25:49 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
migration hanging debug info (10.64 KB, text/plain)
2009-06-18 03:43 UTC, Nan Zhang
no flags Details
log of client migration (8.46 KB, text/plain)
2009-06-26 07:07 UTC, Nan Zhang
no flags Details
target host libvirtd with debug info (56.05 KB, text/plain)
2009-06-26 07:08 UTC, Nan Zhang
no flags Details

Description Nan Zhang 2009-06-18 03:43:53 UTC
Created attachment 348372 [details]
migration hanging debug info

Description of problem:
Configure libvirtd with SASL authentication, it will be hung during migration connecting. See attached detailed debug info.

Version-Release number of selected component (if applicable):
libvirt 0.6.3-10.el5
rhel-5.4 (2.6.128-153.el5)

How reproducible:
Always

Steps to Reproduce:
Server:
# vi /etc/libvirt/libvirtd.conf
listen_tcp = 1
auth_tcp = "sasl"

# vi /etc/sysconfig/libvirtd
LIBVIRTD_ARGS="--listen"

# service libvirtd restart
Stopping libvirtd 

# saslpasswd2 -a libvirt root
Password:
Again (for verification):
# sasldblistusers2 -f /etc/libvirt/passwd.db
fred.redhat.com: userPassword
root.redhat.com: userPassword

Client:
# virsh migrate --live kvmtest qemu+tcp://10.66.70.85/system
Please enter your authentication name:root
Please enter your password:  
  
Actual results:
migration failed through SASL

Expected results:
migration successful with SASL authentication

Additional info:

Comment 1 Daniel Berrangé 2009-06-25 13:44:59 UTC
Please try the same operationg, using an SSH connection instead of SASL

# virsh migrate --live kvmtest qemu+ssh://root.70.85/system


If that also hangs, please provide logs of the client and target host libvirtd both with LIBVIRT_DEBUG=1 set.

Comment 2 Nan Zhang 2009-06-26 07:02:29 UTC
Yes, using an SSH connection also hangs, my current testing environment as following.

[root@dhcp-66-70-85 ~]# rpm -qa|grep 'kvm\|qemu\|kernel'|sort
etherboot-roms-kvm-5.4.4-10.el5
etherboot-zroms-kvm-5.4.4-10.el5
kernel-2.6.18-155.el5
kernel-xen-2.6.18-155.el5
kmod-kvm-83-80.el5
kvm-83-80.el5
kvm-qemu-img-83-80.el5

Comment 3 Nan Zhang 2009-06-26 07:07:35 UTC
Created attachment 349511 [details]
log of client migration

Comment 4 Nan Zhang 2009-06-26 07:08:29 UTC
Created attachment 349512 [details]
target host libvirtd with debug info

Comment 5 Daniel Berrangé 2009-06-26 11:17:12 UTC
At the time that it is hanging, can you see if any VM is running on the destination, eg

 virsh list --all


and grab the log file /var/log/libvirt/qemu/kvmtest.log from both the sopurce and destination host after it hangs.

It looks like the destation VM started up, reported success to libvirt, but then quickly shutdown again.

Comment 6 Nan Zhang 2009-06-29 13:07:34 UTC
Now, this issue cannot be reproduced every time, need to test further.

Comment 7 Gerrit Slomma 2009-07-24 21:32:46 UTC
I have a different error here:

[root@rr020 ~]# virsh migrate --live rr019v4 qemu+tcp://192.168.1.17/system
Please enter your authentication name:root
Please enter your password:
Fehler: Unknown failure

[root@rr017 ~]# tail -f /var/log/libvirt/qemu/rr019v4.log
LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin HOME=/root USER=root LOGNAME=root /usr/libexec/qemu-kvm -S -M pc -m 1536 -smp 2 -name rr019v4 -uuid ff1ba599-5801-fca9-ab33-8962c2dfa46d -monitor pty -pidfile /var/run/libvirt/qemu//rr019v4.pid -boot c -drive file=/dev/disk/by-path/ip-192.168.1.1:3260-iscsi-rr010:01-lun-7,if=ide,index=0,boot=on -net nic,macaddr=00:16:3e:69:94:06,vlan=0,model=e1000 -net tap,fd=17,script=,vlan=0,ifname=vnet0 -serial none -parallel none -usb -vnc 0.0.0.0:0 -k de -incoming tcp:0.0.0.0:49152
char device redirected to /dev/pts/1

[root@rr020 ~]# virsh migrate --live rr019v4 qemu+tcp://192.168.1.17/system tcp:192.168.1.17:4444
Please enter your authentication name:root
Please enter your password:
[root@rr020 ~]#

[root@rr017 ~]# tail -f /var/log/libvirt/qemu/rr019v4.log
LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin HOME=/root USER=root LOGNAME=root /usr/libexec/qemu-kvm -S -M pc -m 1536 -smp 2 -name rr019v4 -uuid ff1ba599-5801-fca9-ab33-8962c2dfa46d -monitor pty -pidfile /var/run/libvirt/qemu//rr019v4.pid -boot c -drive file=/dev/disk/by-path/ip-192.168.1.1:3260-iscsi-rr010:01-lun-7,if=ide,index=0,boot=on -net nic,macaddr=00:16:3e:69:94:06,vlan=0,model=e1000 -net tap,fd=17,script=,vlan=0,ifname=vnet0 -serial none -parallel none -usb -vnc 0.0.0.0:0 -k de -incoming tcp:0.0.0.0:4444
char device redirected to /dev/pts/1

I verified that this is no iptables-issue with disabling iptables.

on both hosts

Installed Packages
etherboot-zroms-kvm.x86_64   5.4.4-10.el5          installed
kernel.x86_64                2.6.18-155.el5        installed
kmod-kvm.x86_64              83-80.el5             installed
kvm.x86_64                   83-80.el5             installed
libvirt.x86_64               0.6.3-11.el5          installed

Comment 8 Hugh Brock 2009-10-29 16:25:49 UTC
Closing this since we were unable to reproduce it in testing (comment #7 is not relevant to this bug)


Note You need to log in before you can comment on or make changes to this bug.