Bug 506641 - virsh migrate hangs for KVM guests
virsh migrate hangs for KVM guests
Status: CLOSED WORKSFORME
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: libvirt (Show other bugs)
5.4
All Linux
low Severity medium
: rc
: ---
Assigned To: Daniel Berrange
Virtualization Bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2009-06-17 23:43 EDT by Nan Zhang
Modified: 2009-12-14 16:17 EST (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2009-10-29 12:25:49 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
migration hanging debug info (10.64 KB, text/plain)
2009-06-17 23:43 EDT, Nan Zhang
no flags Details
log of client migration (8.46 KB, text/plain)
2009-06-26 03:07 EDT, Nan Zhang
no flags Details
target host libvirtd with debug info (56.05 KB, text/plain)
2009-06-26 03:08 EDT, Nan Zhang
no flags Details

  None (edit)
Description Nan Zhang 2009-06-17 23:43:53 EDT
Created attachment 348372 [details]
migration hanging debug info

Description of problem:
Configure libvirtd with SASL authentication, it will be hung during migration connecting. See attached detailed debug info.

Version-Release number of selected component (if applicable):
libvirt 0.6.3-10.el5
rhel-5.4 (2.6.128-153.el5)

How reproducible:
Always

Steps to Reproduce:
Server:
# vi /etc/libvirt/libvirtd.conf
listen_tcp = 1
auth_tcp = "sasl"

# vi /etc/sysconfig/libvirtd
LIBVIRTD_ARGS="--listen"

# service libvirtd restart
Stopping libvirtd 

# saslpasswd2 -a libvirt root
Password:
Again (for verification):
# sasldblistusers2 -f /etc/libvirt/passwd.db
fred@dhcp-66-70-85.nay.redhat.com: userPassword
root@dhcp-66-70-85.nay.redhat.com: userPassword

Client:
# virsh migrate --live kvmtest qemu+tcp://10.66.70.85/system
Please enter your authentication name:root
Please enter your password:  
  
Actual results:
migration failed through SASL

Expected results:
migration successful with SASL authentication

Additional info:
Comment 1 Daniel Berrange 2009-06-25 09:44:59 EDT
Please try the same operationg, using an SSH connection instead of SASL

# virsh migrate --live kvmtest qemu+ssh://root@10.66.70.85/system


If that also hangs, please provide logs of the client and target host libvirtd both with LIBVIRT_DEBUG=1 set.
Comment 2 Nan Zhang 2009-06-26 03:02:29 EDT
Yes, using an SSH connection also hangs, my current testing environment as following.

[root@dhcp-66-70-85 ~]# rpm -qa|grep 'kvm\|qemu\|kernel'|sort
etherboot-roms-kvm-5.4.4-10.el5
etherboot-zroms-kvm-5.4.4-10.el5
kernel-2.6.18-155.el5
kernel-xen-2.6.18-155.el5
kmod-kvm-83-80.el5
kvm-83-80.el5
kvm-qemu-img-83-80.el5
Comment 3 Nan Zhang 2009-06-26 03:07:35 EDT
Created attachment 349511 [details]
log of client migration
Comment 4 Nan Zhang 2009-06-26 03:08:29 EDT
Created attachment 349512 [details]
target host libvirtd with debug info
Comment 5 Daniel Berrange 2009-06-26 07:17:12 EDT
At the time that it is hanging, can you see if any VM is running on the destination, eg

 virsh list --all


and grab the log file /var/log/libvirt/qemu/kvmtest.log from both the sopurce and destination host after it hangs.

It looks like the destation VM started up, reported success to libvirt, but then quickly shutdown again.
Comment 6 Nan Zhang 2009-06-29 09:07:34 EDT
Now, this issue cannot be reproduced every time, need to test further.
Comment 7 Gerrit Slomma 2009-07-24 17:32:46 EDT
I have a different error here:

[root@rr020 ~]# virsh migrate --live rr019v4 qemu+tcp://192.168.1.17/system
Please enter your authentication name:root
Please enter your password:
Fehler: Unknown failure

[root@rr017 ~]# tail -f /var/log/libvirt/qemu/rr019v4.log
LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin HOME=/root USER=root LOGNAME=root /usr/libexec/qemu-kvm -S -M pc -m 1536 -smp 2 -name rr019v4 -uuid ff1ba599-5801-fca9-ab33-8962c2dfa46d -monitor pty -pidfile /var/run/libvirt/qemu//rr019v4.pid -boot c -drive file=/dev/disk/by-path/ip-192.168.1.1:3260-iscsi-rr010:01-lun-7,if=ide,index=0,boot=on -net nic,macaddr=00:16:3e:69:94:06,vlan=0,model=e1000 -net tap,fd=17,script=,vlan=0,ifname=vnet0 -serial none -parallel none -usb -vnc 0.0.0.0:0 -k de -incoming tcp:0.0.0.0:49152
char device redirected to /dev/pts/1

[root@rr020 ~]# virsh migrate --live rr019v4 qemu+tcp://192.168.1.17/system tcp:192.168.1.17:4444
Please enter your authentication name:root
Please enter your password:
[root@rr020 ~]#

[root@rr017 ~]# tail -f /var/log/libvirt/qemu/rr019v4.log
LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin HOME=/root USER=root LOGNAME=root /usr/libexec/qemu-kvm -S -M pc -m 1536 -smp 2 -name rr019v4 -uuid ff1ba599-5801-fca9-ab33-8962c2dfa46d -monitor pty -pidfile /var/run/libvirt/qemu//rr019v4.pid -boot c -drive file=/dev/disk/by-path/ip-192.168.1.1:3260-iscsi-rr010:01-lun-7,if=ide,index=0,boot=on -net nic,macaddr=00:16:3e:69:94:06,vlan=0,model=e1000 -net tap,fd=17,script=,vlan=0,ifname=vnet0 -serial none -parallel none -usb -vnc 0.0.0.0:0 -k de -incoming tcp:0.0.0.0:4444
char device redirected to /dev/pts/1

I verified that this is no iptables-issue with disabling iptables.

on both hosts

Installed Packages
etherboot-zroms-kvm.x86_64   5.4.4-10.el5          installed
kernel.x86_64                2.6.18-155.el5        installed
kmod-kvm.x86_64              83-80.el5             installed
kvm.x86_64                   83-80.el5             installed
libvirt.x86_64               0.6.3-11.el5          installed
Comment 8 Hugh Brock 2009-10-29 12:25:49 EDT
Closing this since we were unable to reproduce it in testing (comment #7 is not relevant to this bug)

Note You need to log in before you can comment on or make changes to this bug.