Bug 1018267

Summary: Libvirt crashes during migration
Product: Red Hat Enterprise Linux 7 Reporter: Michal Privoznik <mprivozn>
Component: libvirtAssignee: Michal Privoznik <mprivozn>
Status: CLOSED CURRENTRELEASE QA Contact: Virtualization Bugs <virt-bugs>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.0CC: acathrow, dyuan, jtomko, mjenner, ydu, zpeng
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-1.1.1-13.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-06-13 12:44:59 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1015215    

Description Michal Privoznik 2013-10-11 15:10:26 UTC
Description of problem:
During migration, if qemu on the destination dies quickly, libvirtd crashes too. That is, if qemu happens to reply on the monitor with QMP greeting, but dies immediately after, libvirtd crashes.


Version-Release number of selected component (if applicable):
libvirt-1.1.1-8.el7

How reproducible:
100% (with patched qemu)

Steps to Reproduce:
1. Apply this patch to qemu to make it die at the correct place:

diff --git a/monitor.c b/monitor.c
index b096c7f..9e0777f 100644
--- a/monitor.c
+++ b/monitor.c
@@ -676,6 +676,7 @@ static int do_qmp_capabilities(Monitor *mon, const QDict *params,
         mon->mc->command_mode = 1;
     }
 
+    abort();
     return 0;
 }

2. Install the qemu on destination
3. Start a migration onto destination
4. Observe libvirtd segfault

Actual results:
libvirtd segfaults

Expected results:
libvirtd doesn't segfault

Additional info:

Comment 3 zhe peng 2013-10-16 12:15:40 UTC
I can reproduce this with build:
libvirt-1.1.1-8.el7

verify with :
libvirt-1.1.1-9.el7

step:
1: build a qemu-kvm pkg with patch on destination
2: Start a migration onto destination
# virsh migrate rhel6 --live qemu+ssh://10.66.106.25/system --verbose --unsafe
root.106.25's password: 
error: Unable to read from monitor: Connection reset by peer

migration failed.

3: check libvirtd both on source and destination,all active,no segfault occured.

verification passed. move to verified.

Comment 4 Michal Privoznik 2013-11-08 05:56:13 UTC
I've just hit the problem myself again. I'm reopening this as we need a small patch. Moreover, see bug 1015215 transcription for more details.

Comment 8 yanbing du 2013-11-15 07:34:52 UTC
Move back to ASSIGNED as the crash problem.

Comment 9 Michal Privoznik 2013-11-15 10:33:44 UTC
Patch proposed upstream:

https://www.redhat.com/archives/libvir-list/2013-November/msg00557.html

Comment 11 Ján Tomko 2013-11-22 15:30:02 UTC
I was getting crashes even with the fix from comment 9
Fixed upstream by v1.1.4-140-g9846402 :
commit 98464021167377bc7373b4d3951b1e868f8db291
Author:     Ján Tomko <jtomko>
CommitDate: 2013-11-22 16:22:31 +0100

    Don't start a nested job in qemuMigrationPrepareAny
    
Patch posted downstream:
http://post-office.corp.redhat.com/archives/rhvirt-patches/2013-November/msg01053.html

Comment 12 zhe peng 2013-11-26 06:31:37 UTC
can reproduce crash issue with libvirt-1.1.1-12.el7

verify with build:libvirt-1.1.1-13.el7

step:
repeat 10 times:
 # virsh migrate --live rhel6 qemu+ssh://10.66.100.118/system --verbose --unsafe --listen-address 01.3.4.4
root.100.118's password: 
error: internal error: early end of file from monitor: possible problem:
qemu-kvm: -incoming tcp:01.3.4.4:49152: Failed to bind socket: Cannot assign requested address

check dst. libvirtd, no crash occured. move to verified.

Comment 13 Ludek Smid 2014-06-13 12:44:59 UTC
This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.