Bug 678548

Summary: Replace exec: migration with fd: migration for domain save
Product: Red Hat Enterprise Linux 6 Reporter: Daniel Berrangé <berrange>
Component: libvirtAssignee: Eric Blake <eblake>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 6.1CC: dallan, dyuan, eblake, gren, juzhang, mkenneth, mzhan, pbonzini, rwu, virt-maint, weizhan, xen-maint, yoyzhang
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-0.9.2-1.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-12-06 10:54:36 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 620363, 672725, 681623, 691034, 691499    
Bug Blocks:    

Description Daniel Berrangé 2011-02-18 11:42:25 UTC
Description of problem:
We recently switched incoming domain restore from exec: to fd: migration. Exec based migration has proved no end of trouble, so we should do the same thing with domain save, though it requires a little magic

Basically we can use 'getfd foo' monitor command to send an open file descriptor to QEMU, and then  'migrate  fd:foo'  to trigger migration using that passed handle. This will avoid the need for QEMU to exec stuff.


Version-Release number of selected component (if applicable):
libvirt 0.8.7

How reproducible:
N/A

Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:

Comment 2 Eric Blake 2011-02-18 15:04:27 UTC
This is a followup to bug 620363

Comment 3 Eric Blake 2011-03-02 03:58:41 UTC
This code is somewhat intertwined with the fixes for bug 672725.  Additionally, since bug 678524 proves that qemu has a race condition with exec: processing, fixing libvirt will be inevitible if qemu rips exec: out, and good practice even if qemu leaves exec: support in.

Comment 4 Dave Allan 2011-03-02 04:11:06 UTC
I have removed the FutureFeature keyword and RFE from the BZ summary as this work is needed to address races in the migration code.

Comment 5 Eric Blake 2011-03-10 15:03:39 UTC
Requesting blocker for 6.1 if qemu bug 678524 is not fixed; if the qemu exec: bug is fixed then the libvirt changes of this bug can be deferred to 6.2.

Comment 6 Eric Blake 2011-03-15 15:32:20 UTC
Looks like bug 678524 will be fixed, and it's much less invasive.  Recommend deferring the libvirt changes to 6.2.

Comment 7 Eric Blake 2011-03-28 19:45:34 UTC
Upstream now has fd: support in 0.9.0 (modulo any fallout regression fixes that might still be necessary), as of the series ending in commit:

commit 15d757ac4e5d4be6972f05f33aa3fa85fbedd0bd
Author: Eric Blake <eblake>
Date:   Fri Mar 25 11:02:27 2011 -0600

    qemu: support fd: migration with compression
    
    Spawn the compressor ourselves, instead of requiring the shell.
    
    * src/qemu/qemu_migration.c (qemuMigrationToFile): Spawn
    compression helper process when needed.

This should automatically be picked up by rebasing in RHEL 6.2.

Comment 10 Daniel Veillard 2011-06-23 02:44:05 UTC
This should be fixed by the libvirt-0.9.2-1.el6 rebase

Comment 11 Min Zhan 2011-06-28 11:47:18 UTC
Verified this bug as PASS. Details please refer to the following:

Environment:
# uname -r
2.6.32-156.el6.x86_64
qemu-kvm-0.12.1.2-2.165.el6.x86_64
libvirt-0.9.2-1.el6.x86_64

Steps:
1. # virsh list --all
 Id Name                 State
----------------------------------
 15 rhel56               running

# virsh save rhel56 rhel56.save
Domain rhel56 saved to rhel56.save

2. From the log file 'getfd' and 'fd:migrate' can be found for domain save

# cat /etc/libvirt/libvirtd.conf
log_level = 1
log_outputs="1:file:/tmp/new"

# cat /tmp/new
...
19:49:13.337: 17266: debug : qemuMonitorMigrateToFd:1523 : mon=0xecf270 fd=22 flags=1
19:49:13.337: 17266: debug : qemuMonitorSendFileHandle:1920 : mon=0xecf270, fdname=migrate fd=22
...
19:49:13.337: 17266: debug : virJSONValueToString:1101 : result={"execute":"getfd","arguments":{"fdname":"migrate"},"id":"libvirt-5"}
19:49:13.337: 17266: debug : qemuMonitorJSONCommandWithFd:226 : Send command '{"execute":"getfd","arguments":{"fdname":"migrate"},"id":"libvirt-5"}' for write with FD 22
...
19:49:13.338: 17266: debug : virJSONValueToString:1101 : result={"execute":"migrate","arguments":{"detach":true,"blk":false,"inc":false,"uri":"fd:migrate"},"id":"libvirt-6"}
19:49:13.338: 17266: debug : qemuMonitorJSONCommandWithFd:226 : Send command '{"execute":"migrate","arguments":{"detach":true,"blk":false,"inc":false,"uri":"fd:migrate"},"id":"libvirt-6"}' for write with FD -1


-----
I can reproduce this bug with libvirt-0.8.7-18.el6.x86_64. It use exec: for domain save.

# virsh save rhel56 rhel56.save
error: Failed to save domain rhel56 to rhel56.save
error: operation failed: domain save job: unexpectedly failed

# cat /etc/libvirt/libvirtd.conf
log_level = 1
log_outputs="1:file:/tmp/old"

# cat /tmp/old
...
19:36:14.966: 14428: debug : qemuMonitorMigrateToFile:1412 : mon=0x7f79a0011af0 argv=0x7f79b75fc910 target=/root/rhel56.save offset=4096 flags=1
...
19:36:14.966: 14428: debug : virJSONValueToString:1062 : result={"execute":"migrate","arguments":{"detach":true,"blk":false,"inc":false,"uri":"exec:cat | { dd bs=4096 seek=1 if=/dev/null && dd bs=1048576; } 1<>'/root/rhel56.save'"}}
19:36:14.966: 14428: debug : qemuMonitorJSONCommandWithFd:218 : Send command '{"execute":"migrate","arguments":{"detach":true,"blk":false,"inc":false,"uri":"exec:cat | { dd bs=4096 seek=1 if=/dev/null && dd bs=1048576; } 1<>'/root/rhel56.save'"}}' for write with FD -1

Comment 12 errata-xmlrpc 2011-12-06 10:54:36 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2011-1513.html