Bug 1374718

Summary: virsh save doesn't work after postcopy migration
Product: Red Hat Enterprise Linux 7 Reporter: Milan Zamazal <mzamazal>
Component: libvirtAssignee: Jiri Denemark <jdenemar>
Status: CLOSED ERRATA QA Contact: zhe peng <zpeng>
Severity: unspecified Docs Contact:
Priority: high    
Version: 7.3CC: dgilbert, dyuan, fjin, jdenemar, jsuchane, lmanasko, michal.skrivanek, mzamazal, rbalakri, sherold, snagar, xuzhang, zpeng
Target Milestone: rcKeywords: ZStream
Target Release: 7.3   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-2.5.0-1.el7 Doc Type: Bug Fix
Doc Text:
Previously, the libvirt library did not disable post-copy on the destination when a migration completed. As a consequence, it was not possible to save such a domain on a disk. With this update, libvirt properly disables post-copy after the migration finishes. As a result, subsequent migration does not use post-copy unless it is requested, and saving the domain to a disk succeeds.
Story Points: ---
Clone Of:
: 1392030 (view as bug list) Environment:
Last Closed: 2017-08-01 17:14:13 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1354343, 1374760, 1392030    
Attachments:
Description Flags
libvirtd.log.xz
none
dummy.log none

Description Milan Zamazal 2016-09-09 13:11:56 UTC
Created attachment 1199461 [details]
libvirtd.log.xz

Description of problem:

When `virsh migrate' is called with --postcopy, `virsh save' doesn't work after the migration.

Version-Release number of selected component (if applicable):

libvirt-2.0.0-8.el7.x86_64
qemu-kvm-rhev-2.6.0-23.el7.x86_64

How reproducible:

100%

Steps to Reproduce:

Create and migrate a VM and the source host using virsh:

# cat /var/tmp/dummy.xml 
<domain type='kvm'>
  <name>dummy</name>
  <uuid>4f836db9-e87f-425b-9278-5693be04a978</uuid>
  <memory unit='KiB'>1048576</memory>
  <os>
    <type arch="x86_64" machine="pc-i440fx-rhel7.2.0">hvm</type>
  </os>
</domain>

# virsh create /var/tmp/dummy.xml
Domain dummy created from /var/tmp/dummy.xml

# virsh migrate dummy qemu+tcp://root@DESTINATION-HOST/system --verbose --live --postcopy
Migration: [100 %]

Then try to save the VM on the destination host:

# virsh list
 Id    Name                           State
----------------------------------------------------
 1     dummy                          running

# virsh save dummy /tmp/xxx
error: Failed to save domain dummy to /tmp/xxx
error: operation failed: domain save job: unexpectedly failed


Actual results:

`virsh save' command fails.


Expected results:

`virsh save' command succeeds, e.g.:

# virsh save dummy /tmp/xxx

Domain dummy saved to /tmp/xxx


Additional info:

libvirtd.log from the destination host is attached.

Saving the VM works when it is migrated without --postcopy switch.

Comment 1 Jiri Denemark 2016-09-09 13:28:33 UTC
Could you please attach /var/log/libvirt/qemu/dummy.log from the destination host?

Comment 2 Milan Zamazal 2016-09-09 13:29:14 UTC
Created attachment 1199464 [details]
dummy.log

Adding /var/log/libvirt/qemu/dummy.log from the destination host, where `virsh save' fails.

Comment 4 zhe peng 2016-09-30 05:22:07 UTC
libvirt qe can reproduce this issue.
build :
libvirt-2.0.0-9.el7.x86_64
qemu-kvm-rhev-2.6.0-26.el7.x86_64

Comment 7 Jiri Denemark 2016-11-02 15:00:10 UTC
Fixed upstream by

commit fe1dd39087ae9a49888bc72dee38ff0e6f639693
Refs: v2.4.0-4-gfe1dd39
Author:     Jiri Denemark <jdenemar>
AuthorDate: Wed Nov 2 14:20:42 2016 +0100
Commit:     Jiri Denemark <jdenemar>
CommitDate: Wed Nov 2 15:50:44 2016 +0100

    qemu: Reset post-copy capability after migration

    Unlike other migration capabilities, post-copy is also set on the
    destination host which means it doesn't disappear once domain is
    migrated. As a result of that other functionality which internally uses
    migration to a file (virDomainManagedSave, virDomainSave,
    virDomainCoreDump) may fail after migration because the post-copy
    capability is still set.

    https://bugzilla.redhat.com/show_bug.cgi?id=1374718

    Signed-off-by: Jiri Denemark <jdenemar>

Comment 11 zhe peng 2017-03-01 08:06:14 UTC
Verify with build:
libvirt-3.0.0-2.el7.x86_64
qemu-kvm-rhev-2.8.0-5.el7.x86_64

step:
scenario A:
1 start a guest on source 
2 on source host do
# virsh migrate rhel7 qemu+ssh://$target/system --postcopy --live --verbose
Migration: [100 %]
3 on target host
# virsh save rhel7 /tmp/rhel7

Domain rhel7 saved to /tmp/rhel7

scenario B:
1.on source host do
 #virsh migrate rhel7 qemu+ssh://$target/system --postcopy --postcopy-after-precopy --live --verbose
Migration: [100 %]
2 on target host
# virsh save rhel7 /tmp/rhel7

Domain rhel7 saved to /tmp/rhel7

# virsh restore /tmp/rhel7
Domain restored from /tmp/rhel7

# virsh list --all
 Id    Name                           State
----------------------------------------------------
 5     rhel7                          running

scenario C:
1. on source host do 
# virsh migrate rhel7 qemu+ssh://$target/system --postcopy --live --verbose
open another console, run
# virsh migrate-postcopy rhel7
2. on target host
# virsh save rhel7 /tmp/rhel7

Domain rhel7 saved to /tmp/rhel7

move to verified.

Comment 12 errata-xmlrpc 2017-08-01 17:14:13 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846

Comment 13 errata-xmlrpc 2017-08-01 23:55:08 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846

Comment 14 errata-xmlrpc 2017-08-02 01:27:35 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846