Bug 1371358

Summary: Failed with unexpected error when doing post-copy migration with --tunnelled
Product: Red Hat Enterprise Linux 7 Reporter: yafu <yafu>
Component: libvirtAssignee: Jiri Denemark <jdenemar>
Status: CLOSED ERRATA QA Contact: zhe peng <zpeng>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.3CC: dyuan, fjin, mzhan, rbalakri, xuzhang, yanqzhan, zpeng
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-2.5.0-1.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-08-01 17:14:13 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description yafu 2016-08-30 02:56:47 UTC
Description of problem:
Failed with unexpected error when doing post-copy migration with --tunnelled.

Version-Release number of selected component (if applicable):
libvirt-2.0.0-6.el7.x86_64
qemu-kvm-rhev-2.6.0-22.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1.Do post-copy migration with --tunnelled:
# virsh migrate rhel7.3  qemu+ssh://10.66.144.74/system --live --p2p --tunnelled --postcopy
error: internal error: qemu unexpectedly closed the monitor: warning: host doesn't support requested feature: CPUID.01H:EDX.ds [bit 21]
warning: host doesn't support requested feature: CPUID.01H:EDX.acpi [bit 22]
warning: host doesn't support requested feature: CPUID.01H:EDX.ht [bit 28]
warning: host doesn't support requested feature: CPUID.01H:EDX.tm [bit 29]
warning: host doesn't support requested feature: CPUID.01H:EDX.pbe [bit 31]
warning: host doesn't support requested feature: CPUID.01H:ECX.dtes64 [bit 2]
warning: host doesn't support requested feature: CPUID.01H:ECX.monitor [bit 3]
warning: host doesn't support requested feature: CPUID.01H:ECX.ds_cpl [bit 4]
warning: host doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
warning: host doesn't support requested feature: CPUID.01H:ECX.est [bit 7]
warning: host doesn't support requested feature: CPUID.01H:ECX.tm2 [bit 8]
warning: host doesn't support requested feature: CPUID.01H:ECX.xtpr [bit 14]
warning: host doesn't support requested feature: CPUID.01H:ECX.pdcm [

2.Check the qemu log on the target host:
#cat /var/log/libvirt/qemu/rhel7.3.log
...
2016-08-25T03:00:33.864959Z qemu-kvm: Not a migration stream
2016-08-25T03:00:33.865025Z qemu-kvm: load of migration failed: Invalid argument
2016-08-25 03:00:34.072+0000: shutting down
...


Actual results:
Migration failed with unclear error.

Expected results:
Should report clear error, such as:
this feature or command is not currently supported

Comment 2 zhe peng 2016-09-21 06:13:55 UTC
I try this in rdma env. also get this error even if not use --tunnelled
# virsh migrate --live rhel7.3  qemu+ssh://192.10.10.1/system --migrateuri rdma://192.10.10.1 --verbose --postcopy --postcopy-after-precopy
error: internal error: qemu unexpectedly closed the monitor: warning: host doesn't support requested feature: CPUID.01H:EDX.ds [bit 21]
warning: host doesn't support requested feature: CPUID.01H:EDX.acpi [bit 22]
warning: host doesn't support requested feature: CPUID.01H:EDX.ht [bit 28]
warning: host doesn't support requested feature: CPUID.01H:EDX.tm [bit 29]
warning: host doesn't support requested feature: CPUID.01H:EDX.pbe [bit 31]
warning: host doesn't support requested feature: CPUID.01H:ECX.dtes64 [bit 2]
warning: host doesn't support requested feature: CPUID.01H:ECX.monitor [bit 3]
warning: host doesn't support requested feature: CPUID.01H:ECX.ds_cpl [bit 4]
warning: host doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
warning: host doesn't support requested feature: CPUID.01H:ECX.smx [bit 6]
warning: host doesn't support requested feature: CPUID.01H:ECX.est [bit 7]
warning: host doesn't support requested feature: CPUID.01H:ECX.tm2 [bit 8]
warning: host doesn't support requested feature: CPUID.01H:ECX.xtpr [bi

Comment 3 zhe peng 2016-09-21 08:43:37 UTC
check qemu log :

warning: host doesn't support requested feature: CPUID.01H:ECX.osxsave [bit 27]
dest_init RDMA Device opened: kernel name mlx4_0 uverbs device name uverbs0, infiniband_verbs class device path /sys/class/infiniband_verbs/uverbs0, infiniband class device path /sys/class/infiniband/mlx4_0, transport: (1) Infiniband
ibv_poll_cq wc.status=12 transport retry counter exceeded!
ibv_poll_cq wrid=CONTROL SEND!
2016-09-21T08:40:22.926841Z qemu-kvm: rdma migration: send polling control error
2016-09-21T08:40:22.926860Z qemu-kvm: Failed to send control buffer!
2016-09-21T08:40:22.926871Z qemu-kvm: RDMA is in an error state waiting migration to abort!
2016-09-21T08:40:22.926879Z qemu-kvm: Not a migration stream
2016-09-21T08:40:22.926888Z qemu-kvm: Early error. Sending error.
ibv_poll_cq wc.status=5 Work Request Flushed Error!
ibv_poll_cq wrid=CONTROL SEND!
2016-09-21T08:40:22.926932Z qemu-kvm: rdma migration: send polling control error
2016-09-21T08:40:22.927846Z qemu-kvm: load of migration failed: Invalid argument
2016-09-21 08:40:22.965+0000: shutting down

if not use post-copy, rdma migration can worked well.

Comment 4 Jiri Denemark 2016-09-21 10:33:29 UTC
Well, post-copy migration over RDMA is certainly completely out of scope of this bug. Anyway, it is not supposed to work (confirmed with Dave Gilbert).

Comment 5 Jiri Denemark 2016-11-29 11:39:49 UTC
This bug is now fixed upstream by

commit ee3ea86b37146ad80d5418b231959491ff28ebae
Refs: v2.5.0-rc1-1-gee3ea86b3
Author:     Jiri Denemark <jdenemar>
AuthorDate: Tue Nov 22 20:23:56 2016 +0100
Commit:     Jiri Denemark <jdenemar>
CommitDate: Tue Nov 29 12:31:25 2016 +0100

    qemu: Report tunnelled post-copy migration as unsupported

    Post-copy migration needs bi-directional communication between the
    source and the destination QEMU processes, which is not supported by
    tunnelled migration.

    https://bugzilla.redhat.com/show_bug.cgi?id=1371358

    Signed-off-by: Jiri Denemark <jdenemar>

Comment 7 zhe peng 2017-03-01 08:32:31 UTC
verify with build:
libvirt-3.0.0-2.el7.x86_64
qemu-kvm-rhev-2.8.0-5.el7.x86_64

step:
1 prepare migration env.
2 Do post-copy migration with --tunnelled:
# virsh migrate rhel7 qemu+ssh://$target/system --postcopy --p2p --tunnelled --live --verbose
error: argument unsupported: post-copy is not supported with tunnelled migration

no output in /var/log/libvirt/qemu/$guest.log

move to verified.

Comment 8 errata-xmlrpc 2017-08-01 17:14:13 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846

Comment 9 errata-xmlrpc 2017-08-01 23:55:08 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846

Comment 10 errata-xmlrpc 2017-08-02 01:27:35 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846