Bug 1584071

Summary: Unknown error occurred on target host when try to get completed domjobinfo
Product: Red Hat Enterprise Linux 7 Reporter: Yanqiu Zhang <yanqzhan>
Component: libvirtAssignee: Jiri Denemark <jdenemar>
Status: CLOSED ERRATA QA Contact: Yanqiu Zhang <yanqzhan>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.6CC: dyuan, dzheng, fjin, lmen, mzhan, xuzhang, yafu, yanqzhan
Target Milestone: rcKeywords: Automation, Regression
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-4.4.0-1.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-10-30 09:55:54 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
libvirtd_qemu_logs none

Description Yanqiu Zhang 2018-05-30 08:11:46 UTC 
Description of problem:
Unknown error occurred on target host when try to get completed domjobinfo

Version-Release number of selected component (if applicable):
libvirt-4.3.0-1.el7.x86_64
qemu-kvm-rhev-2.12.0-2.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Prepare a vm and migration env

2.start the vm
# virsh start V
Domain V started

3.Do migration from source host
# virsh migrate V --live qemu+ssh://$target_host/system --verbose --unsafe
Migration: [100 %] 

4.When migration finished, check completed domjobinfo on target:
# virsh domjobinfo V --completed
error: An error occurred, but the cause is unknown

Actual results:
 In step4, an unknown error occurred on target host when try to get completed domjobinfo.

Expected results:
 Should be able to get the completed domjobinfo without error on target host

Additional info:
 Not reproduced on source host.
Comment 3 Yanqiu Zhang 2018-05-30 08:28:00 UTC 
Created attachment 1445721 [details]
libvirtd_qemu_logs
Comment 4 Jiri Denemark 2018-06-01 08:46:40 UTC 
Patches sent upstream for review: https://www.redhat.com/archives/libvir-list/2018-June/msg00019.html
Comment 5 Jiri Denemark 2018-06-01 15:59:20 UTC 
Fixed upstream by

commit e36b1f6583324133405c7f4552a9da51e6c61161
Refs: v4.4.0-rc2-2-ge36b1f6583
Author:     Jiri Denemark <jdenemar>
AuthorDate: Fri Jun 1 10:32:49 2018 +0200
Commit:     Jiri Denemark <jdenemar>
CommitDate: Fri Jun 1 17:55:44 2018 +0200

    qemu: Fix reporting completed migration stats on destination

    This has been broken since commit v4.0.0-165-g93412bb827 which added
    jobInfo->statsType enum to distinguish various statistics types. During
    migration the type will always be QEMU_DOMAIN_JOB_STATS_TYPE_MIGRATION,
    however the destination code consuming the statistics data from
    migration cookie failed to properly set the type. So even though
    everything was filled in, the type remained *_NONE and any attempt to
    fetch the statistics data of a completed migration on the destination
    host failed.

    https://bugzilla.redhat.com/show_bug.cgi?id=1584071

    Signed-off-by: Jiri Denemark <jdenemar>
Comment 7 Yanqiu Zhang 2018-06-21 07:09:18 UTC 
Verify this bug with:
libvirt-4.4.0-2.el7.x86_64
qemu-kvm-rhev-2.12.0-4.el7.x86_64

Steps to verify:
1. Prepare env and do migration:
# virsh migrate yan-V --live qemu+ssh://$target_host/system --verbose
Migration: [100 %]

[$source_host]# while true; do virsh domjobinfo yan-V;sleep 1; done
...
Job type:         None        

Job type:         Unbounded   
Operation:        Outgoing migration
Time elapsed:     326          ms
Data processed:   3.128 MiB
Data remaining:   1.122 GiB
Data total:       1.126 GiB
Memory processed: 3.128 MiB
Memory remaining: 1.122 GiB
Memory total:     1.126 GiB
Dirty rate:       0            pages/s
Page size:        4096         bytes
Iteration:        1           
Constant pages:   124         
Normal pages:     799         
Normal data:      3.121 MiB
Expected downtime: 300          ms
Setup time:       24           ms
...

[$target_host]# while true; do virsh domjobinfo yan-V;sleep 1; done
...
error: Operation not supported: migration statistics are available only on the source host
...

2. Check completed domjobifo:
[$source_host]# virsh domjobinfo yan-V --completed
Job type:         Completed   
Operation:        Outgoing migration
Time elapsed:     2892         ms
Time elapsed w/o network: 2891         ms
Data processed:   275.222 MiB
Data remaining:   0.000 B
Data total:       1.126 GiB
Memory processed: 275.222 MiB
Memory remaining: 0.000 B
Memory total:     1.126 GiB
Memory bandwidth: 106.944 MiB/s
Dirty rate:       0            pages/s
Page size:        4096         bytes
Iteration:        3           
Constant pages:   227292      
Normal pages:     69821       
Normal data:      272.738 MiB
Total downtime:   80           ms
Downtime w/o network: 79           ms
Setup time:       24           ms

[$target_host]# virsh domjobinfo yan-V --completed
Job type:         Completed   
Operation:        Incoming migration
Time elapsed:     2882         ms
Time elapsed w/o network: 2881         ms
Data processed:   275.222 MiB
Data remaining:   0.000 B
Data total:       1.126 GiB
Memory processed: 275.222 MiB
Memory remaining: 0.000 B
Memory total:     1.126 GiB
Memory bandwidth: 106.944 MiB/s
Dirty rate:       0            pages/s
Page size:        4096         bytes
Iteration:        3           
Constant pages:   227292      
Normal pages:     69821       
Normal data:      272.738 MiB
Total downtime:   80           ms
Downtime w/o network: 79           ms
Setup time:       24           ms

Since the results are as expected. Mark this bug as verified
Comment 9 errata-xmlrpc 2018-10-30 09:55:54 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:3113