Red Hat Bugzilla – Bug 1278727
"virsh domjobinfo" hangs on destination host during migration.
Last modified: 2016-11-03 14:30:27 EDT
Created attachment 1090546 [details] libvirtd log on target host Description of problem: "virsh domjobinfo" hangs on destination host during migration. Version-Release number of selected component (if applicable): libvirt-1.2.17-13.el7.x86_64 qemu-kvm-rhev-2.3.0-31.el7.x86_64 How reproducible: 100% Steps to Reproduce: 1.Prepare a running guest on source host. # virsh list Id Name State ---------------------------------------------------- 35 ceph-admin running 2.Set the migration speed to a small value: 1 # virsh migrate-setspeed ceph-admin 1 3.Migrate the guest to another host: # virsh migrate ceph-admin qemu+ssh://10.66.85.92/system --live --verbose root@10.66.85.92's password: Migration:[1%] Before migration completed, get the domjobinfo on target host, the command hangs for a while, then returns with error: # virsh domjobinfo ceph-admin error: Timed out during operation: cannot acquire state change lock (held by remoteDispatchDomainMigratePrepare3Params) After migration completed, get the domjobinfo on target host again: # virsh domjobinfo ceph-admin --completed Job type: Completed Time elapsed: 408155 ms Time elapsed w/o network: 408144 ms Data processed: 417.593 MiB Data remaining: 0.000 B Data total: 1.102 GiB Memory processed: 417.593 MiB Memory remaining: 0.000 B Memory total: 1.102 GiB Memory bandwidth: 1.026 MiB/s Constant pages: 220266 Normal pages: 106204 Normal data: 414.859 MiB Total downtime: 73 ms Downtime w/o network: 62 ms Setup time: 22 ms Actual results: Can't get domjobinfo on target host before migration completed. Expected results: "Get domjobinfo" successfully on target host before migration completed. Additional info: "virsh domjobinfo <domain>" can get domjobinfo successfully on source host during migration.
This is fixed upstream by commit v1.3.2-47-g3ecd73c: commit 3ecd73c4d8962acfbc4bac05bcd47663f2bd791f Author: Jiri Denemark <jdenemar@redhat.com> AuthorDate: Mon Feb 29 12:34:08 2016 +0100 Commit: Jiri Denemark <jdenemar@redhat.com> CommitDate: Tue Mar 1 15:59:00 2016 +0100 qemu: Don't try to fetch migration stats on destination Migration statistics are not available on the destination host and starting a query job during incoming migration is not allowed. Trying to do that would result in Timed out during operation: cannot acquire state change lock (held by remoteDispatchDomainMigratePrepare3Params) error. We should not even try to start the job. https://bugzilla.redhat.com/show_bug.cgi?id=1278727 Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
This bug was accidentally moved from POST to MODIFIED via an error in automation, please see mmccune@redhat.com with any questions
I can reproduce this. verified with build: libvirt-1.3.3-1.el7.x86_64 qemu-kvm-rhev-2.5.0-4.el7.x86_64 step same as bug's description during the migration, check domjobinfo in target machine. #virsh domjobinfo test error: Operation not supported: migration statistics are available only on the source host worked as expect. move to verified.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2016-2577.html