Red Hat Bugzilla – Bug 1250590
Migration with combined local and network storage fails on assertion error
Last modified: 2015-10-15 19:41:54 EDT
Description of problem: When a guest has both local raw block storage and remote (directly attached) iscsi storage, migration fails after local storage has been trasnfered. Log on source states: qemu-kvm: block/mirror.c:359: mirror_run: Assertion `n > 0' failed. 2015-08-05 13:42:39.769+0000: shutting down Guest is not running on either host when this happens. Version-Release number of selected component (if applicable): 1.5.3-86.el7_1.5 Steps to Reproduce: 1. Create a guest with similar block device configuration to this (snippet from my testing guest): <disk type='block' device='disk'> <driver name='qemu' type='raw' cache='none' io='native' discard='unmap'/> <source dev='/dev/HostVG/virtual_56070'/> <target dev='sda' bus='scsi'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='network' device='disk'> <driver name='qemu' type='raw' cache='none' io='native' discard='unmap'/> <auth username='csuser910'> <secret type='iscsi' usage='iqn.2011-09.cz.hosting90:target774_csuser910'/> </auth> <source protocol='iscsi' name='iqn.2011-09.cz.hosting90:target774/1'> <host name='10.0.3.1' port='3260'/> </source> <target dev='sdb' bus='scsi'/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> 2. Prepare secret objects on target host for the iscsi 3. Start migration to another host with VIR_MIGRATE_NON_SHARED_DISK flag set Actual results: The local storage is copied using nbd but at the end it fails with qemu-kvm: block/mirror.c:359: mirror_run: Assertion `n > 0' failed. Expected results: The guest should successfully migrate Additional info: The guest is using virtio-scsi bus.
Jiri, thanks for taking the time to enter a bug report with us. We use reports like yours to keep improving the quality of our products and releases. That said, we're not able to guarantee the timeliness or suitability of a resolution for issues entered here because this is not a mechanism for requesting support. If this issue is critical or in any way time sensitive, please raise a ticket through your regular Red Hat support channels to make certain it receives the proper attention and prioritization that will result in a timely resolution. For information on how to contact the Red Hat production support team, please visit: https://www.redhat.com/support/process/production/#howto Setting NEEDINFO(QE) to try to reproduce it.
Hi Shaolong, Could you have a try and update it the bz? Best Regards, Junyi
Hi Jiri, In Redhat product, only qemu-kvm-rhev version supports storage migration function, qemu-kvm-1.5.3-86.el7_1.5(without rhev) does not, could you confirm your version? Bests, Shaolong
Created attachment 1064225 [details] Spec file of used build I am aware of that. Since we are using CentOS which currently has no official rhev build of qemu-kvm (we filled a bug for this: https://bugs.centos.org/view.php?id=8281), we are using custom build. Source rpm version is: qemu-kvm-1.5.3-86.el7_1.5 I am attaching spec file of the build.
Note: The bug is also present in package version qemu-kvm-ev-2.1.2-23.el7_1.8.1.x86_64 from CentOS oVirt repository. Due to this I filed a bug in CentOS bug tracker too: https://bugs.centos.org/view.php?id=9497
Workaround/solution: Adding <shareable/> tag to the iSCSI storage solves the issue and results in desired behaviour. I.e. the local storage gets migrated and the iSCSI storage is only re-attached on remote side. This whole thing may have been a missconfiguration. However a more desciptive error message may be a good idea.
Jiri, I checked with our QE and they couldn't reproduce it, so it seems it was a configuration mistake indeed. Since we don't support direct configuration (without using a management tool), I'm closing this bug. Anyway, thanks for reporting it.