Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
Bug 1376542 - RHSA-2016-1756 breaks migration of instances
RHSA-2016-1756 breaks migration of instances
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm (Show other bugs)
7.3
Unspecified Unspecified
urgent Severity high
: rc
: ---
Assigned To: Stefan Hajnoczi
huiqingding
: Regression, ZStream
Depends On: 1372763
Blocks: 1371943 1374364 1374365 1374366 1374367 1374368 1374369 1374623 1380306
  Show dependency treegraph
 
Reported: 2016-09-15 12:26 EDT by Stefan Hajnoczi
Modified: 2017-03-15 17:22 EDT (History)
31 users (show)

See Also:
Fixed In Version: qemu-kvm-1.5.3-126.el7
Doc Type: Bug Fix
Doc Text:
The fix for CVE-2016-5403 caused migrating guest instances to fail with a "Virtqueue size exceeded" error message. With this update, the value of the virtualization queue is recalculated after the migration, and the described problem no longer occurs.
Story Points: ---
Clone Of: 1372763
: 1380306 (view as bug list)
Environment:
Last Closed: 2016-11-03 16:03:09 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 2598111 None None None 2016-09-15 12:26 EDT
Red Hat Product Errata RHSA-2016:2585 normal SHIPPED_LIVE Moderate: qemu-kvm security, bug fix, and enhancement update 2016-11-03 08:09:03 EDT

  None (edit)
Comment 2 Stefan Hajnoczi 2016-09-16 04:50:45 EDT
Please verify as follows:

Create a 4 MB (!) LVM volume on the host.  I have called it /dev/testvg/testlv.

Now create qcow2 metadata on the LVM volume:

$ sudo qemu-img create -f qcow2 /dev/testvg/testlv 10G

(The reason for using an LVM volume instead of a regular file is that its size is fixed.  Since the LVM volume cannot grow automatically we can cause ENOSPC to happen when the guest writes to it.)

shell1$ sudo qemu-system-x86_64 -enable-kvm -m 1024 -cpu host -drive if=virtio,cache=none,format=raw,file=rhel72.img -drive if=virtio,cache=none,format=qcow2,file=/dev/testvg/testlv,werror=stop

guest# dd if=/dev/zero of=/dev/vdb oflag=direct bs=4k

The guest should be paused almost immediately because /dev/testvg/testlv runs out of space and returns an ENOSPC write error.

Now launch the destination QEMU for live migration (on the same host):

shell2$ sudo qemu-system-x86_64 -enable-kvm -m 1024 -cpu host -drive if=virtio,cache=none,format=raw,file=rhel72.img -drive if=virtio,cache=none,format=qcow2,file=/dev/testvg/testlv,werror=stop -incoming tcp::1234
(qemu1) migrate tcp:127.0.0.1:1234

After migration has completed the guest is still paused.  Let's grow the LVM volume so the failed write request can be retried:

$ sudo lvresize -L +4M /dev/testvg/testlv
(qemu2) c

Expected behavior:
Guest resumes successfully when the 'c' monitor command is issued on destination
QEMU.  Note that it will probably pause again very soon because the LVM volume runs out of space again.

Actual behavior:
"Virtqueue size exceeded" error from destination QEMU and guest is terminated
after the 'c' monitor command is issued.
Comment 4 Miroslav Rezanina 2016-09-20 12:03:10 EDT
Fix included in qemu-kvm-1.5.3-126.el7
Comment 6 huiqingding 2016-09-20 23:21:58 EDT
Reproduce this bug using the following version:
kernel-3.10.0-509.el7.x86_64
qemu-kvm-1.5.3-125.el7.x86_64

Do the test as comment #2, the detailed steps as bz1372763 comment #11. After migration and type "c" in destination qemu-kvm, guest cannot resume and destination qemu-kvm quits with error "Virtqueue size exceeded".


Verify this bug using the following version:
kernel-3.10.0-509.el7.x86_64
qemu-kvm-1.5.3-126.el7.x86_64

Do the test as comment #2, the detailed steps as bz1372763 comment #11.  After migration and type "c" in destination qemu-kvm, guest can resume normally and guest is paused with io-error.
Comment 7 huiqingding 2016-09-20 23:23:05 EDT
Based on comment 6, set this bug to be verified.
Comment 10 errata-xmlrpc 2016-11-03 16:03:09 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2016-2585.html
Comment 11 errata-xmlrpc 2016-11-03 17:51:55 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2016-2585.html

Note You need to log in before you can comment on or make changes to this bug.