When QEMU was configured with the aio=native parameter, KVM virtual machines were slowed significantly. With this update, asynchronous I/O (AIO) can correctly process more than one request at a time, and using aio=native no longer has a negative impact on guest performance.
DescriptionPradeep Kumar Surisetty
2015-07-15 18:49:44 UTC
Description of problem:
While running I/O workloads on RHEL 7.1 VM on RHEL 7.1 host using qemu-kvm-rhev:
We realised aio=native is 50% lesser than aio=threads. There are several requests to process. But AIO is proceesing one at a time.
Systemtap logs to monitor
virtio_queue_notify: ts=1436461565480052,vdev=140628771509224,n=0,vq=140628770978384
virtio_blk_handle_read: ts=1436461565480071,req=140628771310288,sector=0,nsectors=8
virtio_blk_handle_read: ts=1436461565480134,req=140628771703808,sector=8,nsectors=8
virtio_blk_handle_read: ts=1436461565480150,req=140628771788928,sector=16,nsectors=8
virtio_blk_handle_read: ts=1436461565480165,req=140628771838192,sector=24,nsectors=8
virtio_blk_handle_read: ts=1436461565480179,req=140628818049344,sector=32,nsectors=8
virtio_blk_handle_read: ts=1436461565480193,req=140628818098608,sector=40,nsectors=8
virtio_blk_handle_read: ts=1436461565480207,req=140628818147872,sector=48,nsectors=8
virtio_blk_handle_read: ts=1436461565480221,req=140628818197136,sector=56,nsectors=8
virtio_blk_handle_read: ts=1436461565480271,req=140628818246400,sector=64,nsectors=8
Version-Release number of selected component (if applicable):
How reproducible:
Steps to Reproduce:
1. Start RHEL7.1 VM on RHEL7.1 host with qemu-kvm-rhev
2. Trigger any IO workload on VM start reading from a disk in chunks. For ex: 4K sequential read.
3. Monitor systemtap traces for io_submit
Actual results:
AIO processing only one request
Expected results:
AIO should batch IO reqeusts/
Additional info:
Comment 4Pradeep Kumar Surisetty
2015-11-12 04:20:11 UTC
stefan has provided upstream fix for this.
Commit id: fc73548e444ae3239f6cef44a5200b5d2c3e85d1
The raw-posix block driver implements Linux AIO batching so multiple
requests can be submitted with a single io_submit(2) system call.
Batching is currently only used by virtio-scsi and
virtio-blk-data-plane.
Enable batching for regular virtio-blk so the number of io_submit(2)
system calls is reduced for workloads with queue depth > 1.
Commit fc73548e444ae3239f6cef44a5200b5d2c3e85d1 is included in qemu-kvm-rhev-2.5.0-1.el7. It was introduced as part of the QEMU 2.5 rebase.
Dropping RHEL 7.2.z flag. Unless there is a specific justification for z-stream this improvement can become available in RHEL 7.3.
Comment 20Miroslav Rezanina
2016-03-17 07:33:27 UTC
*** Bug 1285624 has been marked as a duplicate of this bug. ***
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
https://rhn.redhat.com/errata/RHBA-2016-2673.html