Bug 509383
| Summary: | rotational mode is much faster for virtio-blk disks, but uses non-rotational mode by default | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Richard W.M. Jones <rjones> | ||||||
| Component: | kernel | Assignee: | Justin M. Forbes <jforbes> | ||||||
| Status: | CLOSED ERRATA | QA Contact: | Fedora Extras Quality Assurance <extras-qa> | ||||||
| Severity: | medium | Docs Contact: | |||||||
| Priority: | high | ||||||||
| Version: | rawhide | CC: | chellwig, dwmw2, erikj, gcosta, itamar, jaswinder, jmoyer, kernel-maint, kwolf, loganjerry, markmc, mishu, virt-maint | ||||||
| Target Milestone: | --- | Keywords: | Reopened | ||||||
| Target Release: | --- | ||||||||
| Hardware: | All | ||||||||
| OS: | Linux | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | 2.6.30.8-64.fc11 | Doc Type: | Bug Fix | ||||||
| Doc Text: | Story Points: | --- | |||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2009-09-17 13:36:05 UTC | Type: | --- | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Bug Depends On: | |||||||||
| Bug Blocks: | 480594, 513462 | ||||||||
| Attachments: |
|
||||||||
|
Description
Richard W.M. Jones
2009-07-02 14:40:32 UTC
Created attachment 350290 [details]
mke2fs.sh
This is a libguestfs / guestfish script which reproduces
the problem.
Total runtime goes from 30s (virtio) to 21s (IDE).
HOWEVER that includes the overhead of starting up the
libguestfs appliance. The time taken for the mke2fs
command goes from 8.63s (virtio) to 0.92s (IDE).
That means IDE is ~ 9 times faster than virtio at
running the mkfs.ext2 command.
Created attachment 350293 [details]
mke2fs.sh
Improved test script. This really does need libguestfs
built from source / libguestfs >= 1.0.55.
Output from my laptop:
Testing using virtio backing disk:
500+0 records in
500+0 records out
524288000 bytes (524 MB) copied, 24.7128 s, 21.2 MB/s
elapsed time: 10.89 seconds
Testing using IDE backing disk:
500+0 records in
500+0 records out
524288000 bytes (524 MB) copied, 21.7416 s, 24.1 MB/s
elapsed time: 1.31 seconds
Thanks for Michael Tokarev and Charles Duffy for the following suggestion which has mostly fixed this problem: for f in /sys/block/vd*/queue/rotational; do echo 1 > $f; done (inside the guest obviously). Now the times are down to: virtio: 1.39s IDE: 0.98s So it's not perfect, but the situation is greatly improved by this change. Pushed a cset to libguestfs to implement this: http://git.et.redhat.com/?p=libguestfs.git;a=commitdiff;h=373a9ad0abb09ee079da834d251a744cdbe9ae70 libguestfs-1.0.55-1.fc11 has been submitted as an update for Fedora 11. http://admin.fedoraproject.org/updates/libguestfs-1.0.55-1.fc11 virtio_blk is defaulting to non-rotational mode since these commits in 2.6.29: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=88e740f165 http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=7d116b626b The discussion that lead to the changes: http://lkml.org/lkml/2008/10/27/84 hch: any thoughts on this? Umm, maybe try elevator=deadline ? (In reply to comment #7) > Umm, maybe try elevator=deadline ? On my local machine using the test described below, elevator=deadline in the guest makes no difference. http://rwmj.wordpress.com/2009/07/04/time-filesystem-operations-using-guestfish/ It looks like the current virtio code defintively needs the plugs that rotational=0 removes. That fits quite well into the bigger pictures as qemu with virtio still introduces far too large latencies for small I/Os. libguestfs-1.0.56-1.el5 has been submitted as an update for Fedora EPEL 5. http://admin.fedoraproject.org/updates/libguestfs-1.0.56-1.el5 libguestfs-1.0.56-2.fc11 has been submitted as an update for Fedora 11. http://admin.fedoraproject.org/updates/libguestfs-1.0.56-2.fc11 *** Bug 504376 has been marked as a duplicate of this bug. *** See my comments in bug 504376 with test results on a mkfs.ext3 to a virtio device with and without rotation enabled. I was asked to dupe it here. libguestfs-1.0.56-1.el5 has been pushed to the Fedora EPEL 5 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with su -c 'yum --enablerepo=updates-testing update libguestfs'. You can provide feedback for this update here: http://admin.fedoraproject.org/updates/EL-5/FEDORA-EPEL-2009-0019 rich: the libguestfs update does not resolve this bug, could you remove the bug # from the update, or clone this bug to libguestfs and use that bug number ? OK, I edited the update to remove this bug number. libguestfs-1.0.58-2.fc11 has been submitted as an update for Fedora 11. http://admin.fedoraproject.org/updates/libguestfs-1.0.58-2.fc11 libguestfs-1.0.55-1.fc11 has been pushed to the Fedora 11 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with su -c 'yum --enablerepo=updates-testing update libguestfs'. You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F11/FEDORA-2009-7493 libguestfs-1.0.58-2.fc11 has been pushed to the Fedora 11 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with su -c 'yum --enablerepo=updates-testing update libguestfs'. You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F11/FEDORA-2009-7557 (In reply to comment #16) > OK, I edited the update to remove this bug number. Still there. Bodhi seems to have copied the bug number into half a dozen different updates. I don't want to edit http://admin.fedoraproject.org/updates/F11/FEDORA-2009-7557 yet because if I do that, it will set the status back to pending, and I want it stable ASAP. Instead I'll just keep an eye on this bug and undo any changes that the update system makes. libguestfs-1.0.58-2.fc11 has been pushed to the Fedora 11 stable repository. If problems still persist, please make note of it in this bug report. Damn. Hopefully the updates of doom will leave us alone now. (In reply to comment #9) > It looks like the current virtio code defintively needs the plugs that > rotational=0 removes. That fits quite well into the bigger pictures as qemu > with virtio still introduces far too large latencies for small I/Os. Can we get the default changed back to non-rotational mode, then? I've done a lot of benchmarking, and I can't actually see any difference between rotational=0 vs =1 at all. This is all using raw images, sparse or not. You aren't doing something braindead like using qcow2, are you? (In reply to comment #25) > I've done a lot of benchmarking, and I can't actually see any difference > between rotational=0 vs =1 at all. Strange > This is all using raw images, sparse or not. You aren't doing something > braindead like using qcow2, are you? Nope, looks like Rich was using a non-sparse raw image with cache=none Christoph has posted a patch to to switch back to rotational mode: http://patchwork.kernel.org/patch/45787/ Justin: maybe you could apply the patch? This patch has been added to the F12 kernels. Fixed in F12 since kernel-2.6.31-0.138.rc5.git3.fc12 and F11 since 2.6.30.7-58 * Wed Sep 09 2009 Justin M. Forbes <jforbes> - Revert virio_blk to rotational mode. (#509383) kernel-2.6.30.8-64.fc11 has been submitted as an update for Fedora 11. http://admin.fedoraproject.org/updates/kernel-2.6.30.8-64.fc11 kernel-2.6.30.8-64.fc11 has been pushed to the Fedora 11 stable repository. If problems still persist, please make note of it in this bug report. |