Bug 1277353
| Summary: | Virtio-blk and Virtio-scsi performance of windows guest are quite slow | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Yanhui Ma <yama> | ||||||||||
| Component: | virtio-win | Assignee: | Vadim Rozenfeld <vrozenfe> | ||||||||||
| virtio-win sub component: | virtio-win-prewhql | QA Contact: | Virtualization Bugs <virt-bugs> | ||||||||||
| Status: | CLOSED NOTABUG | Docs Contact: | |||||||||||
| Severity: | high | ||||||||||||
| Priority: | high | CC: | ailan, famz, juzhang, lprosek, michal.skrivanek, michen, pbonzini, stefanha, vfeenstr, vrozenfe, wquan, yama, ykaul, yuhuang | ||||||||||
| Version: | 7.2 | ||||||||||||
| Target Milestone: | rc | ||||||||||||
| Target Release: | --- | ||||||||||||
| Hardware: | x86_64 | ||||||||||||
| OS: | Windows | ||||||||||||
| Whiteboard: | |||||||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||||||
| Doc Text: | Story Points: | --- | |||||||||||
| Clone Of: | Environment: | ||||||||||||
| Last Closed: | 2016-06-30 09:41:44 UTC | Type: | Bug | ||||||||||
| Regression: | --- | Mount Type: | --- | ||||||||||
| Documentation: | --- | CRM: | |||||||||||
| Verified Versions: | Category: | --- | |||||||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||||
| Embargoed: | |||||||||||||
| Bug Depends On: | |||||||||||||
| Bug Blocks: | 1288337 | ||||||||||||
| Attachments: |
|
||||||||||||
|
Description
Yanhui Ma
2015-11-03 06:34:36 UTC
Can we please give it another try with the latest drivers from build 118 ( http://download.eng.bos.redhat.com/brewroot/packages/virtio-win-prewhql/0.1/118/win/virtio-win-prewhql-0.1.zip ) Also, please turn CPU flag "hv_time" on. Thanks, Vadim. I just tried this on Win7 and Fedora 23 on a laptop. No NUMA, no vcpu pinning, far from a true perf lab environment. The results came out in favor of Windows (take with a grain of salt). Definitely not seeing order of magnitude differences as suggested by the description. 4k blocks, vioscsi ramdisk Windows: bw=84125KB/s, iops=21031 Fedora: bw=70754KB/s, iops=17688 256k blocks, vioscsi ramdisk Windows: bw=2849.6MB/s, iops=11398 Fedora: bw=1822.7MB/s, iops=7290 Yanhui Ma, can you please provide the full fio command line you used on RHEL? The result links on the MOJO page you linked seem to be dead. Thanks! (In reply to Ladi Prosek from comment #4) > I just tried this on Win7 and Fedora 23 on a laptop. No NUMA, no vcpu > pinning, far from a true perf lab environment. The results came out in favor > of Windows (take with a grain of salt). Definitely not seeing order of > magnitude differences as suggested by the description. > > 4k blocks, vioscsi ramdisk > Windows: bw=84125KB/s, iops=21031 > Fedora: bw=70754KB/s, iops=17688 > > 256k blocks, vioscsi ramdisk > Windows: bw=2849.6MB/s, iops=11398 > Fedora: bw=1822.7MB/s, iops=7290 > > Yanhui Ma, can you please provide the full fio command line you used on > RHEL? The result links on the MOJO page you linked seem to be dead. Thanks! The full fio command line in RHEL likes this: fio --rw=read --bs=4k --iodepth=64 --runtime=1m --direct=1 --filename=/dev/sdb --name=job1 --ioengine=libaio --thread --group_reporting --numjobs=16 --time_based --output=/tmp/fio_result You can find fio cml in below link. And I have also updated result links. Now you can access them. http://kvm-perf.englab.nay.redhat.com/results/8798-autotest/hp-z800-06.qe.lab.eng.nay.redhat.com/virt.qemu.rhel72-brew.local.Host_RHEL.7.2.raw.virtio_scsi.smp2.virtio_net.RHEL.7.2.x86_64.fio_linux.single_disk.rawdisk.repeat1/debug/virt.qemu.rhel72-brew.local.Host_RHEL.7.2.raw.virtio_scsi.smp2.virtio_net.RHEL.7.2.x86_64.fio_linux.single_disk.rawdisk.repeat1.DEBUG (In reply to Vadim Rozenfeld from comment #3) > Can we please give it another try with the latest drivers from build 118 > ( > http://download.eng.bos.redhat.com/brewroot/packages/virtio-win-prewhql/0.1/ > 118/win/virtio-win-prewhql-0.1.zip ) > > Also, please turn CPU flag "hv_time" on. > Sure, will have a try. > Thanks, > Vadim. Here are the test results comparing rhel7.2 guest and win2012r2 guest (with build 118 and CPU flag "hv_time" on): https://mojo.redhat.com/docs/DOC-1083825 Compared to RHEL7.2 guest, there is a degradation of 30%~100% for Win2012r2_x86_64 guest with both virtio-blk and virtio-scsi driver. (In reply to Yumei Huang from comment #7) > Here are the test results comparing rhel7.2 guest and win2012r2 guest (with > build 118 and CPU flag "hv_time" on): > > https://mojo.redhat.com/docs/DOC-1083825 > > Compared to RHEL7.2 guest, there is a degradation of 30%~100% for > Win2012r2_x86_64 guest with both virtio-blk and virtio-scsi driver. Could yo please tell me about the Windows VM that was used in this test(s) - is it a fresh one, installed from an .iso image, or did you create it from template? Can you give a try to IDE based disk and see if you get the same or close results? Can it be that IO Throttling is turning on for some reason? Can you please give a try to IoMeter and see if you get the same result? Any chance that I you can share the VM image, so I can try running a clone of your Windows VM system on my setup? Thanks and best regards, Vadim. Here are test results with IDE based disk and the performance of win2012 is still much worse than RHEL7.2 guest: -localfs http://kvm-perf.englab.nay.redhat.com/results/request/Bug1277353/localfs/raw.ide.smp2.virtio_net.html -ramdisk http://kvm-perf.englab.nay.redhat.com/results/request/Bug1277353/1ramdisk/raw.ide.smp2.virtio_net.html Also QE have tried with IOMeter and the performance of win2012 is worse too. The results will be attached. Created attachment 1172824 [details]
IOMeter results with rhel7.2 guest
Created attachment 1172825 [details]
IOMeter results with win2012 guest
Created attachment 1174425 [details]
probe_timer tool source code
Created attachment 1174426 [details]
probe_timer tool x64 binary
For optimum I/O performance it's critical to make sure that Windows guests use the Hyper-V reference counter feature. QEMU command line should include -cpu ...,hv_time and -no-hpet and the useplatformclock Windows boot entry option should be disabled: bcdedit /set useplatformclock false The attached probe_timer can be used to analyze guests. It prints the result of QueryPerformanceFrequency, measures the cost of a QueryPerformanceCounter call, and checks the Hyper-V reference counter feature. |