Description of problem: I'm running IOMeter Benchmark on XP guest. Benchmarks results are very bad ( about 1 MB sec for both read/write ) I'm using NFS storage with COW sparse disk . When i perform IO operation manually on my host using dd / cp I'm reaching about 170-200 MB sec write rate ( No storage issues ). My hosts are set to work with deadline scheduler Version-Release number of selected component (if applicable): How reproducible: Always Steps to Reproduce: 1.Copy big file to the guest and measure IO performance 2. 3. Actual results: Expected results: Additional info: Kernel info Linux blue-vdsc.qa.lab.tlv.redhat.com 2.6.18-194.26.1.el5 #1 SMP Fri Oct 29 14:21:16 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux KVM version: kvm-83-164.el5_5.23 kvm command line: vdsm 4386 4377 4 12:17 ? 00:01:27 /usr/libexec/qemu-kvm -no-hpet -usbdevice tablet -rtc-td-hack -startdate 2010-11-24T10:17:44 -name kvmIOBenchmark1 -smp 1,cores=1 -k en-us -m 1024 -boot c -net nic,vlan=1,macaddr=00:1a:4a:16:95:7a,model=virtio -net tap,vlan=1,ifname=virtio_10_1,script=no -drive file=/rhev/data-center/00000000-0000-0000-0000-000000000002/00000000-0000-0000-0000-000000000011/images/ee2485d1-8132-4772-9700-b0ecc513f942/f56fc196-eac0-4cd8-bcf3-2f0b409d3828,media=disk,if=ide,cache=off,index=0,serial=72-9700-b0ecc513f942,boot=off,format=qcow2,werror=stop -drive file=/rhev/data-center/00000000-0000-0000-0000-000000000002/00000000-0000-0000-0000-000000000012/images/11111111-1111-1111-1111-111111111111/RHEV-toolsSetup_2.2_51181.iso,media=cdrom,index=2,if=ide -fda fat:floppy:/tmp/8e204d74-4b59-4754-9a7a-ed6200eeb776v4zryv.floppy -pidfile /var/vdsm/8e204d74-4b59-4754-9a7a-ed6200eeb776.pid -soundhw ac97 -spice sslpassword=,sslciphersuite=DEFAULT,sslcert=/var/vdsm/ts/certs/vdsmcert.pem,sslkey=/var/vdsm/ts/keys/vdsmkey.pem,ssldhfile=/var/vdsm/ts/keys/dh.pem,sslcafile=/var/vdsm/ts/certs/cacert.pem,host=0,secure-channels=main+inputs,ic=on,sport=5890,port=5910 -qxl 1 -cpu qemu64,+sse2 -M rhel5.5.0 -notify all -balloon none -smbios type=1,manufacturer=Red Hat,product=RHEL,version=5Server-5.5.0.2,serial=AEFA9F8F-A3EB-3AA5-A112-12FA3AEF464F_00:1a:64:12:34:99,uuid=8e204d74-4b59-4754-9a7a-ed6200eeb776 -vmchannel di:0200,unix:/var/vdsm/8e204d74-4b59-4754-9a7a-ed6200eeb776.guest.socket,server -monitor unix:/var/vdsm/8e204d74-4b59-4754-9a7a-ed6200eeb776.monitor.socket,server
I'm using IDE disk and using Virio network adapter ( Installed with latest 2.2 RHEV-tools )
Needed info to isolate the problem - Try Linux guest to isolate winXp and ide/virtio driver issues - Don't copy files but use standard benchmark test or DD - Does the host test uses NFS as well? - When you use DD, try it with odirect mode (the dd command, in addition to qemu)
If it's qcow2 only, it's most likely due to the metadata flushes. I'm working on getting the impact of them small in upstream by batching requests, but block-queue is even there a very intrusive patch and I don't see any chance to backport it to RHEL 5. There are a few patches that can reduce the flushes a bit and should be possible to backport to RHEL 5, but they won't be able to compensate for the whole impact. In any case, the performance should be better as soon as you start working on already allocated clusters (or if you preallocate metadata), just the initial growth is slow. Can you confirm you see this behaviour?
I try to run my tests again with RAW ( both preallocated and Sparse ) and i didn't saw the problem. It might happen only with qcow2.
Oded, do you have the data from comment #2?
(In reply to comment #2) > Needed info to isolate the problem > > - Try Linux guest to isolate winXp and ide/virtio driver issues > - Don't copy files but use standard benchmark test or DD > - Does the host test uses NFS as well? > - When you use DD, try it with odirect mode (the dd command, in addition to > qemu) I'm running IOMeter benchmark from a VM snapshot ( COW Sparse ) based on COW Sparse template . I'm using NFS storage. I tried to reproduce this issue with RAW images on NFS ( Both Sparse / Preallocated ) without success. I also tried to reproduce this issue on ISCSI: ( RAW Preallocated , COW Sparse ) without success. I assume that the problem is qcow2 related when using NFS storage. If we need to test other kind of guests / DD odirect modes in order to isolate the problem please ask KVM QE guys to perform these tests.
What do you mean there is no problem? What are the performance?
I run IOMeter benchmark again with NFS setup: 32kb IO size 50% read 50% write sequential write. For RAW / Sparse and RAW preallocated I've got about 4 MB/Sec (2 read and 2 write) For COW Sparse i got about 2.5 MB/Sec ( 1.25 read and 1.25 Write) I'm not sure that this issue is relevant for qcow2 disk type only.
Created attachment 463378 [details] IOMeter benchmark results
The above is pretty slow even for raw. We might want to test it on a faster server.