Bug 520219 - virtio_net performance is very poor
Summary: virtio_net performance is very poor
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: kvm
Version: 5.4
Hardware: All
OS: Linux
low
medium
Target Milestone: rc
: ---
Assignee: Michael S. Tsirkin
QA Contact: Lawrence Lim
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2009-08-29 02:24 UTC by Kirby Zhou
Modified: 2014-03-26 01:01 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2009-12-09 08:02:48 UTC


Attachments (Terms of Use)

Description Kirby Zhou 2009-08-29 02:24:07 UTC
Description of problem:

virtio_net performance is very poor with RHEL 5.4 KVM

Version-Release number of selected component (if applicable):

RHEL-5.4Beta

kernel-2.6.18-155.el5
kvm-83-80.el5
kvm-qemu-img-83-80.el5
kvm-tools-83-80.el5
kmod-kvm-83-80.el5
libvirt-0.6.3-11.el5
libvirt-python-0.6.3-11.el5
python-virtinst-0.400.3-4.el5


How reproducible:

100%

Steps to Reproduce:
1.

Install RHEL-5.4beta as guest under RHEL-5.4beta host. the conf file is here:

<domain type='kvm'>
  <name>vmtest</name>
  <uuid>abd2ee53-bef9-60ad-253a-a9f7768896f9</uuid>
  <memory>1048576</memory>
  <currentMemory>1048576</currentMemory>
  <vcpu>2</vcpu>
  <os>
    <type arch='x86_64' machine='pc'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>
    <disk type='block' device='disk'>
      <source dev='/dev/vgtest/lvtest'/>
      <target dev='vda' bus='virtio'/>
    </disk>
    <disk type='block' device='disk'>
      <source dev='/dev/vgtest/lvtest2'/>
      <target dev='vdb' bus='virtio'/>
    </disk>
    <interface type='bridge'>
      <mac address='54:52:00:73:e5:f8'/>
      <source bridge='xenbr0'/>
      <model type='virtio'/>
    </interface>
    <interface type='bridge'>
      <mac address='54:52:00:73:e5:f9'/>
      <source bridge='xenbr1'/>
      <model type='virtio'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target port='0'/>
    </console>
  </devices>
</domain>

2. run netserver of netperf on another machine

3. run netperf -t UDP_RR -H remote
   run ping -fc 1000 remote
  
Actual results:

Local /Remote
Socket Size   Request  Resp.   Elapsed  Trans.
Send   Recv   Size     Size    Time     Rate
bytes  Bytes  bytes    bytes   secs.    per sec

129024 129024 1        1       10.00     993.18
129024 129024

1000 packets transmitted, 1000 received, 0% packet loss, time 1035ms
rtt min/avg/max/mdev = 0.596/0.949/4.260/0.194 ms, ipg/ewma 1.036/0.921 ms

Expected results:

reduce rtt to less than 0.2ms. 

Additional info:

Xen HVM domu can do it well at the same machine.
[root@localhost ~]# netperf -t TCP_RR -H 10.12.12.111
Local /Remote
Socket Size   Request  Resp.   Elapsed  Trans.
Send   Recv   Size     Size    Time     Rate         
bytes  Bytes  bytes    bytes   secs.    per sec   

16384  87380  1        1       10.00    7355.47   
16384  87380 
[root@localhost ~]# ping -fc 1000 10.12.12.111
PING 10.12.12.111 (10.12.12.111) 56(84) bytes of data.
 
--- 10.12.12.111 ping statistics ---
1000 packets transmitted, 1000 received, 0% packet loss, time 173ms
rtt min/avg/max/mdev = 0.096/0.120/0.175/0.016 ms, ipg/ewma 0.173/0.118 ms
[root@localhost ~]# uname -a
Linux localhost.localdomain 2.6.18-155.el5 #1 SMP Fri Jun 19 17:06:31 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux

Comment 1 Kirby Zhou 2009-09-03 03:23:51 UTC
some more info:

Comment 2 Kirby Zhou 2009-09-03 03:42:51 UTC
some more info: the rtt of virtio is slow than rtl8139 and e1000.

ping -fc100 <kvmguest>

-- with model=virtio
100 packets transmitted, 100 received, 0% packet loss, time 1035ms
rtt min/avg/max/mdev = 0.596/0.949/4.260/0.194 ms, ipg/ewma 1.036/0.921 ms

-- with model=rtl8139
100 packets transmitted, 100 received, 0% packet loss, time 34ms
rtt min/avg/max/mdev = 0.218/0.279/0.539/0.067 ms, ipg/ewma 0.349/0.257 ms

-- model=e1000
100 packets transmitted, 100 received, 0% packet loss, time 31ms
rtt min/avg/max/mdev = 0.181/0.233/0.484/0.056 ms, ipg/ewma 0.317/0.230 ms

iperf -c <kvmguest> from a host in the same LAN

-- with model=virtio
[  3]  0.0-10.0 sec    321 MBytes    270 Mbits/sec

-- with model=rtl8139
[  3]  0.0-10.0 sec    616 MBytes    517 Mbits/sec

-- model=e1000
[  3]  0.0-10.0 sec    645 MBytes    541 Mbits/sec

-- model=xen ** another host in another LAN **
[  3]  0.0-10.0 sec  1.10 GBytes    944 Mbits/sec

iperf -c <kvmguest> to a host in the same LAN

-- with model=virtio
[  3]  0.0-10.0 sec    829 MBytes    695 Mbits/sec

-- with model=rtl8139
[  3]  0.0-10.0 sec    161 MBytes    135 Mbits/sec

-- model=e1000
[  3]  0.0-10.0 sec    298 MBytes    250 Mbits/sec

-- model=xen ** another host in another LAN **
[  3]  0.0-10.0 sec  1.07 GBytes    920 Mbits/sec

Comment 3 Kirby Zhou 2009-09-03 03:44:37 UTC
sorry, a typo happes, the last section above should be:

iperf -c <normalhost> from guest os to a host in the same LAN

-- with model=virtio
[  3]  0.0-10.0 sec    829 MBytes    695 Mbits/sec

-- with model=rtl8139
[  3]  0.0-10.0 sec    161 MBytes    135 Mbits/sec

-- model=e1000
[  3]  0.0-10.0 sec    298 MBytes    250 Mbits/sec

-- model=xen ** another host in another LAN **
[  3]  0.0-10.0 sec  1.07 GBytes    920 Mbits/sec

Comment 5 Dor Laor 2009-11-15 15:19:15 UTC
Can you please retest with kvm-83-105.el5? We removed the tx mitigation timer and it should boost things.

Comment 7 Bill Burns 2009-12-08 20:36:42 UTC
Please see comment #5, Can you retest? Thanks

Comment 8 Kirby Zhou 2009-12-09 06:36:48 UTC
I have retested. It seems OK now.

Comment 9 Dor Laor 2009-12-09 08:02:48 UTC
Closing the bug then. Thanks.


Note You need to log in before you can comment on or make changes to this bug.