Bug 1304818

Summary: Slow networking performance on Windows guests
Product: [Community] Virtualization Tools Reporter: Mohammed Gamal <mohammed.gamal>
Component: virtio-winAssignee: Yvugenfi <yvugenfi>
Status: CLOSED DUPLICATE QA Contact: Virtualization Bugs <virt-bugs>
Severity: high Docs Contact:
Priority: unspecified    
Version: unspecifiedCC: ghammer, juzhang, lijin, michen, sapandit, virt-maint, vrozenfe, wquan, yama, yvugenfi
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-08-01 08:39:30 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 950611    
Bug Blocks:    
Attachments:
Description Flags
iperf stats for Windows 2012 none

Description Mohammed Gamal 2016-02-04 17:20:39 UTC
Description of problem:
We're experiencing relatively slow network performance under KVM guests running Windows 2012 and Windows 2008. We are not seeing this problem with Linux guests nevertheless. I suspect it might be due to virtio drivers.

The VMs are running on on 64-core AMD Opteron(tm) Processor 6386 SE CPU, and are connected to the network through Mellanox Technologies MT26428 Infiniband cards.


Version-Release number of selected component (if applicable):
virtio-win: 62.73.104.11200

Host distro: Debian Wheezy (with custom built 3.12.45 kernel)

Qemu version: 2.0.2

Qemu command line:
/usr/bin/kvm
-uuid 91689c36-c8f6-4244-8841-d596274ed847 \
-m 2048 \
-M pc-1.0 \
-enable-kvm \
-nodefconfig \
-nodefaults \
-rtc base=utc \
-netdev tap,ifname=n02015d6f3294,id=hostnet6,script=no,downscript=no \
-device virtio-net-pci,netdev=hostnet6,id=net6,mac=02:01:5d:6f:32:94,bus=pci.0,addr=0x6 \
-netdev tap,ifname=n0201d4f27e6b,id=hostnet7,script=no,downscript=no \
-device virtio-net-pci,netdev=hostnet7,id=net7,mac=02:01:d4:f2:7e:6b,bus=pci.0,addr=0x7 \
-chardev pty,id=charserial0 \
-device isa-serial,chardev=charserial0,id=serial0 \
-usb \
-device usb-tablet,id=input0 \
-vnc 0.0.0.0:98 \
-vga std \
-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3 \
-cpu Opteron_G4,+cmp_legacy,+mmxext,+nrip_save,+osvw,+vme,+npt \
-smp 2,sockets=32,cores=1,maxcpus=64,threads=1 \
-drive file=uuid://6a47bff5-59b6-41fb-8ed1-91324ecf12b3,if=none,id=drive-virtio-disk5,format=raw,cache=none \
-device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk5,id=virtio-disk5,bootindex=1 \
-L /usr/share/seabios

How reproducible:
Always reproducible

Steps to Reproduce:
1. Start a Debian 8 guest with the parameters above
2. Start a Windows 2012 guest with the parameters above
3. Download iperf3 from http://www.iperf.fr for Both Windows and Linux
4. On the Linux guest, install the iperf3 package and run 'iperf3 -s'
5. On the Windows guest, install the iperf3 package and run 'iperf3 -c <iperf_server_address> -p 5201
6. Check the stats produced by iperf
7. Start another Debian 8 guest. Install iperf3 and run 'iperf3 -c <iperf_server_address> -p 5201
8. Compare stats with Windows guest 

Actual results:
The Linux guest produces the following results:
$ iperf3 -c 10.15.90.11 -p 5201
Connecting to host 10.15.90.11, port 5201
[  4] local 10.15.90.12 port 49989 connected to 10.15.90.11 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec   669 MBytes  5.61 Gbits/sec    0   3.23 MBytes       
[  4]   1.00-2.00   sec   601 MBytes  5.04 Gbits/sec    0   3.23 MBytes       
[  4]   2.00-3.00   sec   612 MBytes  5.14 Gbits/sec    0   3.23 MBytes       
[  4]   3.00-4.00   sec   619 MBytes  5.19 Gbits/sec    0   3.23 MBytes       
[  4]   4.00-5.00   sec   752 MBytes  6.31 Gbits/sec    0   3.23 MBytes       
[  4]   5.00-6.00   sec   664 MBytes  5.57 Gbits/sec    0   3.23 MBytes       
[  4]   6.00-7.00   sec   696 MBytes  5.84 Gbits/sec    0   3.23 MBytes       
[  4]   7.00-8.00   sec   625 MBytes  5.24 Gbits/sec    0   3.23 MBytes       
[  4]   8.00-9.00   sec   635 MBytes  5.32 Gbits/sec    0   3.23 MBytes       
[  4]   9.00-10.00  sec   904 MBytes  7.59 Gbits/sec    0   3.23 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  6.62 GBytes  5.69 Gbits/sec    0             sender
[  4]   0.00-10.00  sec  6.62 GBytes  5.69 Gbits/sec                  receiver

On the other hand Windows guest produce only about 1.5 Gbps average bandwidth (see attached image).

Expected results:
Windows and Linux guests should show similar results.

Additional info:
Host lspci output:
00:00.0 Host bridge: Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (external gfx0 port A) (rev 02)
00:00.2 IOMMU: Advanced Micro Devices [AMD] nee ATI RD990 I/O Memory Management Unit (IOMMU)
00:02.0 PCI bridge: Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (PCI express gpp port B)
00:04.0 PCI bridge: Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (PCI express gpp port D)
00:11.0 SATA controller: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 SATA Controller [AHCI mode]
00:12.0 USB controller: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:12.1 USB controller: Advanced Micro Devices [AMD] nee ATI SB7x0 USB OHCI1 Controller
00:12.2 USB controller: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:13.0 USB controller: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:13.1 USB controller: Advanced Micro Devices [AMD] nee ATI SB7x0 USB OHCI1 Controller
00:13.2 USB controller: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:14.0 SMBus: Advanced Micro Devices [AMD] nee ATI SBx00 SMBus Controller (rev 3d)
00:14.3 ISA bridge: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 LPC host controller
00:14.4 PCI bridge: Advanced Micro Devices [AMD] nee ATI SBx00 PCI to PCI Bridge
00:14.5 USB controller: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 USB OHCI2 Controller
00:18.0 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 0
00:18.1 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 1
00:18.2 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 2
00:18.3 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 3
00:18.4 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 4
00:18.5 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 5
00:19.0 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 0
00:19.1 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 1
00:19.2 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 2
00:19.3 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 3
00:19.4 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 4
00:19.5 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 5
00:1a.0 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 0
00:1a.1 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 1
00:1a.2 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 2
00:1a.3 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 3
00:1a.4 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 4
00:1a.5 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 5
00:1b.0 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 0
00:1b.1 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 1
00:1b.2 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 2
00:1b.3 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 3
00:1b.4 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 4
00:1b.5 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 5
00:1c.0 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 0
00:1c.1 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 1
00:1c.2 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 2
00:1c.3 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 3
00:1c.4 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 4
00:1c.5 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 5
00:1d.0 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 0
00:1d.1 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 1
00:1d.2 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 2
00:1d.3 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 3
00:1d.4 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 4
00:1d.5 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 5
00:1e.0 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 0
00:1e.1 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 1
00:1e.2 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 2
00:1e.3 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 3
00:1e.4 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 4
00:1e.5 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 5
00:1f.0 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 0
00:1f.1 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 1
00:1f.2 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 2
00:1f.3 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 3
00:1f.4 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 4
00:1f.5 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 5
01:01.0 VGA compatible controller: Matrox Electronics Systems Ltd. MGA G200eW WPCM450 (rev 0a)
02:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
02:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
03:00.0 InfiniBand: Mellanox Technologies MT26428 [ConnectX VPI PCIe 2.0 5GT/s - IB QDR / 10GigE] (rev b0)

Comment 1 Mohammed Gamal 2016-02-04 17:24:22 UTC
Created attachment 1121169 [details]
iperf stats for Windows 2012

Comment 3 Mohammed Gamal 2016-02-11 13:46:54 UTC
Any progress on this?

Comment 4 Miya Chen 2016-02-14 02:08:59 UTC
Re-assigning wquan@ and yama@ to follow up this issue.

Comment 5 Yanhui Ma 2016-02-14 03:56:20 UTC
(In reply to Mohammed Gamal from comment #3)
> Any progress on this?

Just come back from spring festival holiday. And the machine is busy now, we will reproduce it once the machine is free.

Comment 6 lijin 2016-02-14 04:08:09 UTC
QE CAN reproduce this issue with virtio-win-prewhql-112:win2012 and rhel7 guest show different results when do iperf,rhel7 guest 19Gbits/sec,win2012 4.97 Gbits/sec.

rhel7 guest:
# iperf3 -c 10.73.69.2 -p 5201
Connecting to host 10.73.69.2, port 5201
[  4] local 10.73.68.159 port 53360 connected to 10.73.69.2 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec  1.93 GBytes  16.6 Gbits/sec    0   2.88 MBytes       
[  4]   1.00-2.00   sec  2.27 GBytes  19.5 Gbits/sec    0   3.07 MBytes       
[  4]   2.00-3.00   sec  2.26 GBytes  19.5 Gbits/sec    0   3.07 MBytes       
[  4]   3.00-4.00   sec  2.17 GBytes  18.6 Gbits/sec    0   3.07 MBytes       
[  4]   4.00-5.00   sec  2.16 GBytes  18.6 Gbits/sec    0   3.07 MBytes       
[  4]   5.00-6.00   sec  2.46 GBytes  21.1 Gbits/sec    0   3.07 MBytes       
[  4]   6.00-7.00   sec  2.29 GBytes  19.7 Gbits/sec    0   3.07 MBytes       
[  4]   7.00-8.00   sec  2.16 GBytes  18.5 Gbits/sec    0   3.07 MBytes       
[  4]   8.00-9.00   sec  2.20 GBytes  18.9 Gbits/sec    0   3.07 MBytes       
[  4]   9.00-10.00  sec  2.20 GBytes  18.9 Gbits/sec    0   3.07 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  22.1 GBytes  19.0 Gbits/sec    0             sender
[  4]   0.00-10.00  sec  22.1 GBytes  19.0 Gbits/sec                  receiver

iperf Done.

windows server2012 guest:
Connecting to host 10.73.69.2, port 5201
[  4] local 10.73.68.126 port 49203 connected to 10.73.69.2 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   591 MBytes  4.95 Gbits/sec                  
[  4]   1.00-2.00   sec   641 MBytes  5.38 Gbits/sec                  
[  4]   2.00-3.00   sec   624 MBytes  5.23 Gbits/sec                  
[  4]   3.00-4.00   sec   678 MBytes  5.69 Gbits/sec                  
[  4]   4.00-5.00   sec   616 MBytes  5.17 Gbits/sec                  
[  4]   5.00-6.00   sec   508 MBytes  4.27 Gbits/sec                  
[  4]   6.00-7.00   sec   580 MBytes  4.87 Gbits/sec                  
[  4]   7.00-8.00   sec   612 MBytes  5.14 Gbits/sec                  
[  4]   8.00-9.00   sec   532 MBytes  4.46 Gbits/sec                  
[  4]   9.00-10.00  sec   547 MBytes  4.59 Gbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  5.79 GBytes  4.97 Gbits/sec                  sender
[  4]   0.00-10.00  sec  5.79 GBytes  4.97 Gbits/sec                  receiver

iperf Done.

package info:
kernel-3.10.0-335.el7.x86_64
qemu-kvm-rhev-2.3.0-31.el7_2.5.x86_64
virtio-win-prewhql-112

qemu cli:
/usr/libexec/qemu-kvm -cpu SandyBridge -M pc -enable-kvm -m 2G -smp 2 -nodefconfig -nodefaults -rtc base=localtime,driftfix=slew -boot order=cd,menu=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=112BLK201264KJ9,if=none,id=drive-ide0-0-0,format=raw,serial=mike_cao,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=isa_serial0 -device usb-tablet,id=input0 -vnc 0.0.0.0:0 -vga cirrus -qmp tcp:0:4444,server,nowait -monitor stdio -drive file=en_windows_server_2012_x64_dvd_915478.iso,if=none,id=drive-ide0-0-1,format=raw,cache=none,media=cdrom -device ide-drive,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 -netdev tap,script=/etc/qemu-ifup,downscript=no,id=hostnet0,vhost=on -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:52:3b:65:ee:ff

According to performance team,windows guest performance is indeed slower than linux guest with netperf tool,let's wait for the detailed data.

Comment 7 Yvugenfi@redhat.com 2016-02-14 08:11:26 UTC
(In reply to lijin from comment #6)
> QE CAN reproduce this issue with virtio-win-prewhql-112:win2012 and rhel7
> guest show different results when do iperf,rhel7 guest 19Gbits/sec,win2012
> 4.97 Gbits/sec.
> 
> rhel7 guest:
> # iperf3 -c 10.73.69.2 -p 5201
> Connecting to host 10.73.69.2, port 5201
> [  4] local 10.73.68.159 port 53360 connected to 10.73.69.2 port 5201
> [ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
> [  4]   0.00-1.00   sec  1.93 GBytes  16.6 Gbits/sec    0   2.88 MBytes     
> 
> [  4]   1.00-2.00   sec  2.27 GBytes  19.5 Gbits/sec    0   3.07 MBytes     
> 
> [  4]   2.00-3.00   sec  2.26 GBytes  19.5 Gbits/sec    0   3.07 MBytes     
> 
> [  4]   3.00-4.00   sec  2.17 GBytes  18.6 Gbits/sec    0   3.07 MBytes     
> 
> [  4]   4.00-5.00   sec  2.16 GBytes  18.6 Gbits/sec    0   3.07 MBytes     
> 
> [  4]   5.00-6.00   sec  2.46 GBytes  21.1 Gbits/sec    0   3.07 MBytes     
> 
> [  4]   6.00-7.00   sec  2.29 GBytes  19.7 Gbits/sec    0   3.07 MBytes     
> 
> [  4]   7.00-8.00   sec  2.16 GBytes  18.5 Gbits/sec    0   3.07 MBytes     
> 
> [  4]   8.00-9.00   sec  2.20 GBytes  18.9 Gbits/sec    0   3.07 MBytes     
> 
> [  4]   9.00-10.00  sec  2.20 GBytes  18.9 Gbits/sec    0   3.07 MBytes     
> 
> - - - - - - - - - - - - - - - - - - - - - - - - -
> [ ID] Interval           Transfer     Bandwidth       Retr
> [  4]   0.00-10.00  sec  22.1 GBytes  19.0 Gbits/sec    0             sender
> [  4]   0.00-10.00  sec  22.1 GBytes  19.0 Gbits/sec                 
> receiver
> 
> iperf Done.
> 
> windows server2012 guest:
> Connecting to host 10.73.69.2, port 5201
> [  4] local 10.73.68.126 port 49203 connected to 10.73.69.2 port 5201
> [ ID] Interval           Transfer     Bandwidth
> [  4]   0.00-1.00   sec   591 MBytes  4.95 Gbits/sec                  
> [  4]   1.00-2.00   sec   641 MBytes  5.38 Gbits/sec                  
> [  4]   2.00-3.00   sec   624 MBytes  5.23 Gbits/sec                  
> [  4]   3.00-4.00   sec   678 MBytes  5.69 Gbits/sec                  
> [  4]   4.00-5.00   sec   616 MBytes  5.17 Gbits/sec                  
> [  4]   5.00-6.00   sec   508 MBytes  4.27 Gbits/sec                  
> [  4]   6.00-7.00   sec   580 MBytes  4.87 Gbits/sec                  
> [  4]   7.00-8.00   sec   612 MBytes  5.14 Gbits/sec                  
> [  4]   8.00-9.00   sec   532 MBytes  4.46 Gbits/sec                  
> [  4]   9.00-10.00  sec   547 MBytes  4.59 Gbits/sec                  
> - - - - - - - - - - - - - - - - - - - - - - - - -
> [ ID] Interval           Transfer     Bandwidth
> [  4]   0.00-10.00  sec  5.79 GBytes  4.97 Gbits/sec                  sender
> [  4]   0.00-10.00  sec  5.79 GBytes  4.97 Gbits/sec                 
> receiver
> 
> iperf Done.
> 
> package info:
> kernel-3.10.0-335.el7.x86_64
> qemu-kvm-rhev-2.3.0-31.el7_2.5.x86_64
> virtio-win-prewhql-112
> 
> qemu cli:
> /usr/libexec/qemu-kvm -cpu SandyBridge -M pc -enable-kvm -m 2G -smp 2
> -nodefconfig -nodefaults -rtc base=localtime,driftfix=slew -boot
> order=cd,menu=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive
> file=112BLK201264KJ9,if=none,id=drive-ide0-0-0,format=raw,serial=mike_cao,
> cache=none -device
> ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -chardev
> pty,id=charserial0 -device isa-serial,chardev=charserial0,id=isa_serial0
> -device usb-tablet,id=input0 -vnc 0.0.0.0:0 -vga cirrus -qmp
> tcp:0:4444,server,nowait -monitor stdio -drive
> file=en_windows_server_2012_x64_dvd_915478.iso,if=none,id=drive-ide0-0-1,
> format=raw,cache=none,media=cdrom -device
> ide-drive,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 -netdev
> tap,script=/etc/qemu-ifup,downscript=no,id=hostnet0,vhost=on -device
> virtio-net-pci,netdev=hostnet0,id=net0,mac=00:52:3b:65:ee:ff
> 
> According to performance team,windows guest performance is indeed slower
> than linux guest with netperf tool,let's wait for the detailed data.

Are those guest to guest results? Both sides are Windows and both sides are  RHEL?

Windows ha deficiency without enabled RSC on receive side and will definitely be slower than Linux. While we have implemented RSC - it can be enabled only in compile time because we cannot yet certify such a driver.

Can you test Windows as iperf client (TX only)?

Comment 11 Mohammed Gamal 2016-02-22 09:14:56 UTC
Any progress with this?

Comment 12 Quan Wenli 2016-02-24 03:24:05 UTC
Test 10 virtio-win-prewhql builds, it's not a regression issue. is an old issue from following results. 

rhel 7 guest as iperf3 receiver (iper3 -s)
win2012 64 bits as iperf3 sender (iper3 -c $rhel7_guest_ip -p 5201)

package info:
host/guest 3.10.0-327.el7.x86_64
qemu-kvm-rhev-2.3.0-31.el7_2.7.x86_64


+---------------+---------------+---------------+---------------+---------------+
| rhel7 guest   |2012-build112  |2012-build109  |2012-build106  |2012-build103  |
+---------------+---------------+---------------+---------------+---------------+
| 17.4 Gbits/sec|8.92 Gbits/sec | 8.84 Gbits/sec| 8.76 Gbits/sec| 8.87 Gbits/sec|
+---------------+---------------+---------------+---------------+---------------+


+---------------+---------------+---------------+---------------+
|2012-build100  |2012-build97   |2012-build94   | 2012-build91  |
+---------------+---------------+---------------+---------------+
|8.84 Gbits/sec | 9.07 Gbits/sec| 9.07 Gbits/sec| 8.91 Gbits/sec|
+---------------+---------------+---------------+---------------+

Comment 13 Yanhui Ma 2016-02-25 04:59:51 UTC
(In reply to Mohammed Gamal from comment #11)
> Any progress with this?

please look at comment12.

Comment 14 Mohammed Gamal 2016-03-04 14:39:29 UTC
(In reply to Yanhui Ma from comment #13)
> (In reply to Mohammed Gamal from comment #11)
> > Any progress with this?
> 
> please look at comment12.

Does this mean it's a known issue? Is there an intention to fix it?

Comment 15 Yvugenfi@redhat.com 2016-03-08 08:35:48 UTC
(In reply to Mohammed Gamal from comment #14)
> (In reply to Yanhui Ma from comment #13)
> > (In reply to Mohammed Gamal from comment #11)
> > > Any progress with this?
> > 
> > please look at comment12.
> 
> Does this mean it's a known issue? Is there an intention to fix it?

Sure, we will look into performance issues for next release.

Comment 16 Mohammed Gamal 2016-04-04 16:30:11 UTC
Any updates on this?

Comment 17 Yvugenfi@redhat.com 2016-04-05 08:12:31 UTC
(In reply to Mohammed Gamal from comment #16)
> Any updates on this?

It is in our development queue.

Comment 22 Yvugenfi@redhat.com 2016-04-11 05:58:49 UTC
The receive side performance is related to RSC feature that is currently disabled in our driver (see BZ #950611).

We are working to have certified driver with this feature turned on.

Comment 25 Mohammed Gamal 2016-07-22 14:43:24 UTC
What's the latest state of this bug? I am not sure exactly what is the state of BZ #950611

Comment 26 Yvugenfi@redhat.com 2016-07-24 11:06:07 UTC
(In reply to Mohammed Gamal from comment #25)
> What's the latest state of this bug? I am not sure exactly what is the state
> of BZ #950611

In progress. Patches will be pushed to upstream soon.

Comment 27 Yvugenfi@redhat.com 2016-08-01 08:39:30 UTC

*** This bug has been marked as a duplicate of bug 950611 ***