Hide Forgot
Description of problem: VM to VM netperf/TCP_RR over OVS-dpdk tunnel only delivered single digit transaction rate per second *** test reults *** [root@localhost ~]# for i in {1..3} > do > echo Test $i > !net netperf -H 172.16.3.120 -t TCP_RR -l 60 > done Test 1 MIGRATED TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 172.16.3.120 () port 0 AF_INET : first burst 0 Local /Remote Socket Size Request Resp. Elapsed Trans. Send Recv Size Size Time Rate bytes Bytes bytes bytes secs. per sec 16384 87380 1 1 60.00 9.05 16384 87380 Test 2 MIGRATED TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 172.16.3.120 () port 0 AF_INET : first burst 0 Local /Remote Socket Size Request Resp. Elapsed Trans. Send Recv Size Size Time Rate bytes Bytes bytes bytes secs. per sec 16384 87380 1 1 60.00 9.03 16384 87380 Test 3 MIGRATED TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 172.16.3.120 () port 0 AF_INET : first burst 0 Local /Remote Socket Size Request Resp. Elapsed Trans. Send Recv Size Size Time Rate bytes Bytes bytes bytes secs. per sec 16384 87380 1 1 60.00 9.62 16384 87380 *** OVS-dpdk vxlan tunnel *** f09d57f0-e921-44b8-a544-8e39b387a7be Bridge "ovsbr0" Port "dpdk0" Interface "dpdk0" type: dpdk Port "ovsbr0" Interface "ovsbr0" type: internal Bridge "ovsbr1" Port "ovsbr1" Interface "ovsbr1" type: internal Port "vhost0" Interface "vhost0" type: dpdkvhostuser Port "vxlan0" Interface "vxlan0" type: vxlan options: {dst_port="8472", key="1000", remote_ip="192.168.9.110"} ovs_version: "2.5.0" Version-Release number of selected component (if applicable): How reproducible:Reproducible Steps to Reproduce: Need two hosts each having a vhostuser guest 1. Configure an OVS-dpdk vxlan tunnel with a vhostuser port on each host. 2. Run netperf/TCP_RR test between the two guests. 3. Please see test results and the OVS-dpdk tunnel from the test-bed above. Actual results: Expected results: Additional info:
For your reference listed below are test results for 64 bytes UDP_STREAM and TCP_STREAM. These two results seem to be normal. *** 64 bytes UDP_STREAM *** [root@localhost ~]# for i in {1..3}; do echo Test $i; netperf -H 172.16.3.120 -t UDP_STREAM -l 60 -- -m 64; done Test 1 MIGRATED UDP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 172.16.3.120 () port 0 AF_INET Socket Message Elapsed Messages Size Size Time Okay Errors Throughput bytes bytes secs # # 10^6bits/sec 212992 64 60.00 27101250 0 231.26 212992 60.00 15965386 136.24 Test 2 MIGRATED UDP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 172.16.3.120 () port 0 AF_INET Socket Message Elapsed Messages Size Size Time Okay Errors Throughput bytes bytes secs # # 10^6bits/sec 212992 64 60.00 27134502 0 231.55 212992 60.00 15451765 131.85 Test 3 MIGRATED UDP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 172.16.3.120 () port 0 AF_INET Socket Message Elapsed Messages Size Size Time Okay Errors Throughput bytes bytes secs # # 10^6bits/sec 212992 64 60.00 27125907 0 231.47 212992 60.00 16206556 138.30 *** TCP_STREAM *** [root@localhost ~]# for i in {1..3}; do echo Test $i; netperf -H 172.16.3.120 -l 60; done Test 1 MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 172.16.3.120 () port 0 AF_INET Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. 10^6bits/sec 87380 16384 16384 60.00 3163.56 Test 2 MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 172.16.3.120 () port 0 AF_INET Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. 10^6bits/sec 87380 16384 16384 60.00 3270.43 Test 3 MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 172.16.3.120 () port 0 AF_INET Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. 10^6bits/sec 87380 16384 16384 60.01 3336.79
Please attach sosreports from the host and guests. This doesn't have any rhel, or ovs versions attached.
*** vhostuser guest of Host netqe9 *** [root@localhost ~]# rpm -qa | grep dpdk dpdk-16.04-4.el7fdb.x86_64 dpdk-tools-16.04-4.el7fdb.x86_64 [root@localhost ~]# rpm -qa | grep openvswitch openvswitch-2.5.0-3.el7.x86_64 [root@localhost ~]# uname -a Linux localhost.localdomain 3.10.0-229.el7.x86_64 #1 SMP Thu Jan 29 18:37:38 EST 2015 x86_64 x86_64 x86_64 GNU/Linux [root@localhost ~]# *** Host netqe9 *** [root@netqe9 ovs-dpdk-tunneling]# rpm -qa | grep dpdk dpdk-16.04-4.el7fdb.x86_64 dpdk-tools-16.04-4.el7fdb.x86_64 [root@netqe9 ovs-dpdk-tunneling]# rpm -qa | grep openvswitch openvswitch-2.5.0-14.git20160727.el7fdb.x86_64 [root@netqe9 ovs-dpdk-tunneling]# uname -a Linux netqe9.knqe.lab.eng.bos.redhat.com 3.10.0-506.el7.x86_64 #1 SMP Mon Sep 12 23:31:02 EDT 2016 x86_64 x86_64 x86_64 GNU/Linux [root@netqe9 ovs-dpdk-tunneling]# *** vhostuser guest of Host netqe10 *** [root@localhost jhsiao]# rpm -qa | grep dpdk dpdk-tools-16.04-4.el7fdb.x86_64 dpdk-16.04-4.el7fdb.x86_64 [root@localhost jhsiao]# rpm -qa | grep openvswitch openvswitch-2.5.0-3.el7.x86_64 [root@localhost jhsiao]# uname -a Linux localhost.localdomain 3.10.0-229.el7.x86_64 #1 SMP Thu Jan 29 18:37:38 EST 2015 x86_64 x86_64 x86_64 GNU/Linux *** Host netqe10 *** [root@netqe10 sos]# rpm -qa | grep dpdk dpdk-tools-16.04-4.el7fdb.x86_64 dpdk-16.04-4.el7fdb.x86_64 [root@netqe10 sos]# rpm -qa | grep openvswitch openvswitch-2.5.0-14.git20160727.el7fdp.x86_64 [root@netqe10 sos]# uname -a Linux netqe10.knqe.lab.eng.bos.redhat.com 3.10.0-506.el7.x86_64 #1 SMP Mon Sep 12 23:31:02 EDT 2016 x86_64 x86_64 x86_64 GNU/Linux
Looks like the issue is gone with ovs-2.6.1-2 & dpdk-16.11-2. [root@localhost dpdk-vxlan-tunnel-4Q-ovs-2.6.1-dpdk-16.11]# netperf -H 172.16.63.2 -t TCP_RR -l 60 MIGRATED TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 172.16.63.2 () port 0 AF_INET : first burst 0 Local /Remote Socket Size Request Resp. Elapsed Trans. Send Recv Size Size Time Rate bytes Bytes bytes bytes secs. per sec 16384 87380 1 1 60.00 21651.47 16384 87380
Closing BZ as fix in latest is confirmed.