Bug 1729442 - ovs kernel throughput running in RHEL-7.7 RC1 is lower than RHEL-7.6
Summary: ovs kernel throughput running in RHEL-7.7 RC1 is lower than RHEL-7.6
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: openvswitch
Version: 7.7
Hardware: Unspecified
OS: Unspecified
Target Milestone: rc
: ---
Assignee: Aaron Conole
QA Contact: Jiying Qiu
Depends On:
TreeView+ depends on / blocked
Reported: 2019-07-12 09:50 UTC by Jiying Qiu
Modified: 2019-10-07 10:30 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed: 2019-10-07 10:30:16 UTC
Target Upstream Version:

Attachments (Terms of Use)

Description Jiying Qiu 2019-07-12 09:50:17 UTC
Description of problem:
ovs kernel throughput running in RHEL-7.7 is lower than RHEL-7.6

Version-Release number of selected component (if applicable):
openvswitch2.11-2.11.0-12.el7fdp.x86_64.rpm & RHEL-7.7-20190702.0
openvswitch2.11-2.11.0-9.el7fdp.x86_64.rpm & RHEL-7.6

How reproducible:

Steps to Reproduce:
1. setup PVP topo, 
2. 1k flows ,ipv4 traffic (xenaL2-3 traffic generator )

Actual results:
for ovs2.11-9 &RHEL-7.6            got 330197 rx_pps
for ovs2.11-12&RHEL-7.7-20190702.0 got 94382  rx_pps

Expected results:
the performance do not drop

Additional info:

Comment 2 Timothy Redaelli 2019-07-18 08:47:40 UTC

can you try to do the performance test with the same ovs version on both rhel 7.6 and 7.7?

Thank you

Comment 3 Jiying Qiu 2019-07-18 09:34:40 UTC
I have submitted jobs, it would take 2+ days to run the whole performance. Will update the results once it finished.

Comment 4 Jiying Qiu 2019-07-22 06:08:43 UTC
                verisons                                      disable_emc                enable_emc 
openvswitch2.11-2.11.0-12.el7fdp&RHEL7.7-RC1                    98902                       64254
openvswitch2.11-2.11.0-12.el7fdp&RHEL7.6                       143340                      301512
openvswitch2.11-2.11.0-9.el7fdp &RHEL7.7-RC1                    90616                       78566
openvswitch2.11-2.11.0-9.el7fdp &RHEL7.6                       330197                       83147

Details can be find in https://docs.google.com/spreadsheets/d/1dE6Iz9RcFDjNLhllWToPOqv4Z2sES4minSzaLC3jmi4/edit?usp=sharing.

Comment 5 Jiying Qiu 2019-07-22 06:46:21 UTC
All these jobs run with RHEL-7.6 guest and testpmd of dpdk-18.11-4.el7_6 version in guest.

Comment 11 Jiying Qiu 2019-08-06 06:46:01 UTC
After donging bisect testing, found Beta-1.1 RHEL-7.7-20190514.2 has a big drop. 
verisons                                                     disable_emc                enable_emc 
openvswitch2.11-2.11.0-12.el7fdp & RHEL-7.7-20190514.0         93630                       298500
openvswitch2.11-2.11.0-12.el7fdp & RHEL-7.7-20190514.2         49190                        49190

Comment 12 Aaron Conole 2019-09-13 20:55:30 UTC
This is a kernel issue - what are the results of the kernel bisect?

Comment 13 Jiying Qiu 2019-09-30 07:29:49 UTC
Have rerun the Beta-1.1 RHEL-7.7-20190514.2 and Beta-1.0 RHEL-7.7-20190514.0
verisons                                                     disable_emc                enable_emc               job url
openvswitch2.11-2.11.0-12.el7fdp & RHEL-7.7-20190514.0         42412                       42412                 https://beaker.engineering.redhat.com/jobs/3815182
openvswitch2.11-2.11.0-12.el7fdp & RHEL-7.7-20190514.2         41658                       37892                 https://beaker.engineering.redhat.com/jobs/3815181 

Both distro are kernel-3.10.0-1048.el7.

will try kernel-3.10.0-1040 "Alpha-1.2" and kernel-3.10.0-1034 "DevelPhaseExit-1.0" .

Comment 14 Jiying Qiu 2019-10-07 10:30:16 UTC
verisons                                                               disable_emc           enable_emc             job url
openvswitch2.11-2.11.0-12.el7fdp & RHEL-7.7-20190514.0                   42412                42412                 https://beaker.engineering.redhat.com/jobs/3815182(qemu2.10) & https://beaker.engineering.redhat.com/jobs/3821690 (qemu2.12)
openvswitch2.11-2.11.0-12.el7fdp & RHEL-7.7-20190514.2                   41658                37892                 https://beaker.engineering.redhat.com/jobs/3815181(qemu2.10) & https://beaker.engineering.redhat.com/jobs/3821689 (qemu2.12)
openvswitch2.11-2.11.0-12.el7fdp & RHEL-7.7-20190612.0(FDP19.D)          42412                43166                 https://beaker.engineering.redhat.com/jobs/3821720
openvswitch2.11-2.11.0-12.el7fdp & RHEL-7.6                              42412                42412                 https://beaker.engineering.redhat.com/jobs/3821719

As FDP19.D and RHEL-7.6 jobs results, I think there are full reasons to say this is not a bug, but a problem of xenaL2-3 traffic generator.

Note You need to log in before you can comment on or make changes to this bug.