Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
The FDP team is no longer accepting new bugs in Bugzilla. Please report your issues under FDP project in Jira. Thanks.

Bug 1797873

Summary: [OVN 2.11][FDP20.B] QoS Max bandwidth limit doesn't work in combination with DVR
Product: Red Hat Enterprise Linux Fast Datapath Reporter: ying xu <yinxu>
Component: ovn2.11Assignee: lorenzo bianconi <lorenzo.bianconi>
Status: CLOSED ERRATA QA Contact: ying xu <yinxu>
Severity: high Docs Contact:
Priority: high    
Version: FDP 20.ACC: ctrautma, dcbw, fhallal, jishi, lorenzo.bianconi, mmichels, ralongi
Target Milestone: ---Keywords: Regression
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: ovn2.11-2.11.1-34.el7fdn Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1778036
: 1798903 (view as bug list) Environment:
Last Closed: 2020-04-14 14:29:49 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1778036    
Bug Blocks: 1798903, 1805581    

Description ying xu 2020-02-04 05:39:47 UTC
This bug can be found on version ovn2.11 FDP20.B
please see the job :
https://beaker.engineering.redhat.com/recipes/7862223/tasks/105609005/logs/taskout.log

+++ This bug was initially created as a clone of Bug #1580542 +++

Description of problem:

   This is the U/S version of the bug:
    
      https://github.com/openvswitch/ovs-issues/issues/150

   When a port has assigned qos_max_rate  / qos_burst and at the same time the port is directly attached to a provider network logical switch (with a physical mapping), or it has a distributed floating IP (which won't go through a tunnel to a gateway chassis), the bandwidth limit has no effect since the queues are only configured to tunnel ports, and not also on physical interfaces of the bridge mappings.


Version-Release number of selected component (if applicable):

2.9.0

How reproducible:

Always

--- Additional comment from Miguel Angel Ajo on 2018-05-21 18:25:55 UTC ---


DescriptionEdit
https://rhos-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/DFG-network-networking-ovn-13_director-rhel-virthost-3cont_2comp-ipv4-geneve-dvr/lastCompletedBuild/testReport/neutron_tempest_plugin.scenario.test_qos/QoSTest/test_qos_id_1f7ed39b_428f_410a_bd2b_db9f465680df_/

--- Additional comment from Mark Michelson on 2018-06-27 18:35:00 UTC ---

Hi Miguel,

I've looked at this a bit, and I think I have a better understanding of what you're saying is the issue.

OVN currently will only allocate QOS queues if it can determine a tunnel interface on which to assign the queue. It does this by looking at interfaces on br-int, and seeing which of those has options:remote_ip set. It then puts the QOS queue on that interface's status:tunnel_egress_interface.

In your case, you have non-tunnel interfaces on which you want to enable QOS queues. So the question here is how OVN can determine that it should add a QOS queue to an interface.

The most explicit way to do it would be to place something in the Interface's options or external_ids, like "qos_egress_interface", and then apply the QOS queue to that. There may be a more "automatic" way of doing this though, such as using the Interface's "type" as a way to determine if the QOS queue should be applied.

Do you have any ideas in mind that would work here? I'm not familiar with what OpenStack's OVN/OVS configuration looks like in this particular case, so seeing what you have could provide some good hints.

--- Additional comment from Scott Lewis on 2018-07-26 16:22:15 UTC ---

Removing TM until this is targeted to a specific release.

--- Additional comment from Jakub Sitnicki on 2018-07-27 16:56:03 UTC ---

Mark,

I belive we now have a better understanding of how OpenStack is using
"localnet" ports after the discussions on MAC flapping and the
explanation in Miguel's post:

https://ajo.es/ovn-distributed-ew-on-vlan/

This is the same setup that Miguel's refers to in this BZ. As you
pointed out, ovn-controller only installs qdisc's on OVN tunnel egress
devices, so it won't work.

What bothers me most, though, is why we don't simply configure qdisc's
on the VIF (be it tap, veth, or OVS internal interface)? This is a
documented solution:

https://github.com/openvswitch/ovs/blob/v2.9.2/Documentation/howto/qos.rst

Is it because we want to limit just the traffic leaving the node? It
seems worth pursuing further to reach an explanation.

Otherwise, ovn-controller logic will have to be extended to find all
egress devices reachable from br-int. Might be tricky.

--- Additional comment from Mark Michelson on 2018-08-09 18:14:57 UTC ---

Hi Miguel,

We could use some guidance on how best to proceed with this. As Jakub pointed out, you have the option of setting up QoS ingress policing based on the link he posted above.

If you want egress QoS similar to what is done on tunnel ports, that requires some changes in OVN. In that case, understanding what would be acceptable from OpenStack's perspective would be useful. As Jakub mentioned, it would be tricky to find all egress devices reachable from br-int, so I would prefer not to go that route if possible.

I previously suggested that in the OpenVswitch Interface table, you could specify "external_ids:qos_egress_iface=<some device>" on the interface that is bound to the localnet logical switch port. I don't think this exact idea will work, because the localnet port will actually be bound to an automatically-created patch port rather than a port you add to br-int.

I think a potential answer might be to follow the patch port to br-phys (or br-ex, or whatever we're calling it), and place a QoS queue on interfaces on that bridge that have something like "external_ids:ovn_qos=yes" set and that are of type "system". This way, we satisfy the need to automatically add egress QoS queues, and we also have confirmation through configuration that the QoS queue is desired on the specific interface.

What do you think about this suggestion? If you don't like it, what would you suggest instead?

--- Additional comment from Miguel Angel Ajo on 2018-09-03 12:25:11 UTC ---

The suggestion seems reasonable to me.

I suspect ovn-controller is already (partially doing that), I discovered when I found the bug, that the egress queue was set to one interface on "br-`ex`", but since sometimes we deploy with several interfaces there (for other purposes), the right one didn't get it.

If that's the case -I wasn't able to follow the code properly- may be we will need to setup a list of interfaces like : external_ids:qos_egress_ifaces=<some device>,<another-device>,..."  ?

--- Additional comment from Mark Michelson on 2018-09-05 20:26:44 UTC ---

I'll have to take another look through the code to see if I can trace where that might be happening. From what I could tell, the only way egress qos qdiscs were set up was for tunnels. It may be that somehow an egress tunnel iface is being set up incorrectly by OVS? I'm not sure.

Your suggestion could work, too. It might be more efficient than my idea as well. I'll provide a patch for you to test and you can tell me if it works.

--- Additional comment from Mark Michelson on 2018-09-25 21:15:00 UTC ---

Hi Miguel. I've created a branch in my fork of OVS where I've attempted what I described earlier. I would appreciate if you could give this a test in your setup to see if it works as expected.

https://github.com/putnopvut/ovs/tree/qos-thing

The method I ended up going with is similar to what I initially discussed, but I decided to make it slightly more generic. On br-ex, you can set 'external-ids:ovn-egress-iface=true' on each interface that should have QoS qdiscs enabled. I have not tested this myself yet, so your feedback will be valuable.

--- Additional comment from Mark Michelson on 2018-11-13 21:45:52 UTC ---

Here's a scratch build of OVN 2.9 I made with the patch applied. Hopefully one of the links below will be useful for testing.

To reiterate how this is intended to work, in the bridge that br-int is connected to, you can specify on the egress interface "external-ids:ovn-egress-interface=true" and we should set up QoS on that interface. Once again, this has not been tested yet because I am not sure about the proper environment to test it in. So this may not work as expected.

A yum repository for the scratch build of openvswitch-2.9.0-81.el7fdn.bz1580542.1 (task 19171354) is available at:

http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/

You can install the rpms locally by putting this .repo file in your /etc/yum.repos.d/ directory:

http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/openvswitch-2.9.0-81.el7fdn.bz1580542.1-scratch.repo

RPMs and build logs can be found in the following locations:
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/aarch64/
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/ppc64le/
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/s390x/
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/x86_64/

The full list of available rpms is:
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/aarch64/openvswitch-2.9.0-81.el7fdn.bz1580542.1.src.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/aarch64/openvswitch-ovn-central-2.9.0-81.el7fdn.bz1580542.1.aarch64.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/aarch64/openvswitch-test-2.9.0-81.el7fdn.bz1580542.1.noarch.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/aarch64/openvswitch-devel-2.9.0-81.el7fdn.bz1580542.1.aarch64.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/aarch64/python-openvswitch-2.9.0-81.el7fdn.bz1580542.1.aarch64.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/aarch64/openvswitch-ovn-vtep-2.9.0-81.el7fdn.bz1580542.1.aarch64.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/aarch64/openvswitch-2.9.0-81.el7fdn.bz1580542.1.aarch64.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/aarch64/openvswitch-ovn-host-2.9.0-81.el7fdn.bz1580542.1.aarch64.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/aarch64/openvswitch-ovn-common-2.9.0-81.el7fdn.bz1580542.1.aarch64.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/aarch64/openvswitch-debuginfo-2.9.0-81.el7fdn.bz1580542.1.aarch64.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/ppc64le/openvswitch-2.9.0-81.el7fdn.bz1580542.1.src.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/ppc64le/openvswitch-ovn-vtep-2.9.0-81.el7fdn.bz1580542.1.ppc64le.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/ppc64le/openvswitch-debuginfo-2.9.0-81.el7fdn.bz1580542.1.ppc64le.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/ppc64le/openvswitch-devel-2.9.0-81.el7fdn.bz1580542.1.ppc64le.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/ppc64le/openvswitch-test-2.9.0-81.el7fdn.bz1580542.1.noarch.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/ppc64le/openvswitch-ovn-central-2.9.0-81.el7fdn.bz1580542.1.ppc64le.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/ppc64le/python-openvswitch-2.9.0-81.el7fdn.bz1580542.1.ppc64le.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/ppc64le/openvswitch-ovn-common-2.9.0-81.el7fdn.bz1580542.1.ppc64le.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/ppc64le/openvswitch-ovn-host-2.9.0-81.el7fdn.bz1580542.1.ppc64le.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/ppc64le/openvswitch-2.9.0-81.el7fdn.bz1580542.1.ppc64le.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/s390x/openvswitch-2.9.0-81.el7fdn.bz1580542.1.src.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/s390x/python-openvswitch-2.9.0-81.el7fdn.bz1580542.1.s390x.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/s390x/openvswitch-test-2.9.0-81.el7fdn.bz1580542.1.noarch.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/s390x/openvswitch-ovn-central-2.9.0-81.el7fdn.bz1580542.1.s390x.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/s390x/openvswitch-ovn-host-2.9.0-81.el7fdn.bz1580542.1.s390x.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/s390x/openvswitch-ovn-common-2.9.0-81.el7fdn.bz1580542.1.s390x.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/s390x/openvswitch-devel-2.9.0-81.el7fdn.bz1580542.1.s390x.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/s390x/openvswitch-2.9.0-81.el7fdn.bz1580542.1.s390x.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/s390x/openvswitch-ovn-vtep-2.9.0-81.el7fdn.bz1580542.1.s390x.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/s390x/openvswitch-debuginfo-2.9.0-81.el7fdn.bz1580542.1.s390x.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/x86_64/openvswitch-2.9.0-81.el7fdn.bz1580542.1.src.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/x86_64/python-openvswitch-2.9.0-81.el7fdn.bz1580542.1.x86_64.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/x86_64/openvswitch-test-2.9.0-81.el7fdn.bz1580542.1.noarch.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/x86_64/openvswitch-debuginfo-2.9.0-81.el7fdn.bz1580542.1.x86_64.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/x86_64/openvswitch-2.9.0-81.el7fdn.bz1580542.1.x86_64.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/x86_64/openvswitch-ovn-central-2.9.0-81.el7fdn.bz1580542.1.x86_64.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/x86_64/openvswitch-ovn-common-2.9.0-81.el7fdn.bz1580542.1.x86_64.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/x86_64/openvswitch-ovn-vtep-2.9.0-81.el7fdn.bz1580542.1.x86_64.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/x86_64/openvswitch-devel-2.9.0-81.el7fdn.bz1580542.1.x86_64.rpm
http://brew-task-repos.usersys.redhat.com/repos/scratch/mmichels/openvswitch/2.9.0/81.el7fdn.bz1580542.1/x86_64/openvswitch-ovn-host-2.9.0-81.el7fdn.bz1580542.1.x86_64.rpm

--- Additional comment from errata-xmlrpc on 2019-10-11 15:41:22 UTC ---

Bug report changed to ON_QA status by Errata System.
A QE request has been submitted for advisory RHBA-2019:47208-01
https://errata.devel.redhat.com/advisory/47208

--- Additional comment from haidong li on 2019-10-29 11:05:18 UTC ---

Verified on the latest version:
[root@dell-per730-42 ovn]# uname -a
Linux dell-per730-42.rhts.eng.pek2.redhat.com 3.10.0-1101.el7.x86_64 #1 SMP Sat Oct 5 04:50:26 EDT 2019 x86_64 x86_64 x86_64 GNU/Linux
[root@dell-per730-42 ovn]# rpm -qa | grep ovn
ovn2.11-host-2.11.1-8.el7fdp.x86_64
ovn2.11-central-2.11.1-8.el7fdp.x86_64
ovn2.11-2.11.1-8.el7fdp.x86_64
[root@dell-per730-42 ovn]# rpm -qa | grep openvswitch
kernel-kernel-networking-openvswitch-ovs_qinq-1.3-34.noarch
openvswitch-selinux-extra-policy-1.0-14.el7fdp.noarch
openvswitch2.11-2.11.0-26.el7fdp.x86_64
[root@dell-per730-42 ovn]# 
[root@dell-per730-42 ovn]# ovn-nbctl show
switch 107877ac-001e-451d-be65-88346a2793eb (s3)
    port hv0_vm00_vnet1
        addresses: ["00:de:ad:00:00:01 172.16.103.11"]
    port s3_r1
        type: router
        addresses: ["00:de:ad:ff:01:03 172.16.103.1"]
        router-port: r1_s3
    port hv0_vm01_vnet1
        addresses: ["00:de:ad:00:01:01 172.16.103.12"]
switch eff2f042-f1dc-412f-867f-b414e0934acb (public)
    port public_r1
        type: router
        router-port: r1_public
    port ln_p1
        type: localnet
        addresses: ["unknown"]
switch b4b3a93e-12f4-429b-ba6e-d2d223241fb6 (s2)
    port hv1_vm01_vnet1
        addresses: ["00:de:ad:01:01:01 172.16.102.12"]
    port s2_r1
        type: router
        addresses: ["00:de:ad:ff:01:02 172.16.102.1"]
        router-port: r1_s2
    port hv1_vm00_vnet1
        addresses: ["00:de:ad:01:00:01 172.16.102.11"]
router a2c242a1-de48-4c0a-ba52-a592642d9767 (r1)
    port r1_s3
        mac: "00:de:ad:ff:01:03"
        networks: ["172.16.103.1/24"]
    port r1_public
        mac: "40:44:00:00:00:03"
        networks: ["172.16.104.1/24"]
    port r1_s2
        mac: "00:de:ad:ff:01:02"
        networks: ["172.16.102.1/24"]
    nat 634324ee-4c29-4df6-a9c2-d4fbc3069982
        external ip: "172.16.104.200"
        logical ip: "172.16.102.11"
        type: "dnat_and_snat"
    nat ef54f72d-14dd-4010-8e63-315fe3b7332f
        external ip: "172.16.104.201"
        logical ip: "172.16.103.11"
        type: "dnat_and_snat"
[root@dell-per730-42 ovn]# 

add qos command with the commands:

 ovn-nbctl set Logical_Switch_Port ln_p1 options:qos_max_rate=1000
   ovn-nbctl set Logical_Switch_Port ln_p1 options:qos_burst=1000

   ovs-vsctl set interface p5p2 external-ids:ovn-egress-iface=true

Then the traffic is low:
[root@dell-per730-42 ovn]# virsh console hv1_vm00
Connected to domain hv1_vm00
Escape character is ^]

[root@localhost ~]# netperf -H 172.16.104.201
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 172.16.104.201 () port 0 AF_INET
Recv   Send    Send                          
Socket Socket  Message  Elapsed              
Size   Size    Size     Time     Throughput  
bytes  bytes   bytes    secs.    10^6bits/sec  

 87380  16384  16384    14.00       0.01   
[root@localhost ~]#

--- Additional comment from errata-xmlrpc on 2019-11-05 12:43:42 UTC ---

Bug report changed to RELEASE_PENDING status by Errata System.
Advisory RHBA-2019:47208-03 has been changed to PUSH_READY status.
https://errata.devel.redhat.com/advisory/47208

--- Additional comment from errata-xmlrpc on 2019-11-06 05:00:08 UTC ---

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:3718

Comment 1 ying xu 2020-02-05 08:17:21 UTC
I found that the issue is the command "ovn-nbctl set Logical_Switch_Port $portname options:qos_max_rate=2000000" doesn't work.

it caused the case ovn_qos fail too.
https://beaker.engineering.redhat.com/recipes/7862225#task105609026
https://beaker.engineering.redhat.com/recipes/7862205#task105608876

Comment 2 ying xu 2020-02-06 02:09:55 UTC
it is ok on last version 20.A.
https://beaker.engineering.redhat.com/recipes/7716240/tasks/103936621/logs/TESTOUT.log

Comment 3 Eelco Chaudron 2020-02-07 09:15:19 UTC
Assigning to OVN team first to figure out if its OVN.

Comment 4 lorenzo bianconi 2020-02-19 14:03:32 UTC
(In reply to ying xu from comment #1)
> I found that the issue is the command "ovn-nbctl set Logical_Switch_Port
> $portname options:qos_max_rate=2000000" doesn't work.
> 
> it caused the case ovn_qos fail too.
> https://beaker.engineering.redhat.com/recipes/7862225#task105609026
> https://beaker.engineering.redhat.com/recipes/7862205#task105608876

Could you please provide the output of the following commands?

$ovn-sbctl show (on the central node)
$ovn-sbctl list port_binding <your localnet lsp> (on the central node)
$tc qdisc show (on the controller node)
$ovs-vsctl list interface <interface connected to the localnet port> (on the controller node)

please remember the shaping is done on the egress traffic

Comment 5 ying xu 2020-02-20 01:44:31 UTC
(In reply to lorenzo bianconi from comment #4)
> (In reply to ying xu from comment #1)
> > I found that the issue is the command "ovn-nbctl set Logical_Switch_Port
> > $portname options:qos_max_rate=2000000" doesn't work.
> > 
> > it caused the case ovn_qos fail too.
> > https://beaker.engineering.redhat.com/recipes/7862225#task105609026
> > https://beaker.engineering.redhat.com/recipes/7862205#task105608876
> 
> Could you please provide the output of the following commands?
> 
> $ovn-sbctl show (on the central node)
# ovn-sbctl show
Chassis "hv1"
    hostname: "dell-per730-03.rhts.eng.pek2.redhat.com"
    Encap geneve
        ip: "20.0.3.25"
        options: {csum="true"}
    Port_Binding "cr-r1_public"
    Port_Binding "hv1_vm00_vnet1"
    Port_Binding "hv1_vm01_vnet1"
Chassis "hv0"
    hostname: "hp-dl388g8-08.rhts.eng.pek2.redhat.com"
    Encap geneve
        ip: "20.0.3.26"
        options: {csum="true"}
    Port_Binding "hv0_vm00_vnet1"
    Port_Binding "hv0_vm01_vnet1"

> $ovn-sbctl list port_binding <your localnet lsp> (on the central node)
# ovn-sbctl list port_binding ln_p1
_uuid               : 40deff79-5199-4aae-9d7f-039cba1f8fdf
chassis             : []
datapath            : c380a7f7-b8c1-41fb-a3bc-8c6bedaa98e5
encap               : []
external_ids        : {}
gateway_chassis     : []
ha_chassis_group    : []
logical_port        : "ln_p1"
mac                 : [unknown]
nat_addresses       : []
options             : {network_name=nattest, qdisc_queue_id="1", qos_burst="2200000", qos_max_rate="2000000"}
parent_port         : []
tag                 : []
tunnel_key          : 1
type                : localnet
virtual_parent      : []


> $tc qdisc show (on the controller node)
# tc qdisc show
qdisc noqueue 0: dev lo root refcnt 2 
qdisc mq 0: dev eno1 root 
qdisc fq_codel 0: dev eno1 parent :1 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
qdisc mq 0: dev eno3 root 
qdisc fq_codel 0: dev eno3 parent :1 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
qdisc mq 0: dev eno4 root 
qdisc fq_codel 0: dev eno4 parent :1 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
qdisc noqueue 0: dev virbr0 root refcnt 2 
qdisc fq_codel 0: dev virbr0-nic root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
qdisc noqueue 0: dev genev_sys_6081 root refcnt 2 
qdisc fq_codel 0: dev vnet0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
qdisc fq_codel 0: dev vnet1 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
qdisc fq_codel 0: dev hv1_vm00_vnet1 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
qdisc fq_codel 0: dev hv1_vm01_vnet1 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
qdisc noqueue 0: dev nat_test root refcnt 2 


> $ovs-vsctl list interface <interface connected to the localnet port> (on the
> controller node)

_uuid               : 1854e06d-7e7b-4763-b1d3-8282b9507639
admin_state         : up
bfd                 : {}
bfd_status          : {}
cfm_fault           : []
cfm_fault_status    : []
cfm_flap_count      : []
cfm_health          : []
cfm_mpid            : []
cfm_remote_mpids    : []
cfm_remote_opstate  : []
duplex              : full
error               : []
external_ids        : {ovn-egress-iface="true"}
ifindex             : 5
ingress_policing_burst: 0
ingress_policing_rate: 0
lacp_current        : []
link_resets         : 1
link_speed          : 1000000000
link_state          : up
lldp                : {}
mac                 : []
mac_in_use          : "44:a8:42:49:41:f7"
mtu                 : 1500
mtu_request         : []
name                : "eno4"
ofport              : 1
ofport_request      : []
options             : {}
other_config        : {}
statistics          : {collisions=0, rx_bytes=475336, rx_crc_err=0, rx_dropped=0, rx_errors=0, rx_frame_err=0, rx_over_err=0, rx_packets=5149, tx_bytes=1720, tx_dropped=0, tx_errors=0, tx_packets=22}
status              : {driver_name="tg3", driver_version="3.137", firmware_version="FFV7.10.59 bc 5720-v1.36"}
type                : ""


> 
> please remember the shaping is done on the egress traffic

Comment 6 ying xu 2020-02-20 03:59:13 UTC
and the output of the shaping:
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 172.16.104.201 () port 0 AF_INET
Recv   Send    Send                          
Socket Socket  Message  Elapsed              
Size   Size    Size     Time     Throughput  
bytes  bytes   bytes    secs.    10^6bits/sec  

 87380  16384  16384    10.03     940.82

Comment 7 ying xu 2020-02-24 02:23:56 UTC
comment 5 and 6 clears the needinfo

Comment 8 lorenzo bianconi 2020-02-24 17:19:18 UTC
I tries ovn2.11 FDP20.B on the setup I used to develop the feature and it works as expected. Could you please help me setting up the environment you are running? I will install ovn/ovs from source to continue debugging the issue. Thanks.

Comment 9 lorenzo bianconi 2020-02-25 17:28:48 UTC
(In reply to ying xu from comment #5)
> (In reply to lorenzo bianconi from comment #4)
> > (In reply to ying xu from comment #1)
> > > I found that the issue is the command "ovn-nbctl set Logical_Switch_Port
> > > $portname options:qos_max_rate=2000000" doesn't work.
> > > 
> > > it caused the case ovn_qos fail too.
> > > https://beaker.engineering.redhat.com/recipes/7862225#task105609026
> > > https://beaker.engineering.redhat.com/recipes/7862205#task105608876
> > 
> > Could you please provide the output of the following commands?
> > 
> > $ovn-sbctl show (on the central node)
> # ovn-sbctl show
> Chassis "hv1"
>     hostname: "dell-per730-03.rhts.eng.pek2.redhat.com"
>     Encap geneve
>         ip: "20.0.3.25"
>         options: {csum="true"}
>     Port_Binding "cr-r1_public"
>     Port_Binding "hv1_vm00_vnet1"
>     Port_Binding "hv1_vm01_vnet1"
> Chassis "hv0"
>     hostname: "hp-dl388g8-08.rhts.eng.pek2.redhat.com"
>     Encap geneve
>         ip: "20.0.3.26"
>         options: {csum="true"}
>     Port_Binding "hv0_vm00_vnet1"
>     Port_Binding "hv0_vm01_vnet1"
> 
> > $ovn-sbctl list port_binding <your localnet lsp> (on the central node)
> # ovn-sbctl list port_binding ln_p1
> _uuid               : 40deff79-5199-4aae-9d7f-039cba1f8fdf
> chassis             : []
> datapath            : c380a7f7-b8c1-41fb-a3bc-8c6bedaa98e5
> encap               : []
> external_ids        : {}
> gateway_chassis     : []
> ha_chassis_group    : []
> logical_port        : "ln_p1"
> mac                 : [unknown]
> nat_addresses       : []
> options             : {network_name=nattest, qdisc_queue_id="1",
> qos_burst="2200000", qos_max_rate="2000000"}
> parent_port         : []
> tag                 : []
> tunnel_key          : 1
> type                : localnet
> virtual_parent      : []
> 
> 
> > $tc qdisc show (on the controller node)
> # tc qdisc show
> qdisc noqueue 0: dev lo root refcnt 2 
> qdisc mq 0: dev eno1 root 
> qdisc fq_codel 0: dev eno1 parent :1 limit 10240p flows 1024 quantum 1514
> target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
> qdisc mq 0: dev eno3 root 
> qdisc fq_codel 0: dev eno3 parent :1 limit 10240p flows 1024 quantum 1514
> target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
> qdisc mq 0: dev eno4 root 
> qdisc fq_codel 0: dev eno4 parent :1 limit 10240p flows 1024 quantum 1514
> target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
> qdisc noqueue 0: dev virbr0 root refcnt 2 
> qdisc fq_codel 0: dev virbr0-nic root refcnt 2 limit 10240p flows 1024
> quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
> qdisc noqueue 0: dev genev_sys_6081 root refcnt 2 
> qdisc fq_codel 0: dev vnet0 root refcnt 2 limit 10240p flows 1024 quantum
> 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
> qdisc fq_codel 0: dev vnet1 root refcnt 2 limit 10240p flows 1024 quantum
> 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
> qdisc fq_codel 0: dev hv1_vm00_vnet1 root refcnt 2 limit 10240p flows 1024
> quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
> qdisc fq_codel 0: dev hv1_vm01_vnet1 root refcnt 2 limit 10240p flows 1024
> quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
> qdisc noqueue 0: dev nat_test root refcnt 2 
> 
> 

Debugging the issue on your environment it does not seem an OVN issue since OVS fails to install the htb qdisc. Could you please try to remove the qdisc before running the test doing something like:

$tc qdisc del dev eno4 root

Are you running some test before this one?

> > $ovs-vsctl list interface <interface connected to the localnet port> (on the
> > controller node)
> 
> _uuid               : 1854e06d-7e7b-4763-b1d3-8282b9507639
> admin_state         : up
> bfd                 : {}
> bfd_status          : {}
> cfm_fault           : []
> cfm_fault_status    : []
> cfm_flap_count      : []
> cfm_health          : []
> cfm_mpid            : []
> cfm_remote_mpids    : []
> cfm_remote_opstate  : []
> duplex              : full
> error               : []
> external_ids        : {ovn-egress-iface="true"}
> ifindex             : 5
> ingress_policing_burst: 0
> ingress_policing_rate: 0
> lacp_current        : []
> link_resets         : 1
> link_speed          : 1000000000
> link_state          : up
> lldp                : {}
> mac                 : []
> mac_in_use          : "44:a8:42:49:41:f7"
> mtu                 : 1500
> mtu_request         : []
> name                : "eno4"
> ofport              : 1
> ofport_request      : []
> options             : {}
> other_config        : {}
> statistics          : {collisions=0, rx_bytes=475336, rx_crc_err=0,
> rx_dropped=0, rx_errors=0, rx_frame_err=0, rx_over_err=0, rx_packets=5149,
> tx_bytes=1720, tx_dropped=0, tx_errors=0, tx_packets=22}
> status              : {driver_name="tg3", driver_version="3.137",
> firmware_version="FFV7.10.59 bc 5720-v1.36"}
> type                : ""
> 
> 
> > 
> > please remember the shaping is done on the egress traffic

Comment 10 ying xu 2020-02-26 06:31:23 UTC
(In reply to lorenzo bianconi from comment #9)
> (In reply to ying xu from comment #5)
> > (In reply to lorenzo bianconi from comment #4)
> > > (In reply to ying xu from comment #1)
> > > > I found that the issue is the command "ovn-nbctl set Logical_Switch_Port
> > > > $portname options:qos_max_rate=2000000" doesn't work.
> > > > 
> > > > it caused the case ovn_qos fail too.
> > > > https://beaker.engineering.redhat.com/recipes/7862225#task105609026
> > > > https://beaker.engineering.redhat.com/recipes/7862205#task105608876
> > > 
> > > Could you please provide the output of the following commands?
> > > 
> > > $ovn-sbctl show (on the central node)
> > # ovn-sbctl show
> > Chassis "hv1"
> >     hostname: "dell-per730-03.rhts.eng.pek2.redhat.com"
> >     Encap geneve
> >         ip: "20.0.3.25"
> >         options: {csum="true"}
> >     Port_Binding "cr-r1_public"
> >     Port_Binding "hv1_vm00_vnet1"
> >     Port_Binding "hv1_vm01_vnet1"
> > Chassis "hv0"
> >     hostname: "hp-dl388g8-08.rhts.eng.pek2.redhat.com"
> >     Encap geneve
> >         ip: "20.0.3.26"
> >         options: {csum="true"}
> >     Port_Binding "hv0_vm00_vnet1"
> >     Port_Binding "hv0_vm01_vnet1"
> > 
> > > $ovn-sbctl list port_binding <your localnet lsp> (on the central node)
> > # ovn-sbctl list port_binding ln_p1
> > _uuid               : 40deff79-5199-4aae-9d7f-039cba1f8fdf
> > chassis             : []
> > datapath            : c380a7f7-b8c1-41fb-a3bc-8c6bedaa98e5
> > encap               : []
> > external_ids        : {}
> > gateway_chassis     : []
> > ha_chassis_group    : []
> > logical_port        : "ln_p1"
> > mac                 : [unknown]
> > nat_addresses       : []
> > options             : {network_name=nattest, qdisc_queue_id="1",
> > qos_burst="2200000", qos_max_rate="2000000"}
> > parent_port         : []
> > tag                 : []
> > tunnel_key          : 1
> > type                : localnet
> > virtual_parent      : []
> > 
> > 
> > > $tc qdisc show (on the controller node)
> > # tc qdisc show
> > qdisc noqueue 0: dev lo root refcnt 2 
> > qdisc mq 0: dev eno1 root 
> > qdisc fq_codel 0: dev eno1 parent :1 limit 10240p flows 1024 quantum 1514
> > target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
> > qdisc mq 0: dev eno3 root 
> > qdisc fq_codel 0: dev eno3 parent :1 limit 10240p flows 1024 quantum 1514
> > target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
> > qdisc mq 0: dev eno4 root 
> > qdisc fq_codel 0: dev eno4 parent :1 limit 10240p flows 1024 quantum 1514
> > target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
> > qdisc noqueue 0: dev virbr0 root refcnt 2 
> > qdisc fq_codel 0: dev virbr0-nic root refcnt 2 limit 10240p flows 1024
> > quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
> > qdisc noqueue 0: dev genev_sys_6081 root refcnt 2 
> > qdisc fq_codel 0: dev vnet0 root refcnt 2 limit 10240p flows 1024 quantum
> > 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
> > qdisc fq_codel 0: dev vnet1 root refcnt 2 limit 10240p flows 1024 quantum
> > 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
> > qdisc fq_codel 0: dev hv1_vm00_vnet1 root refcnt 2 limit 10240p flows 1024
> > quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
> > qdisc fq_codel 0: dev hv1_vm01_vnet1 root refcnt 2 limit 10240p flows 1024
> > quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
> > qdisc noqueue 0: dev nat_test root refcnt 2 
> > 
> > 
> 
> Debugging the issue on your environment it does not seem an OVN issue since
> OVS fails to install the htb qdisc. Could you please try to remove the qdisc
> before running the test doing something like:
> 
> $tc qdisc del dev eno4 root
> 
> Are you running some test before this one?

I reproduce it with your suggestion,but it fails too. and I think it is not affected by other case. I will empty the environment before my case every time.

and I run the case on version 20.A to compare with.It is pass. as what you said, it didn't set up the htb qdisc. And it did on version 20.A.

ovn2.11.1-33

[root@ibm-x3650m4-04 qos]# tc qdisc show dev enp17s0f1
qdisc mq 0: root 
qdisc fq_codel 0: parent :1 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
[root@ibm-x3650m4-04 qos]# tc qdisc show dev enp17s0f0
qdisc mq 0: root 
qdisc fq_codel 0: parent :1 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn 


ovn2.11.1-24

[root@ibm-x3650m4-04 qos]# tc qdisc show dev enp17s0f0
qdisc htb 1: root refcnt 6 r2q 10 default 1 direct_packets_stat 52 direct_qlen 1000
[root@ibm-x3650m4-04 qos]# tc qdisc show dev enp17s0f1
qdisc htb 1: root refcnt 6 r2q 10 default 1 direct_packets_stat 209 direct_qlen 1000


Maybe it is not a bug of ovn, but there may be something wrong with ovs.

> 
> > > $ovs-vsctl list interface <interface connected to the localnet port> (on the
> > > controller node)
> > 
> > _uuid               : 1854e06d-7e7b-4763-b1d3-8282b9507639
> > admin_state         : up
> > bfd                 : {}
> > bfd_status          : {}
> > cfm_fault           : []
> > cfm_fault_status    : []
> > cfm_flap_count      : []
> > cfm_health          : []
> > cfm_mpid            : []
> > cfm_remote_mpids    : []
> > cfm_remote_opstate  : []
> > duplex              : full
> > error               : []
> > external_ids        : {ovn-egress-iface="true"}
> > ifindex             : 5
> > ingress_policing_burst: 0
> > ingress_policing_rate: 0
> > lacp_current        : []
> > link_resets         : 1
> > link_speed          : 1000000000
> > link_state          : up
> > lldp                : {}
> > mac                 : []
> > mac_in_use          : "44:a8:42:49:41:f7"
> > mtu                 : 1500
> > mtu_request         : []
> > name                : "eno4"
> > ofport              : 1
> > ofport_request      : []
> > options             : {}
> > other_config        : {}
> > statistics          : {collisions=0, rx_bytes=475336, rx_crc_err=0,
> > rx_dropped=0, rx_errors=0, rx_frame_err=0, rx_over_err=0, rx_packets=5149,
> > tx_bytes=1720, tx_dropped=0, tx_errors=0, tx_packets=22}
> > status              : {driver_name="tg3", driver_version="3.137",
> > firmware_version="FFV7.10.59 bc 5720-v1.36"}
> > type                : ""
> > 
> > 
> > > 
> > > please remember the shaping is done on the egress traffic

Comment 11 ying xu 2020-02-26 07:01:12 UTC
I did the test on version as below: ovn of 20.A, ovs of 20.B
# rpm -qa|grep ovn
ovn2.11-2.11.1-24.el8fdp.x86_64
ovn2.11-host-2.11.1-24.el8fdp.x86_64
ovn2.11-central-2.11.1-24.el8fdp.x86_64
# rpm -qa|grep openvs
openvswitch2.11-2.11.0-47.el8fdp.x86_64
openvswitch-selinux-extra-policy-1.0-19.el8fdp.noarch
python3-openvswitch2.11-2.11.0-47.el8fdp.x86_64


it works!


does it indicate that it is an issue of ovn? Maybe there is a problem from ovn to ovs,not ovs to htb

Comment 19 ying xu 2020-03-13 04:18:43 UTC
this issue can be reproduce on version 20.B,
https://beaker.engineering.redhat.com/jobs/4054701


and verified on version as below:
# rpm -qa|grep ovn
ovn2.11-2.11.1-37.el7fdp.x86_64
ovn2.11-central-2.11.1-37.el7fdp.x86_64
ovn2.11-host-2.11.1-37.el7fdp.x86_64

the reproducer :/kernel/networking/openvswitch/ovn/basic:ovn_test_qos

{print $5}')ost ~]# export tcp_speed4=$(cat perf_result_tcp_4 | tail -n1 | awk ' 
[root@localhost ~]# echo $?
0
[root@localhost ~]# echo tcp_speed4=$tcp_speed4
tcp_speed4=7163.54
[root@localhost ~]# echo $?
0
[root@localhost ~]# export tcp_speed4=${tcp_speed4%.*}
[root@localhost ~]# echo $?
0
[root@localhost ~]# expr $tcp_speed4 \> 3000
1
[root@localhost ~]# echo $?
0
[root@localhost ~]# logout

Red Hat Enterprise Linux Server 7.7 (Maipo)
Kernel 3.10.0-1062.el7.x86_64 on an x86_64

localhost login: 
spawn virsh console hv1_vm00
Connected to domain hv1_vm00
Escape character is ^]

Red Hat Enterprise Linux Server 7.7 (Maipo)
Kernel 3.10.0-1062.el7.x86_64 on an x86_64

localhost login: root
Password: 
Last login: Thu Mar 12 23:49:50 on ttyS0
REAM -T1 > perf_result_tcp_6-6 -L 2001:db8:102::11 -H 2001:db8:102::21 -t TCP_ST 
[root@localhost ~]# echo $?
0
{print $5}')ost ~]# export tcp_speed6=$(cat perf_result_tcp_6 | tail -n1 | awk ' 
[root@localhost ~]# echo $?
0
[root@localhost ~]# echo tcp_speed6=$tcp_speed6
tcp_speed6=7860.61
[root@localhost ~]# echo $?
0
[root@localhost ~]# export tcp_speed6=${tcp_speed6%.*}
[root@localhost ~]# echo $?
0
[root@localhost ~]# expr $tcp_speed6 \> 3000
1
[root@localhost ~]# echo $?
0
[root@localhost ~]# logout

Red Hat Enterprise Linux Server 7.7 (Maipo)
Kernel 3.10.0-1062.el7.x86_64 on an x86_64

localhost login: 
result=0
spawn virsh console hv1_vm00
Connected to domain hv1_vm00
Escape character is ^]

Red Hat Enterprise Linux Server 7.7 (Maipo)
Kernel 3.10.0-1062.el7.x86_64 on an x86_64

localhost login: root
Password: 
Last login: Thu Mar 12 23:50:04 on ttyS0
tcp_4@localhost ~]# netperf -4 -H 172.16.102.21 -t TCP_STREAM -T1 > perf_result_ 

[root@localhost ~]# echo $?
0
{print $5}')ost ~]# export tcp_speed4=$(cat perf_result_tcp_4 | tail -n1 | awk ' 
[root@localhost ~]# echo $?
0
[root@localhost ~]# echo tcp_speed4=$tcp_speed4
tcp_speed4=9.29
[root@localhost ~]# echo $?
0
[root@localhost ~]# export tcp_speed4=${tcp_speed4%.*}
[root@localhost ~]# echo $?
0
[root@localhost ~]# expr 15 \> $tcp_speed4 && expr $tcp_speed4 \> 5
1
1
[root@localhost ~]# echo $?
0
[root@localhost ~]# logout

Red Hat Enterprise Linux Server 7.7 (Maipo)
Kernel 3.10.0-1062.el7.x86_64 on an x86_64

localhost login: 
spawn virsh console hv1_vm00
Connected to domain hv1_vm00
Escape character is ^]

Red Hat Enterprise Linux Server 7.7 (Maipo)
Kernel 3.10.0-1062.el7.x86_64 on an x86_64

localhost login: root
Password: 
Last login: Thu Mar 12 23:50:18 on ttyS0
REAM -T1 > perf_result_tcp_6-6 -L 2001:db8:102::11 -H 2001:db8:102::21 -t TCP_ST 
[root@localhost ~]# echo $?
0
{print $5}')ost ~]# export tcp_speed6=$(cat perf_result_tcp_6 | tail -n1 | awk ' 
[root@localhost ~]# echo $?
0
[root@localhost ~]# echo tcp_speed6=$tcp_speed6
tcp_speed6=9.21
[root@localhost ~]# echo $?
0
[root@localhost ~]# export tcp_speed6=${tcp_speed6%.*}
[root@localhost ~]# echo $?
0
[root@localhost ~]# expr 15 \> $tcp_speed6 && expr $tcp_speed6 \> 5
1
1
[root@localhost ~]# echo $?
0
[root@localhost ~]# logout

Red Hat Enterprise Linux Server 7.7 (Maipo)
Kernel 3.10.0-1062.el7.x86_64 on an x86_64

localhost login: 
result=0
SYNC_NC: sync_set client ovn_qos
SYNC_NC: sent "ovn_qos" to dell-per730-57.rhts.eng.pek2.redhat.com
ovn_cleanup ...
ovn_cleanup ... end
:: [ 23:50:57 ] :: [   PASS   ] :: Command 'ovn_qos' (Expected 0, got 0)

Comment 20 ying xu 2020-03-17 12:21:39 UTC
hi,lorenzo bianconi

sorry I made a mistake about this bug.
I verified this bug with the case /kernel/networking/openvswitch/ovn/qos
https://beaker.engineering.redhat.com/jobs/4139193(case:qos pass)

but for the case /kernel/networking/openvswitch/ovn/basic:ovn_test_qos, it is failed.
https://beaker.engineering.redhat.com/jobs/4139194 (case:ovn_qos failed)

so I reset the status to ON_QA. I am very sorry about that.


I am not sure whether it is the same issue as before.Can you have a look at it?

Comment 25 ying xu 2020-03-19 10:01:34 UTC
I talked with devel, the secnario of tunnel qos works, so set this bug verified 

verified on version:
# rpm -qa|grep ovn
ovn2.11-2.11.1-37.el7fdp.x86_64
ovn2.11-host-2.11.1-37.el7fdp.x86_64
ovn2.11-central-2.11.1-37.el7fdp.x86_64

[root@localhost ~]# cat log.txt
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 172.16.104.201 () port 0 AF_INET
Recv   Send    Send                          
Socket Socket  Message  Elapsed              
Size   Size    Size     Time     Throughput  
bytes  bytes   bytes    secs.    10^6bits/sec  

 87380  16384  16384    14.00       2.06   
[root@localhost ~]# echo $?
0
[root@localhost ~]# logout

Red Hat Enterprise Linux Server 7.7 (Maipo)
Kernel 3.10.0-1062.el7.x86_64 on an x86_64

localhost login: 
spawn virsh console hv1_vm00
Connected to domain hv1_vm00
Escape character is ^]

Red Hat Enterprise Linux Server 7.7 (Maipo)
Kernel 3.10.0-1062.el7.x86_64 on an x86_64

localhost login: root
Password: 
Last login: Thu Mar 19 05:52:00 on ttyS0
[root@localhost ~]# tail -n 1 log.txt|grep 2.*
 87380  16384  16384    14.00       2.06   
[root@localhost ~]# echo $?
0
[root@localhost ~]# logout

Red Hat Enterprise Linux Server 7.7 (Maipo)
Kernel 3.10.0-1062.el7.x86_64 on an x86_64

localhost login: 
spawn virsh console hv1_vm00
Connected to domain hv1_vm00
Escape character is ^]

Red Hat Enterprise Linux Server 7.7 (Maipo)
Kernel 3.10.0-1062.el7.x86_64 on an x86_64

localhost login: root
Password: 
Last login: Thu Mar 19 05:52:04 on ttyS0
[root@localhost ~]# netperf -H 172.16.104.201 > log.txt
[root@localhost ~]# echo $?
0
[root@localhost ~]# logout

Red Hat Enterprise Linux Server 7.7 (Maipo)
Kernel 3.10.0-1062.el7.x86_64 on an x86_64

localhost login: 
spawn virsh console hv1_vm00
Connected to domain hv1_vm00
Escape character is ^]

Red Hat Enterprise Linux Server 7.7 (Maipo)
Kernel 3.10.0-1062.el7.x86_64 on an x86_64

localhost login: root
Password: 
Last login: Thu Mar 19 05:52:07 on ttyS0
[root@localhost ~]# cat log.txt
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 172.16.104.201 () port 0 AF_INET
Recv   Send    Send                          
Socket Socket  Message  Elapsed              
Size   Size    Size     Time     Throughput  
bytes  bytes   bytes    secs.    10^6bits/sec  

 87380  16384  16384    14.00       0.02   
[root@localhost ~]# echo $?
0
[root@localhost ~]# logout

Red Hat Enterprise Linux Server 7.7 (Maipo)
Kernel 3.10.0-1062.el7.x86_64 on an x86_64

localhost login: 
spawn virsh console hv1_vm00
Connected to domain hv1_vm00
Escape character is ^]

Red Hat Enterprise Linux Server 7.7 (Maipo)
Kernel 3.10.0-1062.el7.x86_64 on an x86_64

localhost login: root
Password: 
Last login: Thu Mar 19 05:52:43 on ttyS0
[root@localhost ~]# tail -n 1 log.txt|grep 0.02
 87380  16384  16384    14.00       0.02   
[root@localhost ~]# echo $?
0
[root@localhost ~]# logout

Red Hat Enterprise Linux Server 7.7 (Maipo)
Kernel 3.10.0-1062.el7.x86_64 on an x86_64

localhost login: 
spawn virsh console hv1_vm00
Connected to domain hv1_vm00
Escape character is ^]

Red Hat Enterprise Linux Server 7.7 (Maipo)
Kernel 3.10.0-1062.el7.x86_64 on an x86_64

localhost login: root
Password: 
Last login: Thu Mar 19 05:52:47 on ttyS0
[root@localhost ~]# netperf -H 172.16.104.201 > log.txt
[root@localhost ~]# echo $?
0
[root@localhost ~]# logout

Red Hat Enterprise Linux Server 7.7 (Maipo)
Kernel 3.10.0-1062.el7.x86_64 on an x86_64

localhost login: 
spawn virsh console hv1_vm00
Connected to domain hv1_vm00
Escape character is ^]

Red Hat Enterprise Linux Server 7.7 (Maipo)
Kernel 3.10.0-1062.el7.x86_64 on an x86_64

localhost login: root
Password: 
Last login: Thu Mar 19 05:52:50 on ttyS0
[root@localhost ~]# cat log.txt
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 172.16.104.201 () port 0 AF_INET
Recv   Send    Send                          
Socket Socket  Message  Elapsed              
Size   Size    Size     Time     Throughput  
bytes  bytes   bytes    secs.    10^6bits/sec  

 87380  16384  16384    10.13     212.03   
[root@localhost ~]# echo $?
0
[root@localhost ~]# logout

Red Hat Enterprise Linux Server 7.7 (Maipo)
Kernel 3.10.0-1062.el7.x86_64 on an x86_64

localhost login: 
spawn virsh console hv1_vm00
Connected to domain hv1_vm00
Escape character is ^]

Red Hat Enterprise Linux Server 7.7 (Maipo)
Kernel 3.10.0-1062.el7.x86_64 on an x86_64

localhost login: root
Password: 
Last login: Thu Mar 19 05:53:04 on ttyS0
[root@localhost ~]# tail -n 1 log.txt|awk '{print $5}'|grep "^2..\."
212.03
[root@localhost ~]# echo $?
0
[root@localhost ~]# logout

Red Hat Enterprise Linux Server 7.7 (Maipo)
Kernel 3.10.0-1062.el7.x86_64 on an x86_64

localhost login: 
spawn virsh console hv1_vm00
Connected to domain hv1_vm00
Escape character is ^]

Red Hat Enterprise Linux Server 7.7 (Maipo)
Kernel 3.10.0-1062.el7.x86_64 on an x86_64

localhost login: root
Password: 
Last login: Thu Mar 19 05:53:07 on ttyS0
[root@localhost ~]# netperf -H 172.16.104.201 > log.txt
[root@localhost ~]# echo $?
0
[root@localhost ~]# logout

Red Hat Enterprise Linux Server 7.7 (Maipo)
Kernel 3.10.0-1062.el7.x86_64 on an x86_64

localhost login: 
spawn virsh console hv1_vm00
Connected to domain hv1_vm00
Escape character is ^]

Red Hat Enterprise Linux Server 7.7 (Maipo)
Kernel 3.10.0-1062.el7.x86_64 on an x86_64

localhost login: root
Password: 
Last login: Thu Mar 19 05:53:11 on ttyS0
[root@localhost ~]# cat log.txt
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 172.16.104.201 () port 0 AF_INET
Recv   Send    Send                          
Socket Socket  Message  Elapsed              
Size   Size    Size     Time     Throughput  
bytes  bytes   bytes    secs.    10^6bits/sec  

 87380  16384  16384    10.03     941.33   
[root@localhost ~]# echo $?
0
[root@localhost ~]# logout

Red Hat Enterprise Linux Server 7.7 (Maipo)
Kernel 3.10.0-1062.el7.x86_64 on an x86_64

localhost login: 
result=0
SYNC_NC: sync_set client ovn_floating_qos
SYNC_NC: sent "ovn_floating_qos" to ibm-x3650m4-04.rhts.eng.pek2.redhat.com
ovn_cleanup ...
ovn_cleanup ... end
:: [ 05:53:35 ] :: [   PASS   ] :: Command 'ovn_floating_qos' (Expected 0, got 0)



and for the fail case it looks like a different issue from this one, so I will open a new one.

Comment 27 ying xu 2020-03-19 10:31:14 UTC
the scenarios of DVR and tunnel interface all work. so set it to verified.

Comment 28 ying xu 2020-03-30 01:49:38 UTC
clear the needinfo according to comment27

Comment 30 errata-xmlrpc 2020-04-14 14:29:49 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:1458