The goal of this RFE is to add QoS minimum bandwidth guarantee rule support in core OVN. This new functionality should close the parity gap between Neutron ML2/OVS and ML2/OVN. The scope of this RFE is limited to ports connected to physical networks (flat, VLAN); that means, VM ports that will egress traffic through a physical bridge. Neutron ML2/OVS =============== Now in ML2/OVS we provide minimum bandwidth only for those ports connected to physical networks (flat, VLAN). The QoS min-BW rules for a VM port are set in the physical bridge port where the VM port traffic leaves the node. The rationale for this is that the min-BW rules pretend to guarantee a certain BW for the VM traffic where the traffic is aggregated; this is in the physical port interface. [1] is the ML2/OVS implementation. In ML2/OVS, the physical bridge interface port has a QoS register associated. This QoS register has several Queue registers, one per VM port. Each Queue register has a specific "queue-num" associated and the corresponding "other-config:min-rate" value. Proposal ======== This RFE proposes to add a new parameter in DB NB "QoS.bandwdth" enumerate: ["rate", "burst", "minimum"]. The "match", "direction" and "priority" columns should work as now. In order to identify the egress port (physical bridge port), this RFE proposed to add a reference in "external_ids" (this is just an idea that could wrong due to my lack of knowledge in OVN internals). Please, for more information you can reach me in IRC (ralonsoh), mail and this BZ. Regards and thank you in advance. [1]https://github.com/openstack/neutron/blob/b072cbf05f079519e379b2bfebe27846ae612275/neutron/plugins/ml2/drivers/openvswitch/agent/extension_drivers/qos_driver.py#L179-L190
One important note is that, at the moment OVN is implementing QoS via netdev [0] and not using OVS meters when the parameters are applied to the LSP.options column [1] (instead of to the QoS table). If we want HW offload, since OVS meters are not yet supported we could take a similar approach using either netdev/iproute to allow for HW offload? [0] https://github.com/ovn-org/ovn/blob/5f7a985de0707539e4df1eab4c4150b24dbce622/controller/binding.c#L221 [1] https://github.com/ovn-org/ovn/blob/b988d5fa62d9dbecacb2c45defab8e7edde4c533/ovn-nb.xml#L1031-L1039
(In reply to Daniel Alvarez Sanchez from comment #1) > One important note is that, at the moment OVN is implementing QoS via netdev > [0] and not using OVS meters when the parameters are applied to the > LSP.options column [1] (instead of to the QoS table). > > If we want HW offload, since OVS meters are not yet supported we could take > a similar approach using either netdev/iproute to allow for HW offload? > > > [0] > https://github.com/ovn-org/ovn/blob/5f7a985de0707539e4df1eab4c4150b24dbce622/ > controller/binding.c#L221 > [1] > https://github.com/ovn-org/ovn/blob/b988d5fa62d9dbecacb2c45defab8e7edde4c533/ > ovn-nb.xml#L1031-L1039 This series should give us hw offload for ovs metering: https://patchwork.ozlabs.org/project/openvswitch/list/?series=293970
Minimalistic / backportable / no OVS metering implementation: https://patchwork.ozlabs.org/project/ovn/list/?series=297048
This is going to be included in upstream 22.06 release.
ovn22.03 fast-datapath-rhel-9 clone created at https://bugzilla.redhat.com/show_bug.cgi?id=2115331
# rpm -qa|grep ovn ovn22.03-central-22.03.0-95.el8fdp.x86_64 ovn22.03-host-22.03.0-95.el8fdp.x86_64 ovn22.03-22.03.0-95.el8fdp.x86_64 # ovn-nbctl show switch 4e657969-0058-4761-943a-477fee48981c (s3) port hv0_vm00_vnet1 addresses: ["00:de:ad:00:00:01 172.16.103.11"] port hv0_vm01_vnet1 addresses: ["00:de:ad:00:01:01 172.16.103.12"] port s3_r1 type: router addresses: ["00:de:ad:ff:01:03 172.16.103.1"] router-port: r1_s3 switch b69e3e01-6850-497b-8c73-b71cc9397b85 (public) port public_r1 type: router router-port: r1_public port ln_p1 type: localnet addresses: ["unknown"] switch 16343f82-4ea6-4fc5-8dcd-564b098fba29 (s2) port hv1_vm00_vnet1 addresses: ["00:de:ad:01:00:01 172.16.102.11"] port hv1_vm01_vnet1 addresses: ["00:de:ad:01:01:01 172.16.102.12"] port s2_r1 type: router addresses: ["00:de:ad:ff:01:02 172.16.102.1"] router-port: r1_s2 router e812e570-59d1-4260-b6a0-04738fb7e3d3 (r1) port r1_s2 mac: "00:de:ad:ff:01:02" networks: ["172.16.102.1/24"] port r1_s3 mac: "00:de:ad:ff:01:03" networks: ["172.16.103.1/24"] port r1_public mac: "40:44:00:00:00:03" networks: ["172.16.104.1/24"] gateway chassis: [hv1] nat 299e7e7e-f916-447e-9d05-32d2a6ad99a0 external ip: "172.16.104.201" logical ip: "172.16.103.11" type: "dnat_and_snat" nat 6d45b21e-f5c5-4ec4-b3ec-2dc3575acb91 external ip: "172.16.104.200" logical ip: "172.16.102.11" type: "dnat_and_snat" :: [ 23:16:59 ] :: [ BEGIN ] :: Running 'ovn-nbctl set Logical_Switch_Port ln_p1 options:qos_max_rate=2000000' :: [ 23:16:59 ] :: [ PASS ] :: Command 'ovn-nbctl set Logical_Switch_Port ln_p1 options:qos_max_rate=2000000' (Expected 0, got 0) :: [ 23:16:59 ] :: [ BEGIN ] :: Running 'ovn-nbctl set Logical_Switch_Port ln_p1 options:qos_min_rate=1000000' :: [ 23:16:59 ] :: [ PASS ] :: Command 'ovn-nbctl set Logical_Switch_Port ln_p1 options:qos_min_rate=1000000' (Expected 0, got 0) :: [ 23:16:59 ] :: [ BEGIN ] :: Running 'ovn-nbctl set Logical_Switch_Port ln_p1 options:qos_burst=2200000' :: [ 23:16:59 ] :: [ PASS ] :: Command 'ovn-nbctl set Logical_Switch_Port ln_p1 options:qos_burst=2200000' (Expected 0, got 0) :: [ 23:16:59 ] :: [ BEGIN ] :: Running 'tc qdisc show | grep htb' qdisc htb 1: dev ens3f0 root refcnt 65 r2q 10 default 0x1 direct_packets_stat 2 direct_qlen 1000 qdisc htb 1: dev ens3f1 root refcnt 65 r2q 10 default 0x1 direct_packets_stat 0 direct_qlen 1000 :: [ 23:16:59 ] :: [ PASS ] :: Command 'tc qdisc show | grep htb' (Expected 0, got 0) :: [ 23:16:59 ] :: [ BEGIN ] :: Running 'tc class show dev ens3f1' class htb 1:fffe root rate 10Gbit ceil 10Gbit burst 1250b cburst 1250b class htb 1:2 parent 1:fffe prio 0 rate 1Mbit ceil 2Mbit burst 275000b cburst 275000b ---------------------------set success :: [ 23:16:59 ] :: [ PASS ] :: Command 'tc class show dev ens3f1' (Expected 0, got 0) MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 172.16.104.201 () port 0 AF_INET Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. 10^6bits/sec 87380 16384 16384 14.00 2.06 --------------traffic shape change qos set :: [ 23:17:26 ] :: [ BEGIN ] :: Running 'ovn-nbctl set Logical_Switch_Port ln_p1 options:qos_max_rate=20000000' :: [ 23:17:26 ] :: [ PASS ] :: Command 'ovn-nbctl set Logical_Switch_Port ln_p1 options:qos_max_rate=20000000' (Expected 0, got 0) :: [ 23:17:26 ] :: [ BEGIN ] :: Running 'ovn-nbctl set Logical_Switch_Port ln_p1 options:qos_burst=22000000' :: [ 23:17:26 ] :: [ PASS ] :: Command 'ovn-nbctl set Logical_Switch_Port ln_p1 options:qos_burst=22000000' (Expected 0, got 0) :: [ 23:17:26 ] :: [ BEGIN ] :: Running 'ovn-nbctl set Logical_Switch_Port ln_p1 options:qos_min_rate=10000000' :: [ 23:17:26 ] :: [ PASS ] :: Command 'ovn-nbctl set Logical_Switch_Port ln_p1 options:qos_min_rate=10000000' (Expected 0, got 0) :: [ 23:17:26 ] :: [ BEGIN ] :: Running 'tc qdisc show | grep htb' qdisc htb 1: dev ens3f0 root refcnt 65 r2q 10 default 0x1 direct_packets_stat 15 direct_qlen 1000 qdisc htb 1: dev ens3f1 root refcnt 65 r2q 10 default 0x1 direct_packets_stat 7 direct_qlen 1000 :: [ 23:17:26 ] :: [ PASS ] :: Command 'tc qdisc show | grep htb' (Expected 0, got 0) :: [ 23:17:27 ] :: [ BEGIN ] :: Running 'tc class show dev ens3f1' class htb 1:fffe root rate 10Gbit ceil 10Gbit burst 1250b cburst 1250b class htb 1:2 parent 1:fffe prio 0 rate 10Mbit ceil 20Mbit burst 2750000b cburst 2750000b -------------------change right :: [ 23:17:27 ] :: [ PASS ] :: Command 'tc class show dev ens3f1' (Expected 0, got 0) MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 172.16.104.201 () port 0 AF_INET Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. 10^6bits/sec 87380 16384 16384 11.16 21.01 -------------traffic shape
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (ovn22.03 bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2022:6870