Description of problem: SR-IOV VF is supposed to support bandwidth-limit max-kbps QoS policy [1] [2]. Testing it with the latest GA public bits (OSP13z7 and RHEL 7.6 3.10.0-957.21.3.el7.x86_64 kernel) environment, it doesn't not work as expected. openstack network qos policy create policy0 openstack network qos rule create policy0 \ --type bandwidth-limit \ --max-kbps 10000 \ --max-burst-kbits 10000 \ --egress openstack network create \ --provider-network-type vlan \ --provider-physical-network niantic_pool \ --provider-segment 406 \ --mtu 9000 \ vlan406 openstack subnet create \ --network vlan406 \ --no-dhcp \ --gateway none \ --subnet-range 172.19.0.0/24 \ vlan406 openstack port create --network vlan406 --fixed-ip ip-address=172.19.0.20 \ --vnic-type direct --binding-profile type=dict --binding-profile trusted=true \ --qos-policy policy0 \ 172.19.0.20 openstack port create --network vlan406 --fixed-ip ip-address=172.19.0.21 \ --vnic-type direct --binding-profile type=dict --binding-profile trusted=true \ --qos-policy policy0 \ 172.19.0.21 openstack server create \ --image fedora \ --flavor nfv \ --nic net-id=$(openstack network show mgmt --format value --column id) \ --nic port-id=$(openstack port show 172.19.0.20 --format value --column id) \ --config-drive \ --key-name undercloud \ vm-qos-sriov1 openstack server create \ --image fedora \ --flavor nfv \ --nic net-id=$(openstack network show mgmt --format value --column id) \ --nic port-id=$(openstack port show 172.19.0.21 --format value --column id) \ --config-drive \ --key-name undercloud \ --wait \ vm-qos-sriov2 Now following test is not done with NFVBench nor TRex but with iperf3 which is not Telco representative, fully agree, but definitely shows how the QoS policy is not enforced. ## Client results ## fedora@vm-qos-sriov1 ~]$ sudo iperf3 -c 172.19.0.20 Connecting to host 172.19.0.20, port 5201 [ 5] local 172.19.0.21 port 43652 connected to 172.19.0.20 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 1.10 GBytes 9.42 Gbits/sec 0 1.41 MBytes [ 5] 1.00-2.00 sec 1.09 GBytes 9.39 Gbits/sec 0 1.41 MBytes [ 5] 2.00-3.00 sec 1.09 GBytes 9.39 Gbits/sec 0 1.41 MBytes [ 5] 3.00-4.00 sec 1.09 GBytes 9.39 Gbits/sec 0 1.41 MBytes [ 5] 4.00-5.00 sec 1.09 GBytes 9.39 Gbits/sec 0 1.41 MBytes [ 5] 5.00-6.00 sec 1.09 GBytes 9.39 Gbits/sec 0 1.41 MBytes [ 5] 6.00-7.00 sec 1.09 GBytes 9.39 Gbits/sec 0 1.41 MBytes [ 5] 7.00-8.00 sec 1.09 GBytes 9.39 Gbits/sec 0 1.41 MBytes [ 5] 8.00-9.00 sec 1.09 GBytes 9.39 Gbits/sec 0 1.41 MBytes [ 5] 9.00-10.00 sec 1.09 GBytes 9.38 Gbits/sec 0 1.41 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 10.9 GBytes 9.39 Gbits/sec 0 sender [ 5] 0.00-10.04 sec 10.9 GBytes 9.35 Gbits/sec receiver ## Server results ## [root@vm-qos-sriov2 ~]# iperf3 -s ----------------------------------------------------------- Server listening on 5201 ----------------------------------------------------------- Accepted connection from 172.19.0.21, port 43650 [ 5] local 172.19.0.20 port 5201 connected to 172.19.0.21 port 43652 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 1.05 GBytes 9.02 Gbits/sec [ 5] 1.00-2.00 sec 1.09 GBytes 9.39 Gbits/sec [ 5] 2.00-3.00 sec 1.09 GBytes 9.39 Gbits/sec [ 5] 3.00-4.00 sec 1.09 GBytes 9.39 Gbits/sec [ 5] 4.00-5.00 sec 1.09 GBytes 9.39 Gbits/sec [ 5] 5.00-6.00 sec 1.09 GBytes 9.39 Gbits/sec [ 5] 6.00-7.00 sec 1.09 GBytes 9.39 Gbits/sec [ 5] 7.00-8.00 sec 1.09 GBytes 9.39 Gbits/sec [ 5] 8.00-9.00 sec 1.09 GBytes 9.39 Gbits/sec [ 5] 9.00-10.00 sec 1.09 GBytes 9.39 Gbits/sec [ 5] 10.00-10.04 sec 44.4 MBytes 9.37 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-10.04 sec 10.9 GBytes 9.35 Gbits/sec receiver The two VMs were running on different compute nodes (both sosreports will shortly follow). [1] https://docs.openstack.org/neutron/queens/admin/config-qos.html [2] https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/13/html/network_functions_virtualization_planning_and_configuration_guide/parent_assembly_tuning#configuring_quality_of_service_qos_in_an_nfvi_environment Version-Release number of selected component (if applicable): OSP13 tested with Z7 and latest GA public bits of RHEL 7.6 kernel 3.10.0-957.21.3 How reproducible: (please also see above OpenStack CLI commands) - Create QoS policy - Create a rule bandwidth-limit type with a small enough number to notice the policy in-action - Create a Neutron network with two SR-IOV VF ports assigning the policy at the creation time - Create two VM - Send traffic between those two VM on the SR-IOV ports Actual results: QoS policy is *not* enforced. Expected results: QoS policy is enforced.
Thanks Rodolfo, QoS policy was not enforced due to the SR-IOV agent misconfiguration. This on my test system (Niantic + RHEL 7.7) with 1Gbps bandwidth-limit policy. # iperf3 -c 10.10.10.1 -i 1 -t 5 Connecting to host 10.10.10.1, port 5201 [ 5] local 10.10.10.2 port 53258 connected to 10.10.10.1 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 115 MBytes 967 Mbits/sec 0 416 KBytes [ 5] 1.00-2.00 sec 114 MBytes 956 Mbits/sec 0 437 KBytes [ 5] 2.00-3.00 sec 114 MBytes 953 Mbits/sec 0 482 KBytes [ 5] 3.00-4.00 sec 113 MBytes 951 Mbits/sec 0 482 KBytes [ 5] 4.00-5.00 sec 113 MBytes 948 Mbits/sec 0 506 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-5.00 sec 569 MBytes 955 Mbits/sec 0 sender [ 5] 0.00-5.04 sec 567 MBytes 944 Mbits/sec receiver I'll open a new BZ against doc to make sure TripleO is correctly configured with NeutronSriovAgentExtensions: "qos" Thanks!