Bug 1733536 - SR-IOV VF QoS seems not working anymore
Summary: SR-IOV VF QoS seems not working anymore
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-neutron
Version: 13.0 (Queens)
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: ---
Assignee: Rodolfo Alonso
QA Contact: Eran Kuris
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-07-26 12:20 UTC by Federico Iezzi
Modified: 2019-08-21 11:58 UTC (History)
12 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-08-21 11:58:40 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description Federico Iezzi 2019-07-26 12:20:17 UTC
Description of problem:
SR-IOV VF is supposed to support bandwidth-limit max-kbps QoS policy [1] [2]. Testing it with the latest GA public bits (OSP13z7 and RHEL 7.6 3.10.0-957.21.3.el7.x86_64 kernel) environment, it doesn't not work as expected.


        openstack network qos policy create policy0

        openstack network qos rule create policy0 \
        --type bandwidth-limit \
        --max-kbps 10000 \
        --max-burst-kbits 10000 \
        --egress

        openstack network create \
        --provider-network-type vlan \
        --provider-physical-network niantic_pool \
        --provider-segment 406 \
        --mtu 9000 \
        vlan406
        openstack subnet create \
        --network vlan406 \
        --no-dhcp \
        --gateway none \
        --subnet-range 172.19.0.0/24 \
        vlan406

        openstack port create --network vlan406 --fixed-ip ip-address=172.19.0.20 \
        --vnic-type direct --binding-profile type=dict --binding-profile trusted=true \
        --qos-policy policy0 \
        172.19.0.20

        openstack port create --network vlan406 --fixed-ip ip-address=172.19.0.21 \
        --vnic-type direct --binding-profile type=dict --binding-profile trusted=true \
        --qos-policy policy0 \
        172.19.0.21

        openstack server create \
        --image fedora \
        --flavor nfv \
        --nic net-id=$(openstack network show mgmt --format value --column id) \
        --nic port-id=$(openstack port show 172.19.0.20 --format value --column id) \
        --config-drive \
        --key-name undercloud \
        vm-qos-sriov1

        openstack server create \
        --image fedora \
        --flavor nfv \
        --nic net-id=$(openstack network show mgmt --format value --column id) \
        --nic port-id=$(openstack port show 172.19.0.21 --format value --column id) \
        --config-drive \
        --key-name undercloud \
        --wait \
        vm-qos-sriov2

Now following test is not done with NFVBench nor TRex but with iperf3 which is not Telco representative, fully agree, but definitely shows how the QoS policy is not enforced.

## Client results ##
fedora@vm-qos-sriov1 ~]$ sudo iperf3 -c 172.19.0.20 
Connecting to host 172.19.0.20, port 5201
[  5] local 172.19.0.21 port 43652 connected to 172.19.0.20 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  1.10 GBytes  9.42 Gbits/sec    0   1.41 MBytes       
[  5]   1.00-2.00   sec  1.09 GBytes  9.39 Gbits/sec    0   1.41 MBytes       
[  5]   2.00-3.00   sec  1.09 GBytes  9.39 Gbits/sec    0   1.41 MBytes       
[  5]   3.00-4.00   sec  1.09 GBytes  9.39 Gbits/sec    0   1.41 MBytes       
[  5]   4.00-5.00   sec  1.09 GBytes  9.39 Gbits/sec    0   1.41 MBytes       
[  5]   5.00-6.00   sec  1.09 GBytes  9.39 Gbits/sec    0   1.41 MBytes       
[  5]   6.00-7.00   sec  1.09 GBytes  9.39 Gbits/sec    0   1.41 MBytes       
[  5]   7.00-8.00   sec  1.09 GBytes  9.39 Gbits/sec    0   1.41 MBytes       
[  5]   8.00-9.00   sec  1.09 GBytes  9.39 Gbits/sec    0   1.41 MBytes       
[  5]   9.00-10.00  sec  1.09 GBytes  9.38 Gbits/sec    0   1.41 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  10.9 GBytes  9.39 Gbits/sec    0             sender
[  5]   0.00-10.04  sec  10.9 GBytes  9.35 Gbits/sec                  receiver

## Server results ##
[root@vm-qos-sriov2 ~]# iperf3 -s
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 172.19.0.21, port 43650
[  5] local 172.19.0.20 port 5201 connected to 172.19.0.21 port 43652
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  1.05 GBytes  9.02 Gbits/sec
[  5]   1.00-2.00   sec  1.09 GBytes  9.39 Gbits/sec
[  5]   2.00-3.00   sec  1.09 GBytes  9.39 Gbits/sec
[  5]   3.00-4.00   sec  1.09 GBytes  9.39 Gbits/sec
[  5]   4.00-5.00   sec  1.09 GBytes  9.39 Gbits/sec
[  5]   5.00-6.00   sec  1.09 GBytes  9.39 Gbits/sec
[  5]   6.00-7.00   sec  1.09 GBytes  9.39 Gbits/sec
[  5]   7.00-8.00   sec  1.09 GBytes  9.39 Gbits/sec
[  5]   8.00-9.00   sec  1.09 GBytes  9.39 Gbits/sec
[  5]   9.00-10.00  sec  1.09 GBytes  9.39 Gbits/sec
[  5]  10.00-10.04  sec  44.4 MBytes  9.37 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.04  sec  10.9 GBytes  9.35 Gbits/sec                  receiver

The two VMs were running on different compute nodes (both sosreports will shortly follow).

[1] https://docs.openstack.org/neutron/queens/admin/config-qos.html
[2] https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/13/html/network_functions_virtualization_planning_and_configuration_guide/parent_assembly_tuning#configuring_quality_of_service_qos_in_an_nfvi_environment

Version-Release number of selected component (if applicable):
OSP13 tested with Z7 and latest GA public bits of RHEL 7.6 kernel 3.10.0-957.21.3


How reproducible:
(please also see above OpenStack CLI commands)
- Create QoS policy
- Create a rule bandwidth-limit type with a small enough number to notice the policy in-action
- Create a Neutron network with two SR-IOV VF ports assigning the policy at the creation time
- Create two VM
- Send traffic between those two VM on the SR-IOV ports 

Actual results:
QoS policy is *not* enforced.

Expected results:
QoS policy is enforced.

Comment 5 Federico Iezzi 2019-08-21 11:58:40 UTC
Thanks Rodolfo,

QoS policy was not enforced due to the SR-IOV agent misconfiguration.

This on my test system (Niantic + RHEL 7.7) with 1Gbps bandwidth-limit policy.

# iperf3 -c 10.10.10.1 -i 1 -t 5
Connecting to host 10.10.10.1, port 5201
[  5] local 10.10.10.2 port 53258 connected to 10.10.10.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   115 MBytes   967 Mbits/sec    0    416 KBytes
[  5]   1.00-2.00   sec   114 MBytes   956 Mbits/sec    0    437 KBytes
[  5]   2.00-3.00   sec   114 MBytes   953 Mbits/sec    0    482 KBytes
[  5]   3.00-4.00   sec   113 MBytes   951 Mbits/sec    0    482 KBytes
[  5]   4.00-5.00   sec   113 MBytes   948 Mbits/sec    0    506 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-5.00   sec   569 MBytes   955 Mbits/sec    0             sender
[  5]   0.00-5.04   sec   567 MBytes   944 Mbits/sec                  receiver

I'll open a new BZ against doc to make sure TripleO is correctly configured with NeutronSriovAgentExtensions: "qos"

Thanks!


Note You need to log in before you can comment on or make changes to this bug.