Bug 1702334 - [NETRO 7.7 Feat] ovs-tc: support OvS internal port offload
Summary: [NETRO 7.7 Feat] ovs-tc: support OvS internal port offload
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux Fast Datapath
Classification: Red Hat
Component: openvswitch2.11
Version: FDP 19.D
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: ---
Assignee: Pablo Cascon (Netronome)
QA Contact: qding
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-04-23 14:03 UTC by Simon Horman
Modified: 2021-03-09 08:25 UTC (History)
14 users (show)

Fixed In Version: openvswitch2.11-2.11.0-10.el7fdn
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-07-12 09:53:02 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
Proposed patch to backport on pkg openvswitch2.11 (68.88 KB, patch)
2019-05-16 13:51 UTC, Pablo Cascon (Netronome)
no flags Details | Diff


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:1750 0 None None None 2019-07-12 09:53:07 UTC

Description Simon Horman 2019-04-23 14:03:31 UTC
In order to support offload of OVS tunnels where the tunnel endpoint is on an OvS bridge Netronome would like to request a backport of the following upstream patches to OVS v2.11 as provided in the fast datapath.



commit 608ff46aaf0dd4330c2488c260ee42992e17a6fd
Author: John Hurley <john.hurley>
Date:   Tue Apr 9 15:36:14 2019 +0100

    ovs-tc: offload datapath rules matching on internal ports
    
    Rules applied to OvS internal ports are not represented in TC datapaths.
    However, it is possible to support rules matching on internal ports in TC.
    The start_xmit ndo of OvS internal ports directs packets back into the OvS
    kernel datapath where they are rematched with the ingress port now being
    that of the internal port. Due to this, rules matching on an internal port
    can be added as TC filters to an egress qdisc for these ports.
    
    Allow rules applied to internal ports to be offloaded to TC as egress
    filters. Rules redirecting to an internal port are also offloaded. These
    are supported by the redirect ingress functionality applied in an earlier
    patch.
    
    Signed-off-by: John Hurley <john.hurley>
    Reviewed-by: Roi Dayan <roid>
    Signed-off-by: Simon Horman <simon.horman>

commit 95255018a83ee460a5e2e4833e3f444a502c5804
Author: John Hurley <john.hurley>
Date:   Tue Apr 9 15:36:13 2019 +0100

    ovs-tc: allow offloading TC rules to egress qdiscs
    
    Offloading rules to a TC datapath only allows the creating of ingress hook
    qdiscs and the application of filters to these. However, there may be
    certain situations where an egress qdisc is more applicable (e.g. when
    offloading to TC rules applied to OvS internal ports).
    
    Extend the TC API in OvS to allow the creation of egress qdiscs and to add
    or interact with flower filters applied to these.
    
    Signed-off-by: John Hurley <john.hurley>
    Reviewed-by: Roi Dayan <roid>
    Signed-off-by: Simon Horman <simon.horman>

commit 4aa2dc04ec6e4b005b1bcf520b8ff02265ecb569
Author: John Hurley <john.hurley>
Date:   Tue Apr 9 15:36:12 2019 +0100

    ovs-tc: allow offloading of ingress mirred TC actions to datapath
    
    The TC datapath only permits the offload of mirred actions if they are
    egress. To offload TC actions that output to OvS internal ports, ingress
    mirred actions are required. At the TC layer, an ingress mirred action
    passes the packet back into the network stack as if it came in the action
    port rather than attempting to egress the port.
    
    Update OvS-TC offloads to support ingress mirred actions. To ensure
    packets that match these rules are properly passed into the network stack,
    add a TC skbedit action along with ingress mirred that sets the pkt_type
    to PACKET_HOST. This mirrors the functionality of the OvS internal port
    kernel module.
    
    Signed-off-by: John Hurley <john.hurley>
    Reviewed-by: Roi Dayan <roid>
    Signed-off-by: Simon Horman <simon.horman>

commit 7808b2b971e83c672700cb7ab306ce46282c84d9
Author: John Hurley <john.hurley>
Date:   Tue Apr 9 15:36:11 2019 +0100

    compat: add compatibility headers for tc skbedit action
    
    OvS includes compat code for several TC actions including vlan, mirred and
    tunnel key. Add support for using skbedit actions when compiling
    user-space code against older kernel headers.
    
    Signed-off-by: John Hurley <john.hurley>
    Reviewed-by: Roi Dayan <roid>
    Signed-off-by: Simon Horman <simon.horman>

Comment 3 Simon Horman 2019-04-25 09:19:27 UTC
This ticket relates to the following tickets:

NFP Firmware: (AOTC-2.10.A.23) (BZ1702330)
NFP Kernel Driver: Backport nfp Flower flow merging (BZ1700452)

Comment 5 Eelco Chaudron 2019-04-30 12:19:56 UTC
(In reply to Eelco Chaudron from comment #4)

Ooops, it needs to be the next branch, fast-datapath-next-rhel-7

Comment 6 Pablo Cascon (Netronome) 2019-05-16 13:51:19 UTC
Created attachment 1569521 [details]
Proposed patch to backport on pkg openvswitch2.11

Adding a patch as a proposal to backport the changes requested in this BZ to pkg openvswitch2.11 .Brew build at https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=21721595

Comment 7 Timothy Redaelli 2019-05-23 15:15:56 UTC
Applied patch in comment 6 on fast-datapath-next-rhel-7.

Aligned fast-datapath-next-rhel-8 to fast-datapath-next-rhel-7 (with this fix).

Comment 8 Pablo Cascon (Netronome) 2019-05-23 15:31:08 UTC
Brilliant thanks!

Comment 10 qding 2019-06-18 09:08:49 UTC
Set it to VERIFIED, though there are still packets tcpdumped on the offloaded port according to bz1700452#c17

[root@dell-per730-04 ~]# uname -r
3.10.0-1055.el7.x86_64
[root@dell-per730-04 ~]# rpm -q openvswitch2.11 linux-firmware
openvswitch2.11-2.11.0-12.el7fdp.x86_64
linux-firmware-20190429-72.gitddde598.el7.noarch
[root@dell-per730-04 ~]# 


Steps to verify:

Host1: (Any host withe Host2 connected back-to-back)

ip netns add ns1
ip netns add ns2

ip link add veth0 type veth peer name veth1
ip link set veth0 up
ip link set veth1 up

ip link add veth10 type veth peer name veth11
ip link set veth10 up
ip link set veth11 up

ip link set veth0 netns ns1
ip link set veth10 netns ns2

ip link set veth0 netns ns1
ip netns exec ns1 ip link set veth0 up
ip netns exec ns1 ip add add 192.168.213.1/24 dev veth0

ip link set veth10 netns ns2
ip netns exec ns2 ip link set veth10 up
ip netns exec ns2 ip add add 192.168.213.10/24 dev veth10

ip link set p4p1 up
ovs-vsctl add-br ovsbr0
ovs-vsctl add-port ovsbr0 p6p1
ovs-vsctl add-port ovsbr0 veth1
ip link set ovsbr0 up
ip add add 172.16.213.2/24 dev ovsbr0


ovs-vsctl add-br ovsbr1
ovs-vsctl add-port ovsbr1 veth11
ovs-vsctl add-port ovsbr1 vxlan1 -- set interface vxlan1 type=vxlan options:key=123 options:remote_ip=172.16.213.1

ip netns exec ns1 ping 192.168.213.10 -i 0

Host2 (Device under test):

ip link set eno1np0 up
ovs-vsctl add-br ovsbr0
ovs-vsctl add-port ovsbr0 eno1np0
ovs-vsctl add-port ovsbr0 vxlan1 -- set interface vxlan1 type=vxlan options:key=123 options:remote_ip=172.16.213.2
ip link set ovsbr0 up
ip add add 172.16.213.1/24 dev ovsbr0
ovs-vsctl set Open_vSwitch . other_config:hw-offload=true
systemctl restart openvswitch



[root@dell-per730-04 ~]# ovs-dpctl dump-flows 
2019-06-18T09:02:11Z|00001|dpif_netlink|INFO|The kernel module does not support meters.
in_port(1),eth(src=00:15:4d:12:2d:ac,dst=3c:fd:fe:bb:1b:6c),eth_type(0x0800),ipv4(frag=no), packets:415342, bytes:40703940, used:1.630s, actions:2
tunnel(tun_id=0x7b,src=172.16.213.2,dst=172.16.213.1,tp_dst=4789,flags(+key)),in_port(3),eth(src=76:c6:f0:d0:bd:c1,dst=9e:b3:92:28:db:4b),eth_type(0x0806), packets:0, bytes:0, used:4.370s, actions:2
tunnel(tun_id=0x7b,src=172.16.213.2,dst=172.16.213.1,tp_dst=4789,flags(+key)),in_port(3),eth(src=76:c6:f0:d0:bd:c1,dst=9e:b3:92:28:db:4b),eth_type(0x0800),ipv4(frag=no), packets:5732672, bytes:561801856, used:0.000s, actions:2
in_port(2),eth(src=9e:b3:92:28:db:4b,dst=76:c6:f0:d0:bd:c1),eth_type(0x0806), packets:0, bytes:0, used:4.370s, actions:set(tunnel(tun_id=0x7b,dst=172.16.213.2,ttl=64,tp_dst=4789,flags(key))),3
in_port(2),eth(src=3c:fd:fe:bb:1b:6c,dst=00:15:4d:12:2d:ac),eth_type(0x0800),ipv4(frag=no), packets:0, bytes:0, used:4.370s, actions:1
in_port(2),eth(src=9e:b3:92:28:db:4b,dst=76:c6:f0:d0:bd:c1),eth_type(0x0800),ipv4(tos=0/0x3,frag=no), packets:30488, bytes:2987824, used:1.630s, actions:set(tunnel(tun_id=0x7b,dst=172.16.213.2,ttl=64,tp_dst=4789,flags(key))),3
[root@dell-per730-04 ~]# 
[root@dell-per730-04 ~]# 
[root@dell-per730-04 ~]# 
[root@dell-per730-04 ~]# 
[root@dell-per730-04 ~]# tcpdump -nnev -i ovsbr0
tcpdump: listening on ovsbr0, link-type EN10MB (Ethernet), capture size 262144 bytes
05:02:31.172531 00:15:4d:12:2d:ac > 3c:fd:fe:bb:1b:6c, ethertype IPv4 (0x0800), length 148: (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 134)
    172.16.213.1.58363 > 172.16.213.2.4789: VXLAN, flags [I] (0x08), vni 123
9e:b3:92:28:db:4b > 76:c6:f0:d0:bd:c1, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 37182, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.213.1 > 192.168.213.10: ICMP echo request, id 21994, seq 58860, length 64
05:02:31.659507 3c:fd:fe:bb:1b:6c > 00:15:4d:12:2d:ac, ethertype IPv4 (0x0800), length 92: (tos 0x0, ttl 64, id 33829, offset 0, flags [DF], proto UDP (17), length 78)
    172.16.213.2.51817 > 172.16.213.1.4789: VXLAN, flags [I] (0x08), vni 123
76:c6:f0:d0:bd:c1 > 9e:b3:92:28:db:4b, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Reply 192.168.213.10 is-at 76:c6:f0:d0:bd:c1, length 28
05:02:32.811044 3c:fd:fe:bb:1b:6c > 00:15:4d:12:2d:ac, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.213.1 tell 172.16.213.2, length 46
05:02:32.811050 00:15:4d:12:2d:ac > 3c:fd:fe:bb:1b:6c, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Reply 172.16.213.1 is-at 00:15:4d:12:2d:ac, length 28
05:02:41.685675 00:15:4d:12:2d:ac > 3c:fd:fe:bb:1b:6c, ethertype IPv4 (0x0800), length 148: (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 134)
    172.16.213.1.63459 > 172.16.213.2.4789: VXLAN, flags [I] (0x08), vni 123
9e:b3:92:28:db:4b > 76:c6:f0:d0:bd:c1, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 34859, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.213.1 > 192.168.213.10: ICMP echo request, id 21994, seq 56535, length 64
05:02:42.667067 3c:fd:fe:bb:1b:6c > 00:15:4d:12:2d:ac, ethertype IPv4 (0x0800), length 92: (tos 0x0, ttl 64, id 39500, offset 0, flags [DF], proto UDP (17), length 78)
    172.16.213.2.51817 > 172.16.213.1.4789: VXLAN, flags [I] (0x08), vni 123
76:c6:f0:d0:bd:c1 > 9e:b3:92:28:db:4b, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.213.1 tell 192.168.213.10, length 28
05:02:46.705035 00:15:4d:12:2d:ac > 3c:fd:fe:bb:1b:6c, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.213.2 tell 172.16.213.1, length 28
05:02:46.705453 3c:fd:fe:bb:1b:6c > 00:15:4d:12:2d:ac, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 172.16.213.2 is-at 3c:fd:fe:bb:1b:6c, length 46
05:03:02.706099 00:15:4d:12:2d:ac > 3c:fd:fe:bb:1b:6c, ethertype IPv4 (0x0800), length 148: (tos 0x0, ttl 64, id 57502, offset 0, flags [DF], proto UDP (17), length 134)
    172.16.213.1.42911 > 172.16.213.2.4789: VXLAN, flags [I] (0x08), vni 123
9e:b3:92:28:db:4b > 76:c6:f0:d0:bd:c1, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 60975, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.213.1 > 192.168.213.10: ICMP echo request, id 21994, seq 15813, length 64
^C
9 packets captured
9 packets received by filter
0 packets dropped by kernel
[root@dell-per730-04 ~]# tcpdump -nnev -i eno1np0
tcpdump: listening on eno1np0, link-type EN10MB (Ethernet), capture size 262144 bytes
05:03:31.009876 3c:fd:fe:bb:1b:6c > 01:80:c2:00:00:0e, ethertype LLDP (0x88cc), length 83: LLDP, length 69
	Chassis ID TLV (1), length 7
	  Subtype MAC address (4): 3c:fd:fe:bb:1b:6c
	Port ID TLV (2), length 7
	  Subtype MAC address (3): 3c:fd:fe:bb:1b:6c
	Time to Live TLV (3), length 2: TTL 121s
	Organization specific TLV (127), length 25: OUI Ethernet bridged (0x0080c2)
	  ETS Configuration Subtype (9)
	    Willing:1, CBS:2, RES:0, Max TCs:0
	    Priority Assignment Table
	     Priority : 0   1   2   3   4   5   6   7
	     Value    : 0   0   0   0   0   0   0   0  
	    TC Bandwidth Table
	     TC%   : 0   1   2   3   4   5   6   7
	     Value : 100 0   0   0   0   0   0   0  
	    TSA Assignment Table
	     Traffic Class: 0   1   2   3   4   5   6   7
	     Value        : 2   0   0   0   0   0   0   0  
	Organization specific TLV (127), length 6: OUI Ethernet bridged (0x0080c2)
	  Priority Flow Control Configuration Subtype (11)
	    Willing: 1, MBC: 0, RES: 0, PFC cap:8 
	    PFC Enable
	     Priority : 0  1  2  3  4  5  6  7
	     Value    : 0  0  0  0  0  0  0  0  
	Organization specific TLV (127), length 8: OUI Ethernet bridged (0x0080c2)
	  Application Priority Subtype (12)
	    RES: 0
	    Application Priority Table
	      Priority: 3, RES: 0, Sel: 1, Protocol ID: 24969
	End TLV (0), length 0
05:03:34.030554 9e:b3:92:28:db:4b > 76:c6:f0:d0:bd:c1, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 64935, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.213.1 > 192.168.213.10: ICMP echo request, id 21994, seq 17498, length 64
05:03:34.030808 00:15:4d:12:2d:ac > 3c:fd:fe:bb:1b:6c, ethertype IPv4 (0x0800), length 148: (tos 0x0, ttl 64, id 3368, offset 0, flags [DF], proto UDP (17), length 134)
    172.16.213.1.42911 > 172.16.213.2.4789: VXLAN, flags [I] (0x08), vni 123
9e:b3:92:28:db:4b > 76:c6:f0:d0:bd:c1, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 64935, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.213.1 > 192.168.213.10: ICMP echo request, id 21994, seq 17498, length 64
05:03:39.962957 3c:fd:fe:bb:1b:6c > 00:15:4d:12:2d:ac, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.213.1 tell 172.16.213.2, length 46
05:03:39.963236 00:15:4d:12:2d:ac > 3c:fd:fe:bb:1b:6c, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Reply 172.16.213.1 is-at 00:15:4d:12:2d:ac, length 28
05:03:40.778969 9e:b3:92:28:db:4b > 76:c6:f0:d0:bd:c1, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.213.10 tell 192.168.213.1, length 46
05:03:40.779122 00:15:4d:12:2d:ac > 3c:fd:fe:bb:1b:6c, ethertype IPv4 (0x0800), length 110: (tos 0x0, ttl 64, id 3934, offset 0, flags [DF], proto UDP (17), length 96)
    172.16.213.1.56605 > 172.16.213.2.4789: VXLAN, flags [I] (0x08), vni 123
9e:b3:92:28:db:4b > 76:c6:f0:d0:bd:c1, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.213.10 tell 192.168.213.1, length 46
05:03:40.779501 3c:fd:fe:bb:1b:6c > 00:15:4d:12:2d:ac, ethertype IPv4 (0x0800), length 92: (tos 0x0, ttl 64, id 30463, offset 0, flags [DF], proto UDP (17), length 78)
    172.16.213.2.51817 > 172.16.213.1.4789: VXLAN, flags [I] (0x08), vni 123
76:c6:f0:d0:bd:c1 > 9e:b3:92:28:db:4b, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Reply 192.168.213.10 is-at 76:c6:f0:d0:bd:c1, length 28
05:03:40.779756 76:c6:f0:d0:bd:c1 > 9e:b3:92:28:db:4b, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Reply 192.168.213.10 is-at 76:c6:f0:d0:bd:c1, length 28
^C
9 packets captured
9 packets received by filter
0 packets dropped by kernel
[root@dell-per730-04 ~]#

Comment 11 Timothy Redaelli 2019-06-21 20:21:34 UTC
*** Bug 1721219 has been marked as a duplicate of this bug. ***

Comment 13 errata-xmlrpc 2019-07-12 09:53:02 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:1750

Comment 14 Timothy Redaelli 2020-03-10 17:05:12 UTC
The commits will be reverted in FDP 20.C due to bz#1737982.


Note You need to log in before you can comment on or make changes to this bug.