Bug 1454753

Summary: HW offload support for Open vSwitch
Product: Red Hat Enterprise Linux 7 Reporter: Simon Horman <simon.horman>
Component: openvswitchAssignee: Rashid Khan <rkhan>
Status: CLOSED ERRATA QA Contact: Amit Supugade <asupugad>
Severity: medium Docs Contact: Ioanna Gkioka <igkioka>
Priority: medium    
Version: 7.5CC: aloughla, a.perevalov, atragler, fbaudin, fherrman, fleitner, kazen, mark.guinther, nick.viljoen, qding, rkhan, simon.horman, sukulkar, tredaelli
Target Milestone: rcKeywords: FutureFeature
Target Release: ---Flags: igkioka: needinfo-
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openvswitch-2.9.0-1.el7fdp Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-03-19 10:19:14 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1474827, 1475436, 1698527    

Description Simon Horman 2017-05-23 12:28:09 UTC
Preface:

The intention of this bug is to provide insight into a feature proposed by Netronome for inclusion in RHEL7.5.

Description:

Working is underway to add support for Open vSwitch to program flows into the kernel and hardware using TC-Flower.This is a mechanism to allow acceleration of OvS using hardware such as Netronome SmartNICs.

Status:

Under public review. The latest patch-set was posted as "[PATCH ovs V8 00/26] Introducing HW offload support for openvswitch". It is hoped this feature will be included in v2.8 of Open vSwitch.

Comment 2 Flavio Leitner 2017-06-07 17:47:02 UTC
The feature will be available when the package is rebased with the upstream version containing it.

Comment 3 Anita Tragler 2017-07-27 07:05:38 UTC
Plan to include OVS 2.9 in fast datapath for RHEL 7.5

Comment 5 Simon Horman 2017-11-30 11:46:36 UTC
Hi,

I'm a little unclear on what "fast datapath for RHEL 7.5" is, could you clarify.

Also, I'd like to point out that while not entirely complete (set-action support is missing) the patchset noted above was included in OvS v2.8.

Comment 6 Flavio Leitner 2018-02-14 19:48:32 UTC
We are going to rebase to 2.9.
It seems all you need will be part of 2.9 official release, otherwise please let us know.

Comment 7 Simon Horman 2018-02-15 07:16:05 UTC
Hi Flavio,

I confirm that at this time we believe that all we need is present in v2.9.

Comment 8 Anita Tragler 2018-02-15 11:58:24 UTC
(In reply to Simon Horman from comment #5)
> Hi,
> 
> I'm a little unclear on what "fast datapath for RHEL 7.5" is, could you
> clarify.
> 
> Also, I'd like to point out that while not entirely complete (set-action
> support is missing) the patchset noted above was included in OvS v2.8.

Fast Datapath is a separate channel or repo for releasing OVS content to RH Openstack (RHOSP), Openshift (OCP) and RH virtualization (RHV)

Comment 12 Amit Supugade 2018-03-06 20:06:09 UTC
Verified on openvswitch-2.9.0-3.el7fdp.x86_64

LOG-
[root@netqe14 /]# uname -r
3.10.0-858.el7.x86_64

[root@netqe14 ~]# change_firmware() {
>     APP=${1:-flower}
>     FWDIR=${2:-/lib/firmware/netronome/}
>     cd ${FWDIR}
>     for FW in *.nffw; do
>       if [ -L ${FW} ]; then
>         ln -sf ${APP}/${FW} ${FW}
>       fi
>     done
> }
[root@netqe14 ~]# 
[root@netqe14 ~]# change_firmware
[root@netqe14 netronome]# 
[root@netqe14 netronome]# cd
[root@netqe14 ~]# rmmod nfp; sleep 3; modprobe nfp
[root@netqe14 ~]# yum install -y http://download-node-02.eng.bos.redhat.com/brewroot/packages/openvswitch/2.9.0/3.el7fdp/x86_64/openvswitch-2.9.0-3.el7fdp.x86_64.rpm
.
.
[root@netqe14 ~]# systemctl start openvswitch
[root@netqe14 ~]# systemctl enable openvswitch
Created symlink from /etc/systemd/system/multi-user.target.wants/openvswitch.service to /usr/lib/systemd/system/openvswitch.service.
[root@netqe14 ~]# 
[root@netqe14 ~]# systemctl status openvswitch
● openvswitch.service - Open vSwitch
   Loaded: loaded (/usr/lib/systemd/system/openvswitch.service; enabled; vendor preset: disabled)
   Active: active (exited) since Tue 2018-03-06 11:12:38 EST; 12s ago
 Main PID: 11148 (code=exited, status=0/SUCCESS)

Mar 06 11:12:38 netqe14.knqe.lab.eng.bos.redhat.com systemd[1]: Starting Open vSwitch...
Mar 06 11:12:38 netqe14.knqe.lab.eng.bos.redhat.com systemd[1]: Started Open vSwitch.

[root@netqe14 ~]# echo 1 > /sys/bus/pci/devices/0000:$(lspci -d 19ee:4000 | cut -d ' ' -f 1)/sriov_numvfs
[root@netqe14 ~]# dmesg | grep nfp | grep Representor
[ 3419.453871] nfp 0000:82:00.0: nfp: Phys Port 0 Representor(eth1) created
[ 3419.461595] nfp 0000:82:00.0: nfp: Phys Port 4 Representor(eth2) created
[ 3419.469416] nfp 0000:82:00.0: nfp: PF0 Representor(eth3) created
[ 3744.862538] nfp 0000:82:00.0: nfp: VF0 Representor(eth1) created

[root@netqe14 ~]# pf=enp130s0np0
[root@netqe14 ~]# vfr=eth1
[root@netqe14 ~]# 
[root@netqe14 ~]# ovs-vsctl --if-exists del-br ovsbr0
[root@netqe14 ~]# ovs-vsctl add-br ovsbr0
[root@netqe14 ~]# ip link set ovsbr0 up
[root@netqe14 ~]# 
[root@netqe14 ~]# ovs-vsctl add-port ovsbr0 $pf
[root@netqe14 ~]# ovs-vsctl add-port ovsbr0 $vfr

[root@netqe14 ~]# ovs-vsctl set Open_vSwitch . other_config:hw-offload=true
[root@netqe14 ~]# ovs-vsctl get Open_vSwitch . other_config:hw-offload
"true"
[root@netqe14 ~]# ovs-ofctl add-flow ovsbr0 in_port=1,actions=output:2
[root@netqe14 ~]# ovs-ofctl add-flow ovsbr0 in_port=2,actions=output:1

[root@netqe14 /]# ovs-ofctl dump-flows ovsbr0
 cookie=0x0, duration=699.591s, table=0, n_packets=34, n_bytes=1578, in_port=enp130s0np0 actions=output:eth1
 cookie=0x0, duration=697.904s, table=0, n_packets=0, n_bytes=0, in_port=eth1 actions=output:enp130s0np0
 cookie=0x0, duration=1299.541s, table=0, n_packets=0, n_bytes=0, priority=0 actions=NORMAL

[root@netqe14 /]# ovs-dpctl dump-flows
in_port(2),eth_type(0x0806), packets:38, bytes:1748, used:0.390s, actions:3
[root@netqe14 /]# ovs-dpctl dump-flows type=offloaded
in_port(2),eth_type(0x0806), packets:42, bytes:1932, used:0.030s, actions:3
[root@netqe14 /]# ovs-dpctl dump-flows type=ovs

[root@netqe14 /]# tc -s filter show dev enp130s0np0 ingress
filter protocol arp pref 1 flower 
filter protocol arp pref 1 flower handle 0x1 
  eth_type arp
  not_in_hw
	action order 1: mirred (Egress Redirect to device eth1) stolen
 	index 1 ref 1 bind 1 installed 152 sec used 0 sec
 	Action statistics:
	Sent 5244 bytes 114 pkt (dropped 0, overlimits 0 requeues 0) 
	backlog 0b 0p requeues 0 
	cookie len 16 4d09eaad24492f6a

Comment 15 Ioanna Gkioka 2018-03-19 09:24:59 UTC
The openvwitch component is delivered through the fast datapath channel, it is not documented in release notes.

Comment 16 errata-xmlrpc 2018-03-19 10:19:14 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0550