Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

The FDP team is no longer accepting new bugs in Bugzilla. Please report your issues under FDP project in Jira. Thanks.

Bug 2011779

Summary:	[OVN] Add configuration option to avoid gateway_mtu matches.
Product:	Red Hat Enterprise Linux Fast Datapath	Reporter:	Dumitru Ceara <dceara>
Component:	OVN	Assignee:	Dumitru Ceara <dceara>
Status:	CLOSED ERRATA	QA Contact:	Jianlin Shi <jishi>
Severity:	high	Docs Contact:
Priority:	medium
Version:	FDP 21.H	CC:	ctrautma, dcbw, jiji, mleitner, moshele
Target Milestone:	---	Keywords:	FutureFeature
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	ovn21.12-21.12.0-24.el8fdp	Doc Type:	Enhancement
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-02-24 17:47:39 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Dumitru Ceara 2021-10-07 11:27:05 UTC

Description of problem:

Users can currently configure Logical_Router_Port.options:gateway_mtu to set a minimum packet size for which ICMP fragmentation needed packet is sent.

This is internally implemented using the check_pkt_larger action for all IP packets received on that logical router port. This action is not offloadable to hardware causing all traffic received on the router port to be processed in software.  This is not really needed, for example for TCP traffic.

A way to avoid that is to add a higher priority openflow that bypasses the flow with the check_pkt_larger action.  This can be configured via an additional logical router port option, "gateway_mtu_bypass" (or similar).

Comment 1 Dumitru Ceara 2021-10-07 11:28:08 UTC

Comment 2 Dan Williams 2021-10-08 14:01:21 UTC

What's the difference in behavior between just not setting gateway_mtu at all, and settting gateway_mtu_bypass?

Comment 3 Dumitru Ceara 2021-10-08 14:44:07 UTC

(In reply to Dan Williams from comment #2)
> What's the difference in behavior between just not setting gateway_mtu at
> all, and settting gateway_mtu_bypass?

Maybe "gateway_mtu_bypass" is not the best name. It's supposed to be a match string.

In any case, assuming on a logical router port we have LRP.Options={gateway_mtu:1400}, the current behavior is to call check_pkt_larger(1400) for all IP packets received on that router port.

If we change the configuration to LRP.Options={gateway_mtu:1400, gateway_mtu_bypass=tcp}, all TCP traffic will not hit the check_pkt_larger(1400) flow anymore and will just advance to the next stage in the pipeline, while for all other IP traffic we'll still perform check_pkt_larger(1400). This differentiation should allow us to offload TCP traffic to hardware, while still generating ICMP Frag Needed for UDP packets that are larger than 1400 bytes.

Like this the performance hit is only for part of the traffic (for UDP).

Comment 4 Marcelo Ricardo Leitner 2021-10-27 14:56:09 UTC

Hi Dumitru, how's it going, any news on this front? Thanks.

Comment 5 Dumitru Ceara 2021-11-01 09:59:50 UTC

(In reply to Marcelo Ricardo Leitner from comment #4)
> Hi Dumitru, how's it going, any news on this front? Thanks.

Hi Marcelo,

I didn't get a chance to implement this yet.  I'll try to post something for review this week.

Thanks!

Comment 6 Dumitru Ceara 2021-11-04 19:18:04 UTC

V1 posted for review: http://patchwork.ozlabs.org/project/ovn/list/?series=270460&state=*

Comment 7 Dumitru Ceara 2021-11-05 09:22:12 UTC

V2 posted for review: http://patchwork.ozlabs.org/project/ovn/list/?series=270522&state=*

Comment 8 Dumitru Ceara 2021-11-26 16:17:09 UTC

V3 posted for review: http://patchwork.ozlabs.org/project/ovn/list/?series=273992&state=*

Comment 9 Dumitru Ceara 2021-11-29 08:55:21 UTC

V3 accepted upstream.

Comment 14 Jianlin Shi 2022-01-18 07:46:36 UTC

tested with following script:

systemctl start openvswitch                                  
systemctl start ovn-northd                                   
                                                             
ovn-nbctl set-connection ptcp:6641                                               
ovn-sbctl set-connection ptcp:6642                 
                                                                            
ovs-vsctl set open . external-ids:system-id=hv0 external-ids:ovn-remote=tcp:127.0.0.1:6642 external-ids:ovn-encap-type=geneve external-ids:ovn-encap-ip=127.0.0.1
                                                             
systemctl restart ovn-controller                                         
ovn-nbctl lr-add lr1
ovn-nbctl lrp-add lr1 lr1ls1 00:01:03:0d:ff:01 192.168.1.254/24 2000::a/64      
ovn-nbctl lrp-add lr1 lr1ls2 00:01:03:0d:ff:02 192.168.2.254/24 2001::a/64
                                           
ovn-nbctl ls-add ls2                                         
ovn-nbctl lsp-add ls2 ls2lr1                                           
ovn-nbctl lsp-set-type ls2lr1 router                            
ovn-nbctl lsp-set-options ls2lr1 router-port=lr1ls2                         
ovn-nbctl lsp-set-addresses ls2lr1 "00:01:03:0d:ff:02 192.168.2.254 2001::a"
ovn-nbctl lsp-add ls2 ls2p1                
ovn-nbctl lsp-set-addresses ls2p1 "00:01:02:03:02:01 192.168.2.1 2001::1"
ovs-vsctl add-port br-int eth-server0 -- set interface eth-server0 type=internal
ip netns add server0                                           
ip link set eth-server0 netns server0                                   
ip netns exec server0 ip link set eth-server0 up                 
ip netns exec server0 ip link set eth-server0 address 00:01:02:03:02:01 
ip netns exec server0 ip addr add 192.168.2.1/24 dev eth-server0
ip netns exec server0 ip route add default via 192.168.2.254 dev eth-server0
ip netns exec server0 ip -6 addr add 2001::1/64 dev eth-server0
ip netns exec server0 ip -6 route add default via 2001::a dev eth-server0            
ovs-vsctl set interface eth-server0 external_ids:iface-id=ls2p1
                                                             
ovn-nbctl ls-add ls1                                         
ovn-nbctl lsp-add ls1 ls1lr1                                 
ovn-nbctl lsp-set-type ls1lr1 router                                             
ovn-nbctl lsp-set-options ls1lr1 router-port=lr1ls1
ovn-nbctl lsp-set-addresses ls1lr1 "00:01:03:0d:ff:01 192.168.1.254 2000::a"
                                                             
ovn-nbctl lsp-add ls1 ls1p1                                  
ovn-nbctl lsp-set-addresses ls1p1 "00:01:02:03:01:01 192.168.1.1 2000::1"
                                        
ovs-vsctl add-port br-int eth-client0 -- set interface eth-client0 type=internal
ip netns add client0                      
ip link set eth-client0 netns client0          
ip netns exec client0 ip link set eth-client0 up             
ip netns exec client0 ip link set eth-client0 address 00:01:02:03:01:01
ip netns exec client0 ip addr add 192.168.1.1/24 dev eth-client0
ip netns exec client0 ip route add default via 192.168.1.254 dev eth-client0
ip netns exec client0 ip -6 addr add 2000::1/64 dev eth-client0
ip netns exec client0 ip a

ip netns exec client0 ip -6 route add default via 2000::a dev eth-client0
ovs-vsctl set interface eth-client0 external_ids:iface-id=ls1p1
ovn-nbctl lrp-set-gateway-chassis lr1ls2 hv0
ovn-nbctl set logical_router_port lr1ls2 options:gateway_mtu=1300
ovn-nbctl set logical_router_port lr1ls2 options:gateway_mtu_bypass=icmp

ip netns exec client0 ping 192.168.2.1 -c 1
ip netns exec client0 ping 192.168.2.1 -c 1 -s 1400
ovn-nbctl --wait=hv set logical_router_port lr1ls2 options:gateway_mtu_bypass=trytest
ip netns exec client0 ping 192.168.2.1 -c 1 
ip netns exec client0 ping 192.168.2.1 -c 1 -s 1400 -w 1 -W 1
ip netns exec client0 ping 192.168.2.1 -c 1 -s 1400 -w 1 -W 1
ip netns exec client0 ping 192.168.2.1 -c 1 -s 1400 -w 1 -W 1
ovn-nbctl --wait=hv set logical_router_port lr1ls2 options:gateway_mtu_bypass=tcp
ip netns exec client0 ip route flush cache
ip netns exec client0 ping 192.168.2.1 -c 1
ip netns exec client0 ping 192.168.2.1 -c 1 -s 1400 -w 1 -W 1
ip netns exec client0 ping 192.168.2.1 -c 1 -s 1400 -w 1 -W 1
ip netns exec client0 ping 192.168.2.1 -c 1 -s 1400 -w 1 -W 1

ovn-nbctl --wait=hv remove logical_router_port lr1ls2 options gateway_mtu_bypass
ip netns exec client0 ip route flush cache
ip netns exec client0 ping 192.168.2.1 -c 1
ip netns exec client0 ping 192.168.2.1 -c 1 -s 1400 -w 1 -W 1
ip netns exec client0 ping 192.168.2.1 -c 1 -s 1400 -w 1 -W 1
ip netns exec client0 ping 192.168.2.1 -c 1 -s 1400 -w 1 -W 1
ovn-nbctl --wait=hv remove logical_router_port lr1ls2 options gateway_mtu
ip netns exec client0 ip route flush cache
ip netns exec client0 ping 192.168.2.1 -c 1
ip netns exec client0 ping 192.168.2.1 -c 1 -s 1400 -w 1 -W 1
ip netns exec client0 ping 192.168.2.1 -c 1 -s 1400 -w 1 -W 1
ip netns exec client0 ping 192.168.2.1 -c 1 -s 1400 -w 1 -W 1
ovn-nbctl set logical_router_port lr1ls2 options:gateway_mtu_bypass=icmp
ip netns exec client0 ip route flush cache
ip netns exec client0 ping 192.168.2.1 -c 1
ip netns exec client0 ping 192.168.2.1 -c 1 -s 1400 -w 1 -W 1
ip netns exec client0 ping 192.168.2.1 -c 1 -s 1400 -w 1 -W 1
ip netns exec client0 ping 192.168.2.1 -c 1 -s 1400 -w 1 -W 1


Verified on ovn-2021-21.12.0-11:

[root@wsfd-advnetlab16 bz2011779]# rpm -qa | grep -E "openvswitch2.16|ovn-2021"
openvswitch2.16-2.16.0-35.el8fdp.x86_64
ovn-2021-host-21.12.0-11.el8fdp.x86_64
python3-openvswitch2.16-2.16.0-35.el8fdp.x86_64
ovn-2021-central-21.12.0-11.el8fdp.x86_64
ovn-2021-21.12.0-11.el8fdp.x86_64

+ ovn-nbctl set logical_router_port lr1ls2 options:gateway_mtu=1300
+ ovn-nbctl set logical_router_port lr1ls2 options:gateway_mtu_bypass=icmp
+ ip netns exec client0 ping 192.168.2.1 -c 1               
PING 192.168.2.1 (192.168.2.1) 56(84) bytes of data.
64 bytes from 192.168.2.1: icmp_seq=1 ttl=63 time=1031 ms
                                                             
--- 192.168.2.1 ping statistics ---              
1 packets transmitted, 1 received, 0% packet loss, time 0ms    
rtt min/avg/max/mdev = 1030.755/1030.755/1030.755/0.000 ms
+ ip netns exec client0 ping 192.168.2.1 -c 1 -s 1400       
PING 192.168.2.1 (192.168.2.1) 1400(1428) bytes of data.
1408 bytes from 192.168.2.1: icmp_seq=1 ttl=63 time=0.093 ms
                                                           
--- 192.168.2.1 ping statistics ---              
1 packets transmitted, 1 received, 0% packet loss, time 0ms                        
rtt min/avg/max/mdev = 0.093/0.093/0.093/0.000 ms

<=== pmtu doesn't work when gateway_mtu and gateway_mtu_bypass are set at the same time

+ ovn-nbctl --wait=hv set logical_router_port lr1ls2 options:gateway_mtu_bypass=trytest
+ ip netns exec client0 ping 192.168.2.1 -c 1        
PING 192.168.2.1 (192.168.2.1) 56(84) bytes of data.      
64 bytes from 192.168.2.1: icmp_seq=1 ttl=63 time=0.088 ms
                                   
--- 192.168.2.1 ping statistics ---                        
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.088/0.088/0.088/0.000 ms              
+ ip netns exec client0 ping 192.168.2.1 -c 1 -s 1400 -w 1 -W 1
PING 192.168.2.1 (192.168.2.1) 1400(1428) bytes of data.         
From 192.168.1.254 icmp_seq=1 Frag needed and DF set (mtu = 1300)
                                   
--- 192.168.2.1 ping statistics ---                                     
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms
                                                                
+ ip netns exec client0 ping 192.168.2.1 -c 1 -s 1400 -w 1 -W 1 
PING 192.168.2.1 (192.168.2.1) 1400(1428) bytes of data.    

--- 192.168.2.1 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms
                                                 
+ ip netns exec client0 ping 192.168.2.1 -c 1 -s 1400 -w 1 -W 1
PING 192.168.2.1 (192.168.2.1) 1400(1428) bytes of data.
1408 bytes from 192.168.2.1: icmp_seq=1 ttl=63 time=0.793 ms

--- 192.168.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.793/0.793/0.793/0.000 ms
+ ovn-nbctl --wait=hv set logical_router_port lr1ls2 options:gateway_mtu_bypass=tcp
+ ip netns exec client0 ip route flush cache
+ ip netns exec client0 ping 192.168.2.1 -c 1
PING 192.168.2.1 (192.168.2.1) 56(84) bytes of data.
64 bytes from 192.168.2.1: icmp_seq=1 ttl=63 time=0.087 ms

--- 192.168.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.087/0.087/0.087/0.000 ms
+ ip netns exec client0 ping 192.168.2.1 -c 1 -s 1400 -w 1 -W 1
PING 192.168.2.1 (192.168.2.1) 1400(1428) bytes of data. 
From 192.168.1.254 icmp_seq=1 Frag needed and DF set (mtu = 1300)

--- 192.168.2.1 ping statistics ---
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms

+ ip netns exec client0 ping 192.168.2.1 -c 1 -s 1400 -w 1 -W 1
PING 192.168.2.1 (192.168.2.1) 1400(1428) bytes of data. 
1408 bytes from 192.168.2.1: icmp_seq=1 ttl=63 time=1.41 ms

--- 192.168.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.410/1.410/1.410/0.000 ms
+ ip netns exec client0 ping 192.168.2.1 -c 1 -s 1400 -w 1 -W 1
PING 192.168.2.1 (192.168.2.1) 1400(1428) bytes of data. 
1408 bytes from 192.168.2.1: icmp_seq=1 ttl=63 time=0.105 ms

--- 192.168.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.105/0.105/0.105/0.000 ms
+ ovn-nbctl --wait=hv remove logical_router_port lr1ls2 options gateway_mtu_bypass
+ ip netns exec client0 ip route flush cache
+ ip netns exec client0 ping 192.168.2.1 -c 1
PING 192.168.2.1 (192.168.2.1) 56(84) bytes of data.
64 bytes from 192.168.2.1: icmp_seq=1 ttl=63 time=0.086 ms

--- 192.168.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.086/0.086/0.086/0.000 ms
+ ip netns exec client0 ping 192.168.2.1 -c 1 -s 1400 -w 1 -W 1
PING 192.168.2.1 (192.168.2.1) 1400(1428) bytes of data. 
From 192.168.1.254 icmp_seq=1 Frag needed and DF set (mtu = 1300)

--- 192.168.2.1 ping statistics ---
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms

+ ip netns exec client0 ping 192.168.2.1 -c 1 -s 1400 -w 1 -W 1
PING 192.168.2.1 (192.168.2.1) 1400(1428) bytes of data. 
1408 bytes from 192.168.2.1: icmp_seq=1 ttl=63 time=0.106 ms

--- 192.168.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.106/0.106/0.106/0.000 ms
+ ip netns exec client0 ping 192.168.2.1 -c 1 -s 1400 -w 1 -W 1
PING 192.168.2.1 (192.168.2.1) 1400(1428) bytes of data. 
1408 bytes from 192.168.2.1: icmp_seq=1 ttl=63 time=0.101 ms

--- 192.168.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.101/0.101/0.101/0.000 ms
+ ovn-nbctl --wait=hv remove logical_router_port lr1ls2 options gateway_mtu
+ ip netns exec client0 ip route flush cache
+ ip netns exec client0 ping 192.168.2.1 -c 1
PING 192.168.2.1 (192.168.2.1) 56(84) bytes of data.
64 bytes from 192.168.2.1: icmp_seq=1 ttl=63 time=0.084 ms

--- 192.168.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.084/0.084/0.084/0.000 ms
+ ip netns exec client0 ping 192.168.2.1 -c 1 -s 1400 -w 1 -W 1
PING 192.168.2.1 (192.168.2.1) 1400(1428) bytes of data. 
1408 bytes from 192.168.2.1: icmp_seq=1 ttl=63 time=0.091 ms

--- 192.168.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.091/0.091/0.091/0.000 ms
+ ip netns exec client0 ping 192.168.2.1 -c 1 -s 1400 -w 1 -W 1
PING 192.168.2.1 (192.168.2.1) 1400(1428) bytes of data. 
1408 bytes from 192.168.2.1: icmp_seq=1 ttl=63 time=0.096 ms

--- 192.168.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.096/0.096/0.096/0.000 ms
+ ip netns exec client0 ping 192.168.2.1 -c 1 -s 1400 -w 1 -W 1
PING 192.168.2.1 (192.168.2.1) 1400(1428) bytes of data. 
1408 bytes from 192.168.2.1: icmp_seq=1 ttl=63 time=0.090 ms

--- 192.168.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.090/0.090/0.090/0.000 ms
+ ovn-nbctl set logical_router_port lr1ls2 options:gateway_mtu_bypass=icmp
+ ip netns exec client0 ip route flush cache
+ ip netns exec client0 ping 192.168.2.1 -c 1
PING 192.168.2.1 (192.168.2.1) 56(84) bytes of data.
64 bytes from 192.168.2.1: icmp_seq=1 ttl=63 time=0.082 ms

--- 192.168.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.082/0.082/0.082/0.000 ms
+ ip netns exec client0 ping 192.168.2.1 -c 1 -s 1400 -w 1 -W 1
PING 192.168.2.1 (192.168.2.1) 1400(1428) bytes of data. 
1408 bytes from 192.168.2.1: icmp_seq=1 ttl=63 time=0.090 ms

--- 192.168.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.090/0.090/0.090/0.000 ms
+ ip netns exec client0 ping 192.168.2.1 -c 1 -s 1400 -w 1 -W 1
PING 192.168.2.1 (192.168.2.1) 1400(1428) bytes of data. 
1408 bytes from 192.168.2.1: icmp_seq=1 ttl=63 time=0.089 ms

--- 192.168.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.089/0.089/0.089/0.000 ms
+ ip netns exec client0 ping 192.168.2.1 -c 1 -s 1400 -w 1 -W 1
PING 192.168.2.1 (192.168.2.1) 1400(1428) bytes of data. 
1408 bytes from 192.168.2.1: icmp_seq=1 ttl=63 time=0.092 ms

--- 192.168.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.092/0.092/0.092/0.000 ms

Comment 16 errata-xmlrpc 2022-02-24 17:47:39 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (ovn bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:0674