Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1936010

Summary: UDP packet fragmentation and dropped
Product: OpenShift Container Platform Reporter: Murali Krishnasamy <murali>
Component: NetworkingAssignee: Tim Rozet <trozet>
Networking sub component: ovn-kubernetes QA Contact: Anurag saxena <anusaxen>
Status: CLOSED DUPLICATE Docs Contact:
Severity: high    
Priority: high CC: aconstan, dblack, hnhan, rkhan, trozet
Version: 4.7Keywords: TestBlocker
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-03-16 15:03:05 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Murali Krishnasamy 2021-03-05 22:43:15 UTC
Description of problem: 

Running Uperf UDP workload from pod to pod across worker is dropping packets and seemed to be dropping because of packet fragmentation. 

UDP packets fragmenting when the payload size is greater than MTU size. If protocol is set to TCP we don't see any packet drop or fragmentation. 


Version-Release number of selected component (if applicable): 
OCP: 4.7.0-fc.5
Kernel version: 4.18.0-240.10.1.el8_3.x86_64
OVN version: 
Version: 0.3.0
Git commit: bc8321aa1afb6b532e73effc0d2bd3212c15a85e
Git branch: rhaos-4.7-rhel-8
Go version: go1.15.5
Build date: 2021-01-30
OS/Arch: linux amd64

OVS version:
$ ovs-vsctl --version
ovs-vsctl (Open vSwitch) 2.13.2
DB Schema 8.2.0

How reproducible:
Frequently on any environment

Steps to Reproduce:
1. Install OCP cluster 4.7.0-fc.5
2. Run UPerf workload (We are using benchmark-operator)
3. UDP protocol and Payload size more than MTU

Actual results:
Packets fragmenting and RX drops at the NIC level

Expected results:
No Fragmentation and no drops as in TCP 

Additional info:
$ ethtool -i ens2f1
driver: i40e
version: 2.8.20-k
firmware-version: 7.10 0x800051ac 19.0.12
expansion-rom-version: 
bus-info: 0000:62:00.1
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes

Cluster network:
status:
  clusterNetwork:
  - cidr: 10.128.0.0/14
    hostPrefix: 23
  clusterNetworkMTU: 8900
  networkType: OVNKubernetes
  serviceNetwork:
  - 172.30.0.0/16

$ ifconfig ens2f1
ens2f1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 9000
        ether 40:a6:b7:00:47:81  txqueuelen 1000  (Ethernet)
        RX packets 752005677  bytes 5000469924153 (4.5 TiB)
        RX errors 0  dropped 83514476  overruns 0  frame 0
        TX packets 156543237  bytes 621736118212 (579.0 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

$ ethtool -k ens2f1
Features for ens2f1:
rx-checksumming: on
tx-checksumming: on
	tx-checksum-ipv4: on
	tx-checksum-ip-generic: off [fixed]
	tx-checksum-ipv6: on
	tx-checksum-fcoe-crc: off [fixed]
	tx-checksum-sctp: on
scatter-gather: on
	tx-scatter-gather: on
	tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
	tx-tcp-segmentation: on
	tx-tcp-ecn-segmentation: on
	tx-tcp-mangleid-segmentation: off
	tx-tcp6-segmentation: on
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: on
receive-hashing: on
highdma: on
rx-vlan-filter: on [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: on
tx-gre-csum-segmentation: on
tx-ipxip4-segmentation: on
tx-ipxip6-segmentation: on
tx-udp_tnl-segmentation: on
tx-udp_tnl-csum-segmentation: on
tx-gso-partial: on
tx-sctp-segmentation: off [fixed]
tx-esp-segmentation: off [fixed]
tx-udp-segmentation: on
tls-hw-rx-offload: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off
hw-tc-offload: on
esp-hw-offload: off [fixed]
esp-tx-csum-hw-offload: off [fixed]
rx-udp_tunnel-port-offload: on
tls-hw-tx-offload: off [fixed]
rx-gro-hw: off [fixed]
tls-hw-record: off [fixed]

Comment 1 Alexander Constantinescu 2021-03-11 16:46:38 UTC
It seems like this is a dupe of https://bugzilla.redhat.com/show_bug.cgi?id=1927047

I am not sure though so I will hold off doing so until I get trozet's input.

Comment 3 Tim Rozet 2021-03-16 15:03:05 UTC
There's not a lot of info in this bugzilla, but I'm fairly certain it is the same as 1927047.

*** This bug has been marked as a duplicate of bug 1927047 ***