Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 2186488

Summary: [17.1][HWOFFLOAD][OVN] Flows reuse with TCP and UDP Traffic
Product: Red Hat OpenStack Reporter: Miguel Angel Nieto <mnietoji>
Component: openvswitchAssignee: OvS team <ovs-bugzilla>
Status: CLOSED NOTABUG QA Contact: Eran Kuris <ekuris>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 17.1 (Wallaby)CC: apevec, chrisw, dceara, echaudro, hakhande, nusiddiq, vkhitrin
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-04-19 13:07:29 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Miguel Angel Nieto 2023-04-13 13:18:22 UTC 
Description of problem:

Sending TCP+UDP traffic with same ips/port I have realized than only one packet is captured in representor port and only one flow is created.

I think that they are 2 different flows and 2 flows should be created

Test reproduction
1. Create 2 vms with offload ports, same issue for either vlan or geneve
2. I check there are no flows
sudo ovs-appctl dpctl/dump-flows -m | grep "fa:16:3e:90:dc:f5" | grep "fa:16:3e:e4:48:b9"

2. I start tcpdump on representor port
3. Start traffic injection. In this case I am using iperf3 and sending udp traffic . It will also send TCP traffic for statistics. It can be reproduced too with scapy sending TCP+UDP packets
iperf3 -s -B 30.30.220.136 -p 9244
iperf3 -c 30.30.220.136 -T s2 -p 9244 -t  10000 -u

4. In the capture, only TCP packets are captured. After first TCP packets are sent, flows are created and following TCP or UDP packets are offloaded. 
compute0
[tripleo-admin@computehwoffload-r730 ~]$ sudo tcpdump -i enp4s0f1np1_0 -nne  ether host fa:16:3e:90:dc:f5 and  ether host fa:16:3e:e4:48:b9
dropped privs to tcpdump
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on enp4s0f1np1_0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
11:01:19.560161 fa:16:3e:e4:48:b9 > fa:16:3e:90:dc:f5, ethertype IPv4 (0x0800), length 74: 30.30.220.118.44634 > 30.30.220.136.9244: Flags [S], seq 2648218620, win 62720, options [mss 8960,sackOK,TS val 2044396347 ecr 0,nop,wscale 7], length 0
11:01:19.560238 fa:16:3e:90:dc:f5 > fa:16:3e:e4:48:b9, ethertype IPv4 (0x0800), length 74: 30.30.220.136.9244 > 30.30.220.118.44634: Flags [S.], seq 451556823, ack 2648218621, win 62636, options [mss 8960,sackOK,TS val 299047599 ecr 2044396347,nop,wscale 7], length 0
11:01:24.956545 fa:16:3e:e4:48:b9 > fa:16:3e:90:dc:f5, ethertype ARP (0x0806), length 56: Request who-has 30.30.220.136 tell 30.30.220.118, length 42
11:01:24.956592 fa:16:3e:90:dc:f5 > fa:16:3e:e4:48:b9, ethertype ARP (0x0806), length 52: Reply 30.30.220.136 is-at fa:16:3e:90:dc:f5, length 38
11:01:24.974485 fa:16:3e:90:dc:f5 > fa:16:3e:e4:48:b9, ethertype ARP (0x0806), length 52: Request who-has 30.30.220.118 tell 30.30.220.136, length 38
11:01:24.974991 fa:16:3e:e4:48:b9 > fa:16:3e:90:dc:f5, ethertype ARP (0x0806), length 56: Reply 30.30.220.118 is-at fa:16:3e:e4:48:b9, length 42

compute1
[tripleo-admin@computehwoffload-r740 ~]$ sudo tcpdump -i ens2f1np1_2 -nne  ether host fa:16:3e:90:dc:f5 and  ether host fa:16:3e:e4:48:b9                                                                   [9/772]
dropped privs to tcpdump                                                                                                                                                                                           
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode                                                                                                                                          
listening on ens2f1np1_2, link-type EN10MB (Ethernet), snapshot length 262144 bytes                                                                                                                                
11:01:19.558706 fa:16:3e:e4:48:b9 > fa:16:3e:90:dc:f5, ethertype IPv4 (0x0800), length 74: 30.30.220.118.44634 > 30.30.220.136.9244: Flags [S], seq 2648218620, win 62720, options [mss 8960,sackOK,TS val 20443963
47 ecr 0,nop,wscale 7], length 0                                                                                                                                                                                   
11:01:19.560721 fa:16:3e:90:dc:f5 > fa:16:3e:e4:48:b9, ethertype IPv4 (0x0800), length 74: 30.30.220.136.9244 > 30.30.220.118.44634: Flags [S.], seq 451556823, ack 2648218621, win 62636, options [mss 8960,sackOK
,TS val 299047599 ecr 2044396347,nop,wscale 7], length 0                                                                                                                                                           
11:01:24.955605 fa:16:3e:e4:48:b9 > fa:16:3e:90:dc:f5, ethertype ARP (0x0806), length 52: Request who-has 30.30.220.136 tell 30.30.220.118, length 38                                                              
11:01:24.956618 fa:16:3e:90:dc:f5 > fa:16:3e:e4:48:b9, ethertype ARP (0x0806), length 56: Reply 30.30.220.136 is-at fa:16:3e:90:dc:f5, length 42                                                                   
11:01:24.974475 fa:16:3e:90:dc:f5 > fa:16:3e:e4:48:b9, ethertype ARP (0x0806), length 56: Request who-has 30.30.220.118 tell 30.30.220.136, length 42
11:01:24.974506 fa:16:3e:e4:48:b9 > fa:16:3e:90:dc:f5, ethertype ARP (0x0806), length 52: Reply 30.30.220.118 is-at fa:16:3e:e4:48:b9, length 38

5. Display flows while still injecting traffic, only 1 flow is created
compute0
[tripleo-admin@computehwoffload-r730 ~]$ sudo ovs-appctl dpctl/dump-flows -m | grep "fa:16:3e:90:dc:f5" | grep "fa:16:3e:e4:48:b9"
ufid:cbfb3fb5-3420-4529-af34-02925f7cd9f3, skb_priority(0/0),skb_mark(0/0),ct_state(0/0),ct_zone(0/0),ct_mark(0/0x2),ct_label(0/0),recirc_id(0),dp_hash(0/0),in_port(mx-bond),packet_type(ns=0/0,id=0/0),eth(src=fa:16:3e:e4:48:b9,dst=fa:16:3e:90:dc:f5),eth_type(0x8100),vlan(vid=174,pcp=0),encap(eth_type(0x0800),ipv4(src=0.0.0.0/0.0.0.0,dst=0.0.0.0/0.0.0.0,proto=0/0,tos=0/0,ttl=0/0,frag=no)), packets:277, bytes:2419003, used:0.180s, offloaded:yes, dp:tc, actions:pop_vlan,enp4s0f1np1_0


compute1
[tripleo-admin@computehwoffload-r740 ~]$ sudo ovs-appctl dpctl/dump-flows -m | grep "fa:16:3e:90:dc:f5" | grep "fa:16:3e:e4:48:b9"
ufid:92c9fa8f-a6f9-4b01-9b42-88651e0a0b9e, skb_priority(0/0),skb_mark(0/0),ct_state(0/0),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),recirc_id(0),dp_hash(0/0),in_port(ens2f1np1_2),packet_type(ns=0/0,id=0/0),eth(src=fa:16:3e:e4:48:b9,dst=fa:16:3e:90:dc:f5),eth_type(0x0800),ipv4(src=0.0.0.0/0.0.0.0,dst=30.30.220.128/255.255.255.192,proto=0/0,tos=0/0,ttl=0/0,frag=no), packets:196, bytes:1691601, used:0.660s, offloaded:yes, dp:tc, actions:push_vlan(vid=174,pcp=0),mx-bond
ufid:a424c11d-b3fa-42d2-9654-40366920ee62, skb_priority(0/0),skb_mark(0/0),ct_state(0/0),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),recirc_id(0),dp_hash(0/0),in_port(ens2f1np1_2),packet_type(ns=0/0,id=0/0),eth(src=fa:16:3e:e4:48:b9,dst=fa:16:3e:90:dc:f5),eth_type(0x0806),arp(sip=0.0.0.0/0.0.0.0,tip=30.30.220.136,op=1,sha=00:00:00:00:00:00/00:00:00:00:00:00,tha=00:00:00:00:00:00/00:00:00:00:00:00), packets:0, bytes:0, used:never, dp:tc, actions:push_vlan(vid=174,pcp=0),mx-bond
ufid:4f7fdc58-4949-4e2a-96db-986c368e2a60, skb_priority(0/0),skb_mark(0/0),ct_state(0/0),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),recirc_id(0),dp_hash(0/0),in_port(ens2f1np1_2),packet_type(ns=0/0,id=0/0),eth(src=fa:16:3e:e4:48:b9,dst=fa:16:3e:90:dc:f5),eth_type(0x0806),arp(sip=0.0.0.0/0.0.0.0,tip=30.30.220.136,op=2,sha=00:00:00:00:00:00/00:00:00:00:00:00,tha=00:00:00:00:00:00/00:00:00:00:00:00), packets:0, bytes:0, used
:never, dp:tc, actions:push_vlan(vid=174,pcp=0),mx-bond
ufid:04292b02-6d25-45aa-92ad-74441b0d95ba, skb_priority(0/0),skb_mark(0/0),ct_state(0/0),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),recirc_id(0),dp_hash(0/0),in_port(mx-bond),packet_type(ns=0/0,id=0/0),eth(src=fa:1
6:3e:90:dc:f5,dst=fa:16:3e:e4:48:b9),eth_type(0x8100),vlan(vid=174,pcp=0),encap(eth_type(0x0806),arp(sip=0.0.0.0/0.0.0.0,tip=0.0.0.0/0.0.0.0,op=0/0,sha=00:00:00:00:00:00/00:00:00:00:00:00,tha=00:00:00:00:00:00/0
0:00:00:00:00:00)), packets:0, bytes:0, used:never, dp:tc, actions:pop_vlan,ens2f1np1_2
ufid:17848894-89d2-4045-8a6c-ee29ba649a06, skb_priority(0/0),skb_mark(0/0),ct_state(0/0),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),recirc_id(0),dp_hash(0/0),in_port(mx-bond),packet_type(ns=0/0,id=0/0),eth(src=fa:1
6:3e:90:dc:f5,dst=fa:16:3e:e4:48:b9),eth_type(0x8100),vlan(vid=174,pcp=0),encap(eth_type(0x0806),arp(sip=0.0.0.0/0.0.0.0,tip=0.0.0.0/0.0.0.0,op=0/0,sha=00:00:00:00:00:00/00:00:00:00:00:00,tha=00:00:00:00:00:00/0
0:00:00:00:00:00)), packets:0, bytes:0, used:never, dp:tc, actions:pop_vlan,ens2f1np1_2


6. stop iperf3
7. show iperf3 stats
server
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-70.00  sec  8.76 MBytes  1.05 Mbits/sec  0.016 ms  0/1026 (0%)  receiver

client
s2:  [ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
s2:  [  5]   0.00-70.03  sec  8.76 MBytes  1.05 Mbits/sec  0.000 ms  0/1026 (0%)  sender
s2:  [  5]   0.00-70.03  sec  0.00 Bytes  0.00 bits/sec  0.000 ms  0/0 (0%)  receiver



Version-Release number of selected component (if applicable):
RHOS-17.1-RHEL-9-20230404.n.1

How reproducible:
1. Deploy offload templates for ovn
2. Create 2 vms with either geneve or vlan offloaded ports
3. Follow the steps in the description


Actual results:
Only 1 flow is created for both UDP and TCP traffic


Expected results:
2 flows should be created, one for UDP and the other one for TCP


Additional info:
Comment 1 Miguel Angel Nieto 2023-04-13 15:06:26 UTC 
More info:

No security groups are used
I reproduced it with mellanox CX5

ovs and ovn versions
openvswitch3.1-3.1.0-13.el9fdp.x86_64
ovn22.12-22.12.0-37.el9fdp.x86_64
Comment 4 Eelco Chaudron 2023-04-17 11:13:28 UTC 
From an OVS perspective it will optimize the datapath flows as much as possible, so if it can combine the dp flows in such a way it does not violate the programmed OpenFlow rules, it will do this. And this is what happens in your case, so it looks like the above is working per design.