Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 2133466

Summary: [RHOSP16.1] ICMP/TCP/UDP Flows Are Not Offloaded In OVS Based Deployment
Product: Red Hat OpenStack Reporter: Miguel Angel Nieto <mnietoji>
Component: openvswitchAssignee: Haresh Khandelwal <hakhande>
Status: CLOSED WONTFIX QA Contact: Eran Kuris <ekuris>
Severity: medium Docs Contact:
Priority: medium    
Version: 16.1 (Train)CC: apevec, averdagu, bcafarel, chrisw, fleitner, gurpsing, hakhande, mleitner, ralonsoh, scohen, spower, vkhitrin
Target Milestone: ---Keywords: Bugfix, Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-10-14 11:20:07 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Miguel Angel Nieto 2022-10-10 14:44:39 UTC 
Description of problem:

Flows are not offloaded, this error looks similar to this other one in 16.2
https://bugzilla.redhat.com/show_bug.cgi?id=2092946

Running performance using vlan provider network, these are the flows:

ufid:9936a152-ab74-4555-b9b2-7dbdad3a0a9a, skb_priority(0/0),skb_mark(0/0),ct_state(0/0x3),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),recirc_id(0),dp_hash(0/0),in_port(enp4s0f1_2),packet_type(ns=0/0,id=0/0),eth(src=fa:16:3e:17:ae:a6,dst=fa:16:3e:67:4f:7b),eth_type(0x0800),ipv4(src=0.0.0.0/0.0.0.0,dst=0.0.0.0/0.0.0.0,proto=0/0,tos=0/0,ttl=0/0,frag=no), packets:38867722, bytes:1787915212, used:0.000s, dp:tc, actions:push_vlan(vid=127,pcp=0),mx-bond
 
ufid:9f2da1b6-8a4f-4d67-b8e5-68a3709cbcff, skb_priority(0/0),skb_mark(0/0),ct_state(0/0x3),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),recirc_id(0),dp_hash(0/0),in_port(enp4s0f1_3),packet_type(ns=0/0,id=0/0),eth(src=fa:16:3e:4d:eb:97,dst=fa:16:3e:51:6c:e9),eth_type(0x0800),ipv4(src=0.0.0.0/0.0.0.0,dst=0.0.0.0/0.0.0.0,proto=0/0,tos=0/0,ttl=0/0,frag=no), packets:36740135, bytes:1690046210, used:0.000s, dp:tc, actions:push_vlan(vid=126,pcp=0),mx-bond
 
ufid:c3afd70b-301c-4593-b218-3ac59db692f4, skb_priority(0/0),skb_mark(0/0),ct_state(0/0),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),recirc_id(0),dp_hash(0/0),in_port(mx-bond),packet_type(ns=0/0,id=0/0),eth(src=fa:16:3e:4d:eb:97,dst=fa:16:3e:51:6c:e9),eth_type(0x8100),vlan(vid=127,pcp=0),encap(eth_type(0x0800),ipv4(src=0.0.0.0/0.0.0.0,dst=0.0.0.0/0.0.0.0,proto=0/0,tos=0/0,ttl=0/0,frag=no)), packets:32170024, bytes:1930169184, used:0.680s, offloaded:yes, dp:tc, actions:pop_vlan,enp4s0f1_2
 
ufid:7577598d-a012-437e-af22-d36d1ed82e75, skb_priority(0/0),skb_mark(0/0),ct_state(0/0),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),recirc_id(0),dp_hash(0/0),in_port(mx-bond),packet_type(ns=0/0,id=0/0),eth(src=fa:16:3e:17:ae:a6,dst=fa:16:3e:67:4f:7b),eth_type(0x8100),vlan(vid=126,pcp=0),encap(eth_type(0x0800),ipv4(src=0.0.0.0/0.0.0.0,dst=0.0.0.0/0.0.0.0,proto=0/0,tos=0/0,ttl=0/0,frag=no)), packets:33861333, bytes:2031671104, used:0.680s, offloaded:yes, dp:tc, actions:pop_vlan,enp4s0f1_3

I think first 2 flows should have offloaded:yes

I have tried to add in my templates:
  ComputeSriovOffloadExtraConfig:
    neutron::config::ovs_agent_config:
      "ovs/disable_packet_marking":
        value: true

but this is not populated in file 
/var/lib/config-data/puppet-generated/neutron/etc/neutron/plugins/ml2/openvswitch_agent.ini



Version-Release number of selected component (if applicable):
RHOS-16.1-RHEL-8-20220917.n.0

How reproducible:
Run performance in a ml2-ovs environment using vlan provider network. Performance is very low. 
Execute "ovs-appctl dpctl/dump-flows -m" while running performance in a compute


Actual results: Flows not offloaded, low performance


Expected results: Flows offloaded, higher performance


Additional info:
Comment 11 Miguel Angel Nieto 2022-10-24 08:17:16 UTC 
Hi, I have configured root.eng.tlv2.redhat.com

There is a tmux session in undercloud with 4 windows:
1. vm1
2. vm2
3. compute0
4. compute1

Instead of using trex as when i created the bz, I am using iperf to reproduce it. I run these commands in vms:
iperf3 -s -B 10.10.178.112 -p 8298
iperf3 -c 10.10.178.112 -T s2 -p 8298 -t 10000

I get the flows in computes, only one flow is offloaded
[root@computehwoffload-r740-0 heat-admin]# sudo ovs-appctl dpctl/dump-flows -m 
ufid:c05db900-f2ed-4d6c-bb84-53fd2aaba367, skb_priority(0/0),skb_mark(0/0),ct_state(0/0x3),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),recirc_id(0),dp_hash(0/0),in_port(ens2f1_4),packet_type(ns=0/0,id=0/0),eth(src=fa:16:3e:9e:a8:6a,dst=fa:16:3e:a5:fc:1f),eth_type(0x0800),ipv4(src=0.0.0.0/0.0.0.0,dst=0.0.0.0/0.0.0.0,proto=0/0,tos=0/0,ttl=0/0,frag=no), packets:4168554, bytes:6039661208, used:0.000s, dp:tc, actions:push_vlan(vid=178,pcp=0),mx-bond
ufid:7c7af88c-a39a-4489-80da-a8241f333477, skb_priority(0/0),skb_mark(0/0),ct_state(0/0),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),recirc_id(0),dp_hash(0/0),in_port(mx-bond),packet_type(ns=0/0,id=0/0),eth(src=fa:16:3e:a5:fc:1f,dst=fa:16:3e:9e:a8:6a),eth_type(0x8100),vlan(vid=178,pcp=0),encap(eth_type(0x0800),ipv4(src=0.0.0.0/0.0.0.0,dst=0.0.0.0/0.0.0.0,proto=0/0,tos=0/0,ttl=0/0,frag=no)), packets:613280, bytes:40476492, used:0.580s, offloaded:yes, dp:tc, actions:pop_vlan,ens2f1_4
ufid:cd416ca5-f1a5-4488-af5c-8a0880b86acf, skb_priority(0/0),tunnel(tun_id=0x7937,src=10.10.171.163,dst=10.10.171.108,ttl=0/0,tp_dst=4789,flags(+key)),skb_mark(0/0),ct_state(0/0),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),recirc_id(0),dp_hash(0/0),in_port(vxlan_sys_4789),packet_type(ns=0/0,id=0/0),eth(src=00:00:00:00:00:00/00:00:00:00:00:00,dst=00:00:00:00:00:00/00:00:00:00:00:00),eth_type(0x0800),ipv4(src=0.0.0.0/0.0.0.0,dst=0.0.0.0/0.0.0.0,proto=0/0,tos=0/0,ttl=0/0,frag=no), packets:116537, bytes:4661480, used:0.660s, dp:tc, actions:drop
ufid:4f1cd0e7-eefe-4a42-b446-68149ac75b36, recirc_id(0),dp_hash(0/0),skb_priority(0/0),in_port(eno3),skb_mark(0/0),ct_state(0/0x20),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),eth(src=fa:16:3e:c2:40:28,dst=fa:16:3e:df:e2:41),eth_type(0x8100),vlan(vid=174,pcp=0),encap(eth_type(0x0800),ipv4(src=0.0.0.0/0.0.0.0,dst=0.0.0.0/0.0.0.0,proto=6,tos=0/0,ttl=0/0,frag=no),tcp(src=0/0,dst=0/0),tcp_flags(0/0)), packets:2943, bytes:6658639, used:0.626s, flags:SFP., dp:ovs, actions:pop_vlan,ct(zone=11),recirc(0x5e7)
ufid:48849008-c64c-4f8d-83d2-38ccb635df4a, recirc_id(0x5e7),dp_hash(0/0),skb_priority(0/0),in_port(eno3),skb_mark(0/0),ct_state(0x2/0xe),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),eth(src=00:00:00:00:00:00/00:00:00:00:00:00,dst=00:00:00:00:00:00/00:00:00:00:00:00),eth_type(0x0800),ipv4(src=0.0.0.0/0.0.0.0,dst=0.0.0.0/0.0.0.0,proto=6,tos=0/0,ttl=0/0,frag=no),tcp(src=0/0,dst=22),tcp_flags(0/0), packets:1242, bytes:96641, used:0.626s, flags:P., dp:ovs, actions:tap9b3ec30d-cd
ufid:5e8c3f06-d997-40ba-b5b4-b1246a2da4e2, recirc_id(0),dp_hash(0/0),skb_priority(0/0),in_port(tap9b3ec30d-cd),skb_mark(0/0),ct_state(0/0x20),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),eth(src=fa:16:3e:df:e2:41,dst=00:00:00:00:00:00/00:00:00:00:00:00),eth_type(0x0800),ipv4(src=10.10.174.172,dst=0.0.0.0/0.0.0.0,proto=6,tos=0/0,ttl=0/0,frag=no),tcp(src=0/0,dst=0/0),tcp_flags(0/0), packets:2070, bytes:254440, used:0.626s, flags:SFP., dp:ovs, actions:ct(zone=11),recirc(0x5e8)
ufid:3bf0ed4e-cbe3-4408-a6a8-cec52752da4f, recirc_id(0x5e8),dp_hash(0/0),skb_priority(0/0),in_port(tap9b3ec30d-cd),skb_mark(0/0),ct_state(0x2a/0x3f),ct_zone(0xb),ct_mark(0),ct_label(0/0),eth(src=fa:16:3e:df:e2:41,dst=fa:16:3e:c2:40:28),eth_type(0x0800),ipv4(src=0.0.0.0/0.0.0.0,dst=0.0.0.0/0.0.0.0,proto=0/0,tos=0/0,ttl=0/0,frag=no), packets:1026, bytes:184737, used:0.627s, flags:P., dp:ovs, actions:push_vlan(vid=11,pcp=0),br-int,pop_vlan,push_vlan(vid=174,pcp=0),eno3


[root@computehwoffload-r730-0 heat-admin]# sudo ovs-appctl dpctl/dump-flows -m
ufid:8578dcf5-ead9-4357-8648-75bc8ab77beb, skb_priority(0/0),skb_mark(0/0),ct_state(0/0x3),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),recirc_id(0),dp_hash(0/0),in_port(enp4s0f1_9),packet_type(ns=0/0,id=0/0),eth(src=fa:16:3e:a5:fc:1f,dst=fa:16:3e:9e:a8:6a),eth_type(0x0800),ipv4(src=0.0.0.0/0.0.0.0,dst=0.0.0.0/0.0.0.0,proto=0/0,tos=0/0,ttl=0/0,frag=no), packets:3768230, bytes:195947972, used:0.000s, dp:tc, actions:push_vlan(vid=178,pcp=0),mx-bond
ufid:70803db5-1a6c-415e-8a42-8651a81b0b2e, skb_priority(0/0),tunnel(tun_id=0x7937,src=10.10.171.163,dst=10.10.171.133,ttl=0/0,tp_dst=4789,flags(+key)),skb_mark(0/0),ct_state(0/0),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),recirc_id(0),dp_hash(0/0),in_port(vxlan_sys_4789),packet_type(ns=0/0,id=0/0),eth(src=00:00:00:00:00:00/00:00:00:00:00:00,dst=00:00:00:00:00:00/00:00:00:00:00:00),eth_type(0x0800),ipv4(src=0.0.0.0/0.0.0.0,dst=0.0.0.0/0.0.0.0,proto=0/0,tos=0/0,ttl=0/0,frag=no), packets:116436, bytes:4657440, used:1.420s, dp:tc, actions:drop
ufid:c3b5bf3b-54ca-4c70-ab9a-4124082e4fe2, skb_priority(0/0),skb_mark(0/0),ct_state(0/0),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),recirc_id(0),dp_hash(0/0),in_port(mx-bond),packet_type(ns=0/0,id=0/0),eth(src=fa:16:3e:9e:a8:6a,dst=fa:16:3e:a5:fc:1f),eth_type(0x8100),vlan(vid=178,pcp=0),encap(eth_type(0x0800),ipv4(src=0.0.0.0/0.0.0.0,dst=0.0.0.0/0.0.0.0,proto=0/0,tos=0/0,ttl=0/0,frag=no)), packets:22417930, bytes:33932856812, used:0.020s, offloaded:yes, dp:tc, actions:pop_vlan,enp4s0f1_9
ufid:83caba4d-febd-4381-baf2-4c1dd35c3c0a, recirc_id(0),dp_hash(0/0),skb_priority(0/0),in_port(tapa171e71f-50),skb_mark(0/0),ct_state(0/0x20),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),eth(src=fa:16:3e:d8:f9:be,dst=00:00:00:00:00:00/00:00:00:00:00:00),eth_type(0x0800),ipv4(src=10.10.174.134,dst=0.0.0.0/0.0.0.0,proto=17,tos=0/0,ttl=0/0,frag=no),udp(src=32768/0x8000,dst=0/0), packets:3, bytes:270, used:4.789s, dp:ovs, actions:ct(zone=21),recirc(0x5f4)
ufid:fb939f48-5f3e-4ebb-8957-5db2afc4d8d6, recirc_id(0x5f4),dp_hash(0/0),skb_priority(0/0),in_port(tapa171e71f-50),skb_mark(0/0),ct_state(0x2a/0x3f),ct_zone(0x15),ct_mark(0),ct_label(0/0),eth(src=fa:16:3e:d8:f9:be,dst=fa:16:3e:c2:40:28),eth_type(0x0800),ipv4(src=0.0.0.0/0.0.0.0,dst=0.0.0.0/0.0.0.0,proto=0/0,tos=0/0,ttl=0/0,frag=no), packets:1566, bytes:212037, used:0.425s, flags:P., dp:ovs, actions:push_vlan(vid=21,pcp=0),br-int,pop_vlan,push_vlan(vid=174,pcp=0),enp6s0f0
ufid:ee274290-5a00-42ea-aa5c-8443662275ab, recirc_id(0),dp_hash(0/0),skb_priority(0/0),in_port(enp6s0f0),skb_mark(0/0),ct_state(0/0x20),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),eth(src=fa:16:3e:c2:40:28,dst=fa:16:3e:d8:f9:be),eth_type(0x8100),vlan(vid=174,pcp=0),encap(eth_type(0x0800),ipv4(src=0.0.0.0/0.0.0.0,dst=0.0.0.0/0.0.0.0,proto=6,tos=0/0,ttl=0/0,frag=no),tcp(src=0/0,dst=0/0),tcp_flags(0/0)), packets:3137, bytes:6671665, used:0.424s, flags:SFP., dp:ovs, actions:pop_vlan,ct(zone=21),recirc(0x5f5)
ufid:5b1ab5da-af52-4c7e-8745-cadc6793b9e0, recirc_id(0),dp_hash(0/0),skb_priority(0/0),in_port(enp6s0f0),skb_mark(0/0),ct_state(0/0x20),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),eth(src=fa:16:3e:c2:40:28,dst=fa:16:3e:d8:f9:be),eth_type(0x8100),vlan(vid=174,pcp=0),encap(eth_type(0x0800),ipv4(src=0.0.0.0/0.0.0.0,dst=0.0.0.0/0.0.0.0,proto=17,tos=0/0,ttl=0/0,frag=no),udp(src=96/0xffe0,dst=0/0)), packets:3, bytes:282, used:4.723s, dp:ovs, actions:pop_vlan,ct(zone=21),recirc(0x5f5)
ufid:4e5805ff-fd36-46a8-adb9-f95150658a4d, recirc_id(0),dp_hash(0/0),skb_priority(0/0),in_port(tapa171e71f-50),skb_mark(0/0),ct_state(0/0x20),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),eth(src=fa:16:3e:d8:f9:be,dst=00:00:00:00:00:00/00:00:00:00:00:00),eth_type(0x0800),ipv4(src=10.10.174.134,dst=0.0.0.0/0.0.0.0,proto=6,tos=0/0,ttl=0/0,frag=no),tcp(src=0/0,dst=0/0),tcp_flags(0/0), packets:2682, bytes:291288, used:0.425s, flags:SFP., dp:ovs, actions:ct(zone=21),recirc(0x5f4)
ufid:ff625a1d-218d-4d89-a6e9-7f2702d68bf7, recirc_id(0x5f5),dp_hash(0/0),skb_priority(0/0),in_port(enp6s0f0),skb_mark(0/0),ct_state(0x2a/0x3f),ct_zone(0x15),ct_mark(0),ct_label(0/0),eth(src=00:00:00:00:00:00/00:00:00:00:00:00,dst=00:00:00:00:00:00/00:00:00:00:00:00),eth_type(0x0800),ipv4(src=0.0.0.0/0.0.0.0,dst=0.0.0.0/0.0.0.0,proto=0/0,tos=0/0,ttl=0/0,frag=no), packets:3, bytes:270, used:4.723s, dp:ovs, actions:tapa171e71f-50
ufid:eca1f831-5df2-4899-bf01-fa07601ff713, recirc_id(0x5f5),dp_hash(0/0),skb_priority(0/0),in_port(enp6s0f0),skb_mark(0/0),ct_state(0x2/0xe),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),eth(src=00:00:00:00:00:00/00:00:00:00:00:00,dst=00:00:00:00:00:00/00:00:00:00:00:00),eth_type(0x0800),ipv4(src=0.0.0.0/0.0.0.0,dst=0.0.0.0/0.0.0.0,proto=6,tos=0/0,ttl=0/0,frag=no),tcp(src=0/0,dst=22),tcp_flags(0/0), packets:1668, bytes:121309, used:0.424s, flags:P., dp:ovs, actions:tapa171e71f-50
ufid:ac0aa18b-97a6-467a-8989-046b84f80596, recirc_id(0x5f4),dp_hash(0/0),skb_priority(0/0),in_port(tapa171e71f-50),skb_mark(0/0),ct_state(0x1/0xf),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),eth(src=fa:16:3e:d8:f9:be,dst=fa:16:3e:c2:40:28),eth_type(0x0800),ipv4(src=0.0.0.0/0.0.0.0,dst=0.0.0.0/0.0.0.0,proto=0/0,tos=0/0,ttl=0/0,frag=no), packets:3, bytes:270, used:4.789s, dp:ovs, actions:ct(commit,zone=21),push_vlan(vid=21,pcp=0),br-int,pop_vlan,push_vlan(vid=174,pcp=0),enp6s0f0
Comment 12 Miguel Angel Nieto 2022-10-24 08:35:35 UTC 
If i execute tcpdump in representor port, I see all traffic all traffic in one way (not offloaded) but not in the other way (offloaded)

[root@computehwoffload-r740-0 heat-admin]# tcpdump -i ens2f1_4 -nne
08:34:20.699124 fa:16:3e:9e:a8:6a > fa:16:3e:a5:fc:1f, ethertype IPv4 (0x0800), length 65226: 10.10.178.148.39972 > 10.10.178.112.8298: Flags [.], seq 472017200:472082360, ack 1, win 229, options [nop,nop,TS val 1959365 ecr 1963340], length 65160
08:34:20.699232 fa:16:3e:9e:a8:6a > fa:16:3e:a5:fc:1f, ethertype IPv4 (0x0800), length 65226: 10.10.178.148.39972 > 10.10.178.112.8298: Flags [.], seq 472082360:472147520, ack 1, win 229, options [nop,nop,TS val 1959365 ecr 1963340], length 65160
08:34:20.699350 fa:16:3e:9e:a8:6a > fa:16:3e:a5:fc:1f, ethertype IPv4 (0x0800), length 65226: 10.10.178.148.39972 > 10.10.178.112.8298: Flags [.], seq 472147520:472212680, ack 1, win 229, options [nop,nop,TS val 1959365 ecr 1963341], length 65160
08:34:20.699462 fa:16:3e:9e:a8:6a > fa:16:3e:a5:fc:1f, ethertype IPv4 (0x0800), length 65226: 10.10.178.148.39972 > 10.10.178.112.8298: Flags [.], seq 472212680:472277840, ack 1, win 229, options [nop,nop,TS val 1959365 ecr 1963341], length 65160
08:34:20.699572 fa:16:3e:9e:a8:6a > fa:16:3e:a5:fc:1f, ethertype IPv4 (0x0800), length 65226: 10.10.178.148.39972 > 10.10.178.112.8298: Flags [.], seq 472277840:472343000, ack 1, win 229, options [nop,nop,TS val 1959365 ecr 1963341], length 65160
08:34:20.699673 fa:16:3e:9e:a8:6a > fa:16:3e:a5:fc:1f, ethertype IPv4 (0x0800), length 65226: 10.10.178.148.39972 > 10.10.178.112.8298: Flags [.], seq 472343000:472408160, ack 1, win 229, options [nop,nop,TS val 1959365 ecr 1963341], length 65160
08:34:20.699736 fa:16:3e:9e:a8:6a > fa:16:3e:a5:fc:1f, ethertype IPv4 (0x0800), length 5858: 10.10.178.148.39972 > 10.10.178.112.8298: Flags [.], seq 472408160:472413952, ack 1, win 229, options [nop,nop,TS val 1959365 ecr 1963341], length 5792
08:34:20.699743 fa:16:3e:9e:a8:6a > fa:16:3e:a5:fc:1f, ethertype IPv4 (0x0800), length 626: 10.10.178.148.39972 > 10.10.178.112.8298: Flags [P.], seq 472413952:472414512, ack 1, win 229, options [nop,nop,TS val 1959366 ecr 1963342], length 560
08:34:20.699810 fa:16:3e:9e:a8:6a > fa:16:3e:a5:fc:1f, ethertype IPv4 (0x0800), length 65226: 10.10.178.148.39972 > 10.10.178.112.8298: Flags [.], seq 472414512:472479672, ack 1, win 229, options [nop,nop,TS val 1959366 ecr 1963342], length 65160
08:34:20.699924 fa:16:3e:9e:a8:6a > fa:16:3e:a5:fc:1f, ethertype IPv4 (0x0800), length 65226: 10.10.178.148.39972 > 10.10.178.112.8298: Flags [.], seq 472479672:472544832, ack 1, win 229, options [nop,nop,TS val 1959366 ecr 1963342], length 65160
08:34:20.700030 fa:16:3e:9e:a8:6a > fa:16:3e:a5:fc:1f, ethertype IPv4 (0x0800), length 65226: 10.10.178.148.39972 > 10.10.178.112.8298: Flags [.], seq 472544832:472609992, ack 1, win 229, options [nop,nop,TS val 1959366 ecr 1963342], length 65160
08:34:20.700131 fa:16:3e:9e:a8:6a > fa:16:3e:a5:fc:1f, ethertype IPv4 (0x0800), length 65226: 10.10.178.148.39972 > 10.10.178.112.8298: Flags [.], seq 472609992:472675152, ack 1, win 229, options [nop,nop,TS val 1959366 ecr 1963342], length 65160
08:34:20.700250 fa:16:3e:9e:a8:6a > fa:16:3e:a5:fc:1f, ethertype IPv4 (0x0800), length 65226: 10.10.178.148.39972 > 10.10.178.112.8298: Flags [.], seq 472675152:472740312, ack 1, win 229, options [nop,nop,TS val 1959366 ecr 1963342], length 65160
08:34:20.700363 fa:16:3e:9e:a8:6a > fa:16:3e:a5:fc:1f, ethertype IPv4 (0x0800), length 65226: 10.10.178.148.39972 > 10.10.178.112.8298: Flags [.], seq 472740312:472805472, ack 1, win 229, options [nop,nop,TS val 1959366 ecr 1963342], length 65160
08:34:20.700465 fa:16:3e:9e:a8:6a > fa:16:3e:a5:fc:1f, ethertype IPv4 (0x0800), length 65226: 10.10.178.148.39972 > 10.10.178.112.8298: Flags [.], seq 472805472:472870632, ack 1, win 229, options [nop,nop,TS val 1959366 ecr 1963342], length 65160
08:34:20.700582 fa:16:3e:9e:a8:6a > fa:16:3e:a5:fc:1f, ethertype IPv4 (0x0800), length 65226: 10.10.178.148.39972 > 10.10.178.112.8298: Flags [.], seq 472870632:472935792, ack 1, win 229, options [nop,nop,TS val 1959366 ecr 1963342], length 65160
08:34:20.700708 fa:16:3e:9e:a8:6a > fa:16:3e:a5:fc:1f, ethertype IPv4 (0x0800), length 65226: 10.10.178.148.39972 > 10.10.178.112.8298: Flags [.], seq 472935792:473000952, ack 1, win 229, options [nop,nop,TS val 1959366 ecr 1963342], length 65160
08:34:20.700808 fa:16:3e:9e:a8:6a > fa:16:3e:a5:fc:1f, ethertype IPv4 (0x0800), length 65226: 10.10.178.148.39972 > 10.10.178.112.8298: Flags [.], seq 473000952:473066112, ack 1, win 229, options [nop,nop,TS val 1959366 ecr 1963342], length 65160
08:34:20.700926 fa:16:3e:9e:a8:6a > fa:16:3e:a5:fc:1f, ethertype IPv4 (0x0800), length 65226: 10.10.178.148.39972 > 10.10.178.112.8298: Flags [.], seq 473066112:473131272, ack 1, win 229, options [nop,nop,TS val 1959366 ecr 1963342], length 65160
08:34:20.701042 fa:16:3e:9e:a8:6a > fa:16:3e:a5:fc:1f, ethertype IPv4 (0x0800), length 65226: 10.10.178.148.39972 > 10.10.178.112.8298: Flags [.], seq 473131272:473196432, ack 1, win 229, options [nop,nop,TS val 1959366 ecr 1963342], length 65160
08:34:20.701160 fa:16:3e:9e:a8:6a > fa:16:3e:a5:fc:1f, ethertype IPv4 (0x0800), length 65226: 10.10.178.148.39972 > 10.10.178.112.8298: Flags [.], seq 473196432:473261592, ack 1, win 229, options [nop,nop,TS val 1959366 ecr 1963342], length 65160
08:34:20.701263 fa:16:3e:9e:a8:6a > fa:16:3e:a5:fc:1f, ethertype IPv4 (0x0800), length 65226: 10.10.178.148.39972 > 10.10.178.112.8298: Flags [.], seq 473261592:473326752, ack 1, win 229, options [nop,nop,TS val 1959367 ecr 1963342], length 65160
08:34:20.701378 fa:16:3e:9e:a8:6a > fa:16:3e:a5:fc:1f, ethertype IPv4 (0x0800), length 65226: 10.10.178.148.39972 > 10.10.178.112.8298: Flags [.], seq 473326752:473391912, ack 1, win 229, options [nop,nop,TS val 1959367 ecr 1963343], length 65160



[root@computehwoffload-r730-0 heat-admin]# tcpdump -i enp4s0f1_9 -nne
08:33:58.963419 fa:16:3e:a5:fc:1f > fa:16:3e:9e:a8:6a, ethertype IPv4 (0x0800), length 66: 10.10.178.112.8298 > 10.10.178.148.39972: Flags [.], ack 745381209, win 24576, options [nop,nop,TS val 1941609 ecr 1937631], length 0                                               
08:33:58.963422 fa:16:3e:a5:fc:1f > fa:16:3e:9e:a8:6a, ethertype IPv4 (0x0800), length 66: 10.10.178.112.8298 > 10.10.178.148.39972: Flags [.], ack 745388449, win 24576, options [nop,nop,TS val 1941609 ecr 1937631], length 0                                               
08:33:58.963425 fa:16:3e:a5:fc:1f > fa:16:3e:9e:a8:6a, ethertype IPv4 (0x0800), length 66: 10.10.178.112.8298 > 10.10.178.148.39972: Flags [.], ack 745395689, win 24576, options [nop,nop,TS val 1941609 ecr 1937631], length 0                                               
08:33:58.963429 fa:16:3e:a5:fc:1f > fa:16:3e:9e:a8:6a, ethertype IPv4 (0x0800), length 66: 10.10.178.112.8298 > 10.10.178.148.39972: Flags [.], ack 745402929, win 24576, options [nop,nop,TS val 1941609 ecr 1937631], length 0                                               
08:33:58.963432 fa:16:3e:a5:fc:1f > fa:16:3e:9e:a8:6a, ethertype IPv4 (0x0800), length 66: 10.10.178.112.8298 > 10.10.178.148.39972: Flags [.], ack 745410169, win 24576, options [nop,nop,TS val 1941609 ecr 1937631], length 0                                               
08:33:58.963434 fa:16:3e:a5:fc:1f > fa:16:3e:9e:a8:6a, ethertype IPv4 (0x0800), length 66: 10.10.178.112.8298 > 10.10.178.148.39972: Flags [.], ack 745417409, win 24576, options [nop,nop,TS val 1941609 ecr 1937631], length 0                                               
08:33:58.963437 fa:16:3e:a5:fc:1f > fa:16:3e:9e:a8:6a, ethertype IPv4 (0x0800), length 66: 10.10.178.112.8298 > 10.10.178.148.39972: Flags [.], ack 745423201, win 24576, options [nop,nop,TS val 1941609 ecr 1937631], length 0                                               
08:33:58.963440 fa:16:3e:a5:fc:1f > fa:16:3e:9e:a8:6a, ethertype IPv4 (0x0800), length 66: 10.10.178.112.8298 > 10.10.178.148.39972: Flags [.], ack 745447817, win 24576, options [nop,nop,TS val 1941609 ecr 1937631], length 0                                               
08:33:58.963444 fa:16:3e:a5:fc:1f > fa:16:3e:9e:a8:6a, ethertype IPv4 (0x0800), length 66: 10.10.178.112.8298 > 10.10.178.148.39972: Flags [.], ack 745459401, win 24576, options [nop,nop,TS val 1941609 ecr 1937631], length 0                                               
08:33:58.963447 fa:16:3e:a5:fc:1f > fa:16:3e:9e:a8:6a, ethertype IPv4 (0x0800), length 66: 10.10.178.112.8298 > 10.10.178.148.39972: Flags [.], ack 745468089, win 24576, options [nop,nop,TS val 1941609 ecr 1937631], length 0                                               
08:33:58.963450 fa:16:3e:a5:fc:1f > fa:16:3e:9e:a8:6a, ethertype IPv4 (0x0800), length 66: 10.10.178.112.8298 > 10.10.178.148.39972: Flags [.], ack 745475329, win 24576, options [nop,nop,TS val 1941609 ecr 1937631], length 0                                               
08:33:58.963453 fa:16:3e:a5:fc:1f > fa:16:3e:9e:a8:6a, ethertype IPv4 (0x0800), length 66: 10.10.178.112.8298 > 10.10.178.148.39972: Flags [.], ack 745484017, win 24530, options [nop,nop,TS val 1941609 ecr 1937631], length 0                                               
08:33:58.963457 fa:16:3e:a5:fc:1f > fa:16:3e:9e:a8:6a, ethertype IPv4 (0x0800), length 66: 10.10.178.112.8298 > 10.10.178.148.39972: Flags [.], ack 745491257, win 24576, options [nop,nop,TS val 1941610 ecr 1937631], length 0                                               
08:33:58.963460 fa:16:3e:a5:fc:1f > fa:16:3e:9e:a8:6a, ethertype IPv4 (0x0800), length 66: 10.10.178.112.8298 > 10.10.178.148.39972: Flags [.], ack 745518769, win 24576, options [nop,nop,TS val 1941610 ecr 1937631], length 0                                               
08:33:58.963463 fa:16:3e:a5:fc:1f > fa:16:3e:9e:a8:6a, ethertype IPv4 (0x0800), length 66: 10.10.178.112.8298 > 10.10.178.148.39972: Flags [.], ack 745531801, win 24576, options [nop,nop,TS val 1941610 ecr 1937631], length 0                                               
08:33:58.963466 fa:16:3e:a5:fc:1f > fa:16:3e:9e:a8:6a, ethertype IPv4 (0x0800), length 66: 10.10.178.112.8298 > 10.10.178.148.39972: Flags [.], ack 745540489, win 24576, options [nop,nop,TS val 1941610 ecr 1937631], length 0                                               
08:33:58.963470 fa:16:3e:a5:fc:1f > fa:16:3e:9e:a8:6a, ethertype IPv4 (0x0800), length 66: 10.10.178.112.8298 > 10.10.178.148.39972: Flags [.], ack 745547729, win 24576, options [nop,nop,TS val 1941610 ecr 1937631], length 0                                               
08:33:58.963473 fa:16:3e:a5:fc:1f > fa:16:3e:9e:a8:6a, ethertype IPv4 (0x0800), length 66: 10.10.178.112.8298 > 10.10.178.148.39972: Flags [.], ack 745552073, win 24576, options [nop,nop,TS val 1941610 ecr 1937631], length 0                                               
08:33:58.963476 fa:16:3e:a5:fc:1f > fa:16:3e:9e:a8:6a, ethertype IPv4 (0x0800), length 66: 10.10.178.112.8298 > 10.10.178.148.39972: Flags [.], ack 745559313, win 24576, options [nop,nop,TS val 1941610 ecr 1937631], length 0
Comment 20 spower 2022-10-26 12:40:33 UTC 
Blocker + given but it needs all other acks for release flag.
Comment 41 Miguel Angel Nieto 2022-11-25 09:10:59 UTC 
*** Bug 2067163 has been marked as a duplicate of this bug. ***
Comment 42 Gurpreet Singh 2023-10-14 11:20:07 UTC 
No plan to fix this in 16.1 , hence closing this bz