Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
The FDP team is no longer accepting new bugs in Bugzilla. Please report your issues under FDP project in Jira. Thanks.

Bug 2173387

Summary: [HWOL]tc(revalidator58)|ERR|failed to parse flower classifier terse options
Product: Red Hat Enterprise Linux Fast Datapath Reporter: mhou <mhou>
Component: openvswitch3.1Assignee: Eelco Chaudron <echaudro>
Status: CLOSED CURRENTRELEASE QA Contact: mhou <mhou>
Severity: medium Docs Contact:
Priority: medium    
Version: FDP 23.BCC: ctrautma, fleitner, jhsiao, kzhang, mleitner, ralongi
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-06-24 02:20:17 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2172622    

Description mhou 2023-02-26 09:42:38 UTC 
Description of problem:
run performance test and got err message on ovs-vswitchd.log
beaker job:https://beaker.engineering.redhat.com/jobs/7569059
ovs-vswitchd.log: https://beaker-archive.hosts.prod.psi.bos.redhat.com/beaker-logs/2023/02/75690/7569059/13454524/156739454/ovs-vswitchd.log


Version-Release number of selected component (if applicable):
kernel version: kernel-rt-5.14.0-70.43.1.rt21.114.el9_0 
ovs version: openvswitch3.1-3.1.0-2.el9fdp

How reproducible: only once
I run same test job on stock kernel but can't hit this issue.
kernel-5.14.0-70.43.1.el9_0+openvswitch3.1-3.1.0-2.el9fdp
https://beaker.engineering.redhat.com/jobs/7569995 --pass(no err message)
I change the distro from RHEL-9.0.0-updates-20230215.28 to RHEL-9.0.0-updates-20230224.29 and keep same rt kernel(kernel-rt-5.14.0-70.43.1.rt21.114.el9_0) but still no luck.
https://beaker.engineering.redhat.com/jobs/7570563 --pass(no err message)

Steps to Reproduce:
1. host cpu topo as below:
# numactl --hardware
available: 2 nodes (0-1)
node 0 cpus: 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54
node 0 size: 31260 MB
node 0 free: 2313 MB
node 1 cpus: 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55
node 1 size: 31177 MB
node 1 free: 2827 MB
node distances:
node   0   1 
  0:  10  20 
  1:  20  10 

2. tunning host as below:
# cat /proc/cmdline 
BOOT_IMAGE=(hd0,msdos1)/vmlinuz-5.14.0-70.43.1.rt21.114.el9_0.x86_64 root=/dev/mapper/rhel_dell--per750--03-root ro ksdevice=bootif pci=realloc resume=/dev/mapper/rhel_dell--per750--03-swap rd.lvm.lv=rhel_dell-per750-03/root rd.lvm.lv=rhel_dell-per750-03/swap console=ttyS0,115200n81 skew_tick=1 isolcpus=managed_irq,domain,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54 intel_pstate=disable nosoftlockup tsc=nowatchdog nohz=on nohz_full=4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54 rcu_nocbs=4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54 irqaffinity=0,1,2,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,33,35,37,39,41,43,45,47,49,51,53,55 crashkernel=1G-4G:384M,4G-16G:512M,16G-64G:1G,64G-128G:2G,128G-:4G default_hugepagesz=1G hugepagesz=1G hugepages=48 intel_iommu=on iommu=pt intel_idle.max_cstate=0 processor.max_cstate=0 intel_pstate=disable idle=poll rcu_nocb_poll

3. configure ovs hardware offload and guest testpmd as below
ovs-vsctl show
fd85b93a-4988-4a8d-b869-c01d6baa608d
    Bridge ovsbr0
        Port ens1f0np0
            Interface ens1f0np0
        Port eth0
            Interface eth0
        Port ovsbr0
            Interface ovsbr0
                type: internal
    Bridge ovsbr1
        Port eth1
            Interface eth1
        Port ens1f1np1
            Interface ens1f1np1
        Port ovsbr1
            Interface ovsbr1
                type: internal
    ovs_version: "3.1.1"
ovs-vsctl get Open_vSwitch . other_config
{hw-offload="true", max-idle="180000", tc-policy=none}

ovs-vsctl set Open_vSwitch . other_config:max-idle=3600000
ovs-vsctl set Open_vSwitch . other_config:max-revalidator=3600000
ovs-vsctl set bridge ovsbr0 other-config:mac-aging-time=3600
ovs-vsctl set bridge ovsbr0 other-config:mac-table-size=$((2*1024+8192))
ovs-vsctl set bridge ovsbr1 other-config:mac-aging-time=3600
ovs-vsctl set bridge ovsbr1 other-config:mac-table-size=$((2*1024+8192))

Actual results:
check error message on ovs-vswitchd.log
ovs_offload_pvp_flows1024_queue1_vcpus3_size9000_vlan0_hwoffload_on_tcpolicy_none_start
2023-02-24T12:50:42.388Z|00015|dpif_netlink(handler1)|WARN|system@ovs-system: lost packet on handler 0
2023-02-24T12:50:42.427Z|00016|dpif_netlink(handler1)|WARN|system@ovs-system: lost packet on handler 0
2023-02-24T12:52:06.793Z|00017|dpif_netlink(handler1)|WARN|system@ovs-system: lost packet on handler 0
2023-02-24T12:52:06.860Z|00018|dpif_netlink(handler1)|WARN|system@ovs-system: lost packet on handler 0
2023-02-24T12:52:06.934Z|00019|dpif_netlink(handler1)|WARN|system@ovs-system: lost packet on handler 0
2023-02-24T13:00:31.109Z|00045|ofproto_dpif_upcall(revalidator67)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
2023-02-24T13:00:31.109Z|00042|dpif(revalidator65)|WARN|Dropped 235 log messages in last 604 seconds (most recently, 603 seconds ago) due to excessive rate
2023-02-24T13:00:31.109Z|00043|dpif(revalidator65)|WARN|system@ovs-system: failed to flow_get (No such file or directory) ufid:b663dbc8-68a2-4674-9791-0a5a7e85e87b <empty>, packets:0, bytes:0, used:never
2023-02-24T13:00:31.109Z|00044|ofproto_dpif_upcall(revalidator65)|WARN|Dropped 176 log messages in last 603 seconds (most recently, 603 seconds ago) due to excessive rate
2023-02-24T13:00:31.109Z|00045|ofproto_dpif_upcall(revalidator65)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:b663dbc8-68a2-4674-9791-0a5a7e85e87b
2023-02-24T13:00:31.111Z|00040|ofproto_dpif_upcall(revalidator66)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
2023-02-24T13:00:31.112Z|00024|dpif(revalidator60)|WARN|system@ovs-system: failed to flow_get (No such file or directory) ufid:2417bbb5-24e0-4357-b8eb-5ce7d9c8c09f <empty>, packets:0, bytes:0, used:never
2023-02-24T13:00:31.112Z|00025|ofproto_dpif_upcall(revalidator60)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:2417bbb5-24e0-4357-b8eb-5ce7d9c8c09f
2023-02-24T13:00:31.112Z|00041|dpif(revalidator68)|WARN|system@ovs-system: failed to flow_get (No such file or directory) ufid:dbbf9465-cdcd-46a9-b2e8-65daea9715b7 <empty>, packets:0, bytes:0, used:never
2023-02-24T13:00:31.112Z|00042|ofproto_dpif_upcall(revalidator68)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:dbbf9465-cdcd-46a9-b2e8-65daea9715b7
2023-02-24T13:00:31.112Z|00043|dpif(revalidator68)|WARN|system@ovs-system: failed to flow_get (No such file or directory) ufid:6902839f-55a3-4a61-b8e8-ab194e8bb31f <empty>, packets:0, bytes:0, used:never
2023-02-24T13:00:31.112Z|00044|ofproto_dpif_upcall(revalidator68)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:6902839f-55a3-4a61-b8e8-ab194e8bb31f
2023-02-24T13:00:31.112Z|00045|dpif(revalidator68)|WARN|system@ovs-system: failed to flow_get (No such file or directory) ufid:8053d01b-45c6-486a-b807-4db5e08a2395 <empty>, packets:0, bytes:0, used:never
2023-02-24T13:00:31.112Z|00046|ofproto_dpif_upcall(revalidator68)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:8053d01b-45c6-486a-b807-4db5e08a2395
2023-02-24T13:00:31.113Z|00042|ofproto_dpif_upcall(revalidator70)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:4a7fc5e4-50ce-4a32-95e3-2e172ba369e2
2023-02-24T13:00:31.114Z|00039|ofproto_dpif_upcall(revalidator58)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:e89e2106-a58c-40d3-bda9-1db21ccd0c91
2023-02-24T13:00:31.115Z|00054|ofproto_dpif_upcall(revalidator63)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:08e94e98-7d87-407e-b325-37e4a110e715
2023-02-24T13:00:31.115Z|00055|ofproto_dpif_upcall(revalidator63)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:f78cb484-4ed6-4fb3-bc50-1cc3f54ff4cc
2023-02-24T13:00:31.115Z|00056|ofproto_dpif_upcall(revalidator63)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:f5bf9130-7272-427f-93d5-311aa5a8569d
2023-02-24T13:00:31.117Z|00046|ofproto_dpif_upcall(revalidator67)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
2023-02-24T13:00:31.118Z|00046|ofproto_dpif_upcall(revalidator65)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:6c14b912-b665-4f69-96aa-56a2e0af9758
2023-02-24T13:00:31.118Z|00037|ofproto_dpif_upcall(revalidator61)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:2b226e6b-6075-45c0-9a9b-c6b8e644f489
2023-02-24T13:00:31.118Z|00038|ofproto_dpif_upcall(revalidator61)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:5c6aa84e-8d09-430d-933e-bd1875fb86b8
2023-02-24T13:00:31.119Z|00023|ofproto_dpif_upcall(revalidator59)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:3f523701-b994-4bb8-a7d4-159e40613da2
2023-02-24T13:00:31.120Z|00024|ofproto_dpif_upcall(revalidator59)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:b860e6d4-fa44-4c74-8cda-c5890a21df92
2023-02-24T13:00:31.120Z|00025|ofproto_dpif_upcall(revalidator59)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:3bd6ae28-ca7d-4781-bd5e-de431b3377ca
2023-02-24T13:00:31.121Z|00041|ofproto_dpif_upcall(revalidator66)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:356d8913-0525-4357-b477-3d2868416b6d
2023-02-24T13:00:31.122Z|00026|ofproto_dpif_upcall(revalidator60)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:d57a8d4d-83eb-45d4-9c4a-fc24b47e73f6
2023-02-24T13:00:31.122Z|00039|ofproto_dpif_upcall(revalidator71)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:50980ca5-329a-4746-8dc8-f8cc971e91d7
2023-02-24T13:00:31.122Z|00040|ofproto_dpif_upcall(revalidator71)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:c8379809-231a-43dc-9a09-9ef22673edcc
2023-02-24T13:00:31.122Z|00041|ofproto_dpif_upcall(revalidator71)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:000f44c1-37e5-4ba3-aba1-41158a707c04
2023-02-24T13:00:31.122Z|00042|ofproto_dpif_upcall(revalidator71)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:6aeef551-8779-4107-898f-9f2eb4947034
2023-02-24T13:00:31.122Z|00043|ofproto_dpif_upcall(revalidator64)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:e5f4d291-d41f-40de-89d0-4b30946cb3d4
2023-02-24T13:00:31.122Z|00044|ofproto_dpif_upcall(revalidator64)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:1490bf2f-b910-466f-b6fb-066565a059df
2023-02-24T13:00:31.122Z|00047|ofproto_dpif_upcall(revalidator68)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:28256d66-a9f6-42bd-8dde-9d21bfc09f80
2023-02-24T13:00:31.122Z|00048|ofproto_dpif_upcall(revalidator68)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:933b46fe-7ccb-4cd3-aa65-4c10be1b3dd1
2023-02-24T13:00:31.123Z|00049|ofproto_dpif_upcall(revalidator68)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:5dfb0b14-7a50-48d6-9762-74e8e362f44d
2023-02-24T13:00:31.123Z|00050|ofproto_dpif_upcall(revalidator68)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:13c69a5a-fa15-4503-992c-4f51c4a1c2e7
2023-02-24T13:00:31.124Z|00043|ofproto_dpif_upcall(revalidator70)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:159c99fd-26c5-4a87-814b-4779b0fbc476
2023-02-24T13:00:31.125Z|00044|ofproto_dpif_upcall(revalidator70)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:d212da82-1a43-4c47-8b18-2a1d36c3d35e
2023-02-24T13:00:31.125Z|00045|ofproto_dpif_upcall(revalidator70)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:a08fec1a-e201-4306-84fa-ff6a06156ee4
2023-02-24T13:00:31.125Z|00046|ofproto_dpif_upcall(revalidator70)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:34c06fd0-0ddc-4369-90b6-055ada1bc0c3
2023-02-24T13:00:31.125Z|00033|ofproto_dpif_upcall(revalidator62)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:abf95ede-2a9a-4be1-a23d-50ddd21c5e60
2023-02-24T13:00:31.125Z|00034|ofproto_dpif_upcall(revalidator62)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:e1eb3e25-4cc8-4055-a54b-68869af495ce
2023-02-24T13:00:31.126Z|00057|ofproto_dpif_upcall(revalidator63)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:a0057f9c-87a7-4db6-a61d-793e384abc89
2023-02-24T13:00:31.126Z|00058|ofproto_dpif_upcall(revalidator63)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:aad87004-836e-4582-be49-739ad3402700
2023-02-24T13:00:31.126Z|00059|ofproto_dpif_upcall(revalidator63)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:2fb10015-9f00-4879-a781-df5f9b38ef83
2023-02-24T13:00:31.126Z|00030|ofproto_dpif_upcall(revalidator69)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
2023-02-24T13:00:31.127Z|00060|ofproto_dpif_upcall(revalidator63)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:881c04a7-aeb9-474a-b280-9b30b71506ea
2023-02-24T13:00:31.127Z|00039|ofproto_dpif_upcall(revalidator72)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:015c605f-4cef-4dc2-a456-635797a34980
2023-02-24T13:00:31.127Z|00040|ofproto_dpif_upcall(revalidator72)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:684ad8d7-037b-4c77-a88e-5fb0717c9717
2023-02-24T13:00:31.127Z|00041|ofproto_dpif_upcall(revalidator72)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:c1a81141-5422-4ebf-bcff-f1e74aef2d7b
2023-02-24T13:00:31.127Z|00042|ofproto_dpif_upcall(revalidator72)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:6b7e7791-1780-4257-ace5-27561aab0c51
2023-02-24T13:00:31.127Z|00043|ofproto_dpif_upcall(revalidator72)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:cafd0329-373d-4e7e-b453-87419bd9b4d8
2023-02-24T13:00:31.127Z|00044|ofproto_dpif_upcall(revalidator72)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:88f7be86-1a1a-4e9d-95b4-23839c7fbe78
2023-02-24T13:00:31.128Z|00061|ofproto_dpif_upcall(revalidator63)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
2023-02-24T13:00:31.128Z|00040|ofproto_dpif_upcall(revalidator58)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:741cc7d3-9266-4e40-ac9f-a9f2b14d636f
2023-02-24T13:00:31.128Z|00041|ofproto_dpif_upcall(revalidator58)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:a3a9f64d-80df-48dc-8c3c-06f501b0958a
2023-02-24T13:00:31.129Z|00047|ofproto_dpif_upcall(revalidator67)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:6b7f6d26-0722-422d-9698-9b47809e3efd
2023-02-24T13:00:31.129Z|00042|ofproto_dpif_upcall(revalidator58)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:3aed4f6d-9a74-47e4-848d-0ad6ce215cbf
2023-02-24T13:00:31.129Z|00048|ofproto_dpif_upcall(revalidator67)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:48461d1a-6170-412c-abbc-a5932754ad8a
2023-02-24T13:00:31.129Z|00049|ofproto_dpif_upcall(revalidator67)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:93883be2-a647-4de2-91dc-4b3baa630dbb
2023-02-24T13:00:31.129Z|00050|ofproto_dpif_upcall(revalidator67)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:3e566165-a865-4de5-a73b-fa55ab8cb12f
2023-02-24T13:00:31.129Z|00051|ofproto_dpif_upcall(revalidator67)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:a9fe23dd-39bf-4cfa-a62e-1e95abc0a1d7
2023-02-24T13:00:31.129Z|00052|ofproto_dpif_upcall(revalidator67)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:d59bc318-cdc4-4443-a362-5d4a98ef0ac7
2023-02-24T13:00:31.130Z|00053|ofproto_dpif_upcall(revalidator67)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:97a9ea98-4528-4976-81d0-258f71ee3440
2023-02-24T13:00:31.130Z|00054|ofproto_dpif_upcall(revalidator67)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
2023-02-24T13:00:31.130Z|00062|ofproto_dpif_upcall(revalidator63)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:09ee7476-6632-40c4-aec4-c411f9f0dc27
2023-02-24T13:00:31.130Z|00039|ofproto_dpif_upcall(revalidator61)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:4675fa2b-5094-4756-ae6f-ab6ab2235a39
2023-02-24T13:00:31.130Z|00043|ofproto_dpif_upcall(revalidator58)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:f0582e64-d8e1-4501-a033-3fa02548472b
2023-02-24T13:00:31.130Z|00047|ofproto_dpif_upcall(revalidator70)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:966d3987-ae43-40ce-953e-8bad8c7ccfa1
2023-02-24T13:00:31.131Z|00048|ofproto_dpif_upcall(revalidator70)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:34b80755-4327-4ffe-a1fe-ed14b1bcad5c
2023-02-24T13:00:31.131Z|00031|ofproto_dpif_upcall(revalidator69)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (No such file or directory): ufid:a0bdba5c-b449-4cc6-a25a-1d5a44d54723
2023-02-24T13:00:31.135Z|00043|ofproto_dpif_upcall(revalidator71)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
2023-02-24T13:00:31.135Z|00044|ofproto_dpif_upcall(revalidator71)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
2023-02-24T13:00:31.136Z|00040|ofproto_dpif_upcall(revalidator61)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
2023-02-24T13:00:31.137Z|00035|ofproto_dpif_upcall(revalidator62)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
2023-02-24T13:00:31.138Z|00045|ofproto_dpif_upcall(revalidator71)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
2023-02-24T13:00:31.139Z|00041|ofproto_dpif_upcall(revalidator61)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
2023-02-24T13:00:31.145Z|00042|ofproto_dpif_upcall(revalidator66)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
2023-02-24T13:00:31.160Z|00055|ofproto_dpif_upcall(revalidator67)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
2023-02-24T13:00:31.163Z|00032|ofproto_dpif_upcall(revalidator69)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
2023-02-24T13:00:31.167Z|00036|ofproto_dpif_upcall(revalidator62)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
2023-02-24T13:00:31.181Z|00051|ofproto_dpif_upcall(revalidator68)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
2023-02-24T13:00:31.181Z|00043|ofproto_dpif_upcall(revalidator66)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
2023-02-24T13:00:31.185Z|00047|ofproto_dpif_upcall(revalidator65)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
2023-02-24T13:00:31.193Z|00044|ofproto_dpif_upcall(revalidator58)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
2023-02-24T13:00:31.193Z|00045|tc(revalidator58)|ERR|failed to parse flower classifier terse options
2023-02-24T13:00:31.195Z|00044|ofproto_dpif_upcall(revalidator66)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
2023-02-24T13:00:31.197Z|00052|ofproto_dpif_upcall(revalidator68)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
2023-02-24T13:00:31.198Z|00037|ofproto_dpif_upcall(revalidator62)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
2023-02-24T13:00:31.202Z|00046|ofproto_dpif_upcall(revalidator58)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
2023-02-24T13:00:31.202Z|00033|ofproto_dpif_upcall(revalidator69)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
2023-02-24T13:00:31.202Z|00049|ofproto_dpif_upcall(revalidator70)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
2023-02-24T13:00:31.202Z|00056|ofproto_dpif_upcall(revalidator67)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
2023-02-24T13:00:31.202Z|00045|ofproto_dpif_upcall(revalidator64)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
ovs_offload_pvp_xnuma_no_flows1024_queue1_vcpus3_size9000_vlan0_hwoffload_on_tcpolicy_none_end

Expected results:
no more error message on ovs-vswitchd.log

Additional info:
Comment 1 mhou 2023-03-09 07:49:36 UTC 
Run kernel-rt-5.14.0-283.rt14.283.el9 + openvswitch3.1-3.1.0-2.el9fdp hit very similar issue.
https://beaker.engineering.redhat.com/jobs/7602386
ovs log: https://beaker.engineering.redhat.com/recipes/13511072/tasks/157222558/logs/ovs-vswitchd.log
:: [ 06:05:05 ] :: [  BEGIN   ] :: Running 'grep -C 20 '|ERR|' /var/log/openvswitch/ovs-vswitchd.log'
2023-03-08T11:05:03.301Z|13313|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.302Z|13314|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.302Z|13315|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.302Z|13316|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.302Z|13317|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.302Z|13318|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.302Z|13319|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.302Z|13320|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.302Z|13321|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.302Z|13322|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.302Z|13323|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.302Z|13324|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.303Z|13325|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.303Z|13326|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.303Z|13327|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.303Z|13328|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.303Z|13329|ofproto_dpif_upcall(revalidator4)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
2023-03-08T11:05:03.303Z|13330|netdev_offload_tc(revalidator4)|DBG|flow get (dev ens1f0np0 prio 2 handle 768 block_id 0)
2023-03-08T11:05:03.304Z|13331|dpif_netlink(revalidator4)|DBG|found flow from netdev, translating to dpif flow
2023-03-08T11:05:03.304Z|13332|netdev_offload_tc(revalidator4)|DBG|flow get (dev ens1f0np0 prio 2 handle 1338 block_id 0)
2023-03-08T11:05:03.304Z|13333|netdev_offload_tc(revalidator4)|ERR|flow get failed (dev ens1f0np0 prio 2 handle 1338): No such file or directory
2023-03-08T11:05:03.304Z|13334|netdev_offload_tc(revalidator4)|DBG|flow get (dev ovsbr1 prio 2 handle 1338 block_id 0)
2023-03-08T11:05:03.304Z|13335|netdev_offload_tc(revalidator4)|ERR|flow get failed (dev ovsbr1 prio 2 handle 1338): No such file or directory
2023-03-08T11:05:03.304Z|13336|netdev_offload_tc(revalidator4)|DBG|flow get (dev eth1 prio 2 handle 1338 block_id 0)
2023-03-08T11:05:03.304Z|13337|netdev_offload_tc(revalidator4)|ERR|flow get failed (dev eth1 prio 2 handle 1338): No such file or directory
2023-03-08T11:05:03.304Z|13338|netdev_offload_tc(revalidator4)|DBG|flow get (dev ens1f1np1 prio 2 handle 1338 block_id 0)
2023-03-08T11:05:03.304Z|13339|netdev_offload_tc(revalidator4)|ERR|flow get failed (dev ens1f1np1 prio 2 handle 1338): No such file or directory
2023-03-08T11:05:03.304Z|13340|netdev_offload_tc(revalidator4)|DBG|flow get (dev ovsbr0 prio 2 handle 1338 block_id 0)
2023-03-08T11:05:03.304Z|13341|netdev_offload_tc(revalidator4)|ERR|flow get failed (dev ovsbr0 prio 2 handle 1338): No such file or directory
2023-03-08T11:05:03.304Z|13342|netdev_offload_tc(revalidator4)|DBG|flow get (dev eth0 prio 2 handle 1338 block_id 0)
2023-03-08T11:05:03.304Z|13343|netdev_offload_tc(revalidator4)|DBG|flow get (dev ovs-system prio 2 handle 1338 block_id 0)
2023-03-08T11:05:03.304Z|13344|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.304Z|13345|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.305Z|13346|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.305Z|13347|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.305Z|13348|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.305Z|13349|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.305Z|13350|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.305Z|13351|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.305Z|13352|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.305Z|13353|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.305Z|13354|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.305Z|13355|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.305Z|13356|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.305Z|13357|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.305Z|13358|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.306Z|13359|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.306Z|13360|dpif_netlink(revalidator4)|DBG|added flow
2023-03-08T11:05:03.306Z|13361|dpif_netlink(revalidator4)|DBG|added flow
Comment 2 mhou 2023-03-16 08:22:51 UTC 
Run same test on same environment with kernel-rt-5.14.0-284.1.1.rt14.285.el9_2 can also hit this issue.

https://beaker.engineering.redhat.com/jobs/7628881
full of ovs-vswitchd.log: https://beaker-archive.hosts.prod.psi.bos.redhat.com/beaker-logs/2023/03/76288/7628881/13552994/157546339/ovs-vswitchd.log
Snippet of ovs-vswitchd.log
:: [ 22:22:57 ] :: [  BEGIN   ] :: Running 'grep -C 20 '|ERR|' /tmp/ovs_temp_log.log'
2023-03-16T02:22:36.970Z|08658|dpif_netlink(revalidator4)|DBG|added flow
2023-03-16T02:22:36.970Z|08659|dpif_netlink(revalidator4)|DBG|added flow
2023-03-16T02:22:36.970Z|08660|dpif_netlink(revalidator4)|DBG|added flow
2023-03-16T02:22:36.970Z|08661|ofproto_dpif_upcall(revalidator4)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
2023-03-16T02:22:36.971Z|08662|ofproto_dpif_upcall(revalidator4)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
2023-03-16T02:22:36.971Z|08663|ofproto_dpif_upcall(revalidator4)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
2023-03-16T02:22:36.972Z|08664|ofproto_dpif_upcall(revalidator4)|INFO|Unexpected ukey transition from state 3 (last transitioned from thread 0 at ../ofproto/ofproto-dpif-upcall.c:2540)
2023-03-16T02:22:36.988Z|08665|netdev_offload_tc(revalidator4)|DBG|flow get (dev ovsbr1 prio 4 handle 22 block_id 0)
2023-03-16T02:22:36.988Z|08666|dpif_netlink(revalidator4)|DBG|found flow from netdev, translating to dpif flow
2023-03-16T02:22:36.988Z|08667|netdev_offload_tc(revalidator4)|DBG|flow get (dev ovsbr1 prio 4 handle 71 block_id 0)
2023-03-16T02:22:36.989Z|08668|dpif_netlink(revalidator4)|DBG|found flow from netdev, translating to dpif flow
2023-03-16T02:22:36.989Z|08669|netdev_offload_tc(revalidator4)|DBG|flow get (dev ovsbr1 prio 4 handle 81 block_id 0)
2023-03-16T02:22:36.989Z|08670|dpif_netlink(revalidator4)|DBG|found flow from netdev, translating to dpif flow
2023-03-16T02:22:36.989Z|08671|netdev_offload_tc(revalidator4)|DBG|flow get (dev ovsbr1 prio 4 handle 134 block_id 0)
2023-03-16T02:22:36.989Z|08672|dpif_netlink(revalidator4)|DBG|found flow from netdev, translating to dpif flow
2023-03-16T02:22:36.989Z|08673|netdev_offload_tc(revalidator4)|DBG|flow get (dev ovsbr1 prio 4 handle 208 block_id 0)
2023-03-16T02:22:36.989Z|08674|dpif_netlink(revalidator4)|DBG|found flow from netdev, translating to dpif flow
2023-03-16T02:22:36.989Z|08675|netdev_offload_tc(revalidator4)|DBG|flow get (dev ovsbr1 prio 4 handle 224 block_id 0)
2023-03-16T02:22:36.989Z|08676|dpif_netlink(revalidator4)|DBG|found flow from netdev, translating to dpif flow
2023-03-16T02:22:36.989Z|08677|netdev_offload_tc(revalidator4)|DBG|flow get (dev ovsbr1 prio 4 handle 876 block_id 0)
2023-03-16T02:22:36.989Z|08678|netdev_offload_tc(revalidator4)|ERR|flow get failed (dev ovsbr1 prio 4 handle 876): No such file or directory
2023-03-16T02:22:36.989Z|08679|netdev_offload_tc(revalidator4)|DBG|flow get (dev eth1 prio 4 handle 876 block_id 0)
2023-03-16T02:22:36.989Z|08680|netdev_offload_tc(revalidator4)|ERR|flow get failed (dev eth1 prio 4 handle 876): No such file or directory
2023-03-16T02:22:36.989Z|08681|netdev_offload_tc(revalidator4)|DBG|flow get (dev ens1f1np1 prio 4 handle 876 block_id 0)
2023-03-16T02:22:36.989Z|08682|netdev_offload_tc(revalidator4)|ERR|flow get failed (dev ens1f1np1 prio 4 handle 876): No such file or directory
2023-03-16T02:22:36.989Z|08683|netdev_offload_tc(revalidator4)|DBG|flow get (dev eth0 prio 4 handle 876 block_id 0)
2023-03-16T02:22:36.989Z|08684|netdev_offload_tc(revalidator4)|ERR|flow get failed (dev eth0 prio 4 handle 876): No such file or directory
2023-03-16T02:22:36.989Z|08685|netdev_offload_tc(revalidator4)|DBG|flow get (dev ovsbr0 prio 4 handle 876 block_id 0)
2023-03-16T02:22:36.989Z|08686|netdev_offload_tc(revalidator4)|ERR|flow get failed (dev ovsbr0 prio 4 handle 876): No such file or directory
2023-03-16T02:22:36.989Z|08687|netdev_offload_tc(revalidator4)|DBG|flow get (dev ens1f0np0 prio 4 handle 876 block_id 0)
2023-03-16T02:22:36.989Z|08688|netdev_offload_tc(revalidator4)|DBG|flow get (dev ovs-system prio 4 handle 876 block_id 0)
2023-03-16T02:22:36.989Z|08689|netdev_offload_tc(revalidator4)|DBG|flow get (dev ovsbr1 prio 4 handle 252 block_id 0)
2023-03-16T02:22:36.989Z|08690|dpif_netlink(revalidator4)|DBG|found flow from netdev, translating to dpif flow
2023-03-16T02:22:36.989Z|08691|netdev_offload_tc(revalidator4)|DBG|flow get (dev ovsbr1 prio 4 handle 287 block_id 0)
2023-03-16T02:22:36.989Z|08692|dpif_netlink(revalidator4)|DBG|found flow from netdev, translating to dpif flow
2023-03-16T02:22:36.989Z|08693|netdev_offload_tc(revalidator4)|DBG|flow get (dev ovsbr1 prio 4 handle 343 block_id 0)
2023-03-16T02:22:36.989Z|08694|dpif_netlink(revalidator4)|DBG|found flow from netdev, translating to dpif flow
2023-03-16T02:22:36.989Z|08695|netdev_offload_tc(revalidator4)|DBG|flow get (dev ovsbr1 prio 4 handle 353 block_id 0)
2023-03-16T02:22:36.989Z|08696|dpif_netlink(revalidator4)|DBG|found flow from netdev, translating to dpif flow
2023-03-16T02:22:36.989Z|08697|netdev_offload_tc(revalidator4)|DBG|flow get (dev ovsbr1 prio 4 handle 395 block_id 0)
2023-03-16T02:22:36.990Z|08698|dpif_netlink(revalidator4)|DBG|found flow from netdev, translating to dpif flow
2023-03-16T02:22:36.990Z|08699|netdev_offload_tc(revalidator4)|DBG|flow get (dev ovsbr1 prio 4 handle 455 block_id 0)
2023-03-16T02:22:36.990Z|08700|dpif_netlink(revalidator4)|DBG|found flow from netdev, translating to dpif flow
2023-03-16T02:22:36.990Z|08701|netdev_offload_tc(revalidator4)|DBG|flow get (dev ovsbr1 prio 4 handle 4 block_id 0)
2023-03-16T02:22:36.990Z|08702|dpif_netlink(revalidator4)|DBG|found flow from netdev, translating to dpif flow
2023-03-16T02:22:36.990Z|08703|netdev_offload_tc(revalidator4)|DBG|flow get (dev ovsbr1 prio 4 handle 5 block_id 0)
2023-03-16T02:22:36.990Z|08704|dpif_netlink(revalidator4)|DBG|found flow from netdev, translating to dpif flow
2023-03-16T02:22:36.990Z|08705|netdev_offload_tc(revalidator4)|DBG|flow get (dev ovsbr1 prio 4 handle 28 block_id 0)
2023-03-16T02:22:36.990Z|08706|dpif_netlink(revalidator4)|DBG|found flow from netdev, translating to dpif flow
Comment 3 mhou 2023-06-16 02:10:11 UTC 
This issue still can be reproduced on kernel-5.14.0-325.el9+rt + openvswitch3.1-3.1.0-24.el9fdp

beaker job:https://beaker.engineering.redhat.com/jobs/7975038

:: [ 14:55:18 ] :: [  BEGIN   ] :: Running 'check_error_message /tmp/ovs_temp_log.log'
2023-06-15T18:55:17.388Z|01588|tc|ERR|failed to parse flower classifier options
:: [ 14:55:18 ] :: [  BEGIN   ] :: Running 'grep -C 20 '|ERR|' /tmp/ovs_temp_log.log'
2023-06-15T18:55:17.386Z|41416|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.386Z|41417|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.386Z|41418|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.387Z|41419|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.387Z|41420|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.387Z|41421|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.387Z|41422|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.387Z|41423|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.387Z|41424|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.387Z|41425|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.387Z|41426|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.387Z|41427|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.388Z|41428|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.388Z|41429|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.388Z|41430|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.388Z|41431|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.388Z|41432|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.388Z|41433|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.388Z|41434|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.388Z|41435|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.388Z|01588|tc|ERR|failed to parse flower classifier options
2023-06-15T18:55:17.388Z|41436|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.388Z|41437|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.388Z|41438|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.388Z|41439|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.389Z|41440|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.389Z|41441|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.389Z|41442|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.389Z|41443|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.389Z|41444|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.389Z|41445|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.389Z|41446|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.389Z|41447|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.389Z|41448|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.389Z|41449|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.390Z|41450|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.390Z|41451|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.390Z|41452|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.390Z|41453|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.391Z|41454|dpif_netlink(revalidator4)|DBG|added flow
2023-06-15T18:55:17.391Z|41455|dpif_netlink(revalidator4)|DBG|added flow
:: [ 14:55:19 ] :: [   PASS   ] :: Command 'grep -C 20 '|ERR|' /tmp/ovs_temp_log.log' (Expected 0, got 0)
Comment 4 Eelco Chaudron 2023-06-19 14:27:26 UTC 
Hi @mhou, it looks like 2182942, 2173387, and 2143570 are related. Would it be possible to get access to a setup where I can replicate and further debug the issue?
Comment 5 mhou 2023-06-24 02:20:17 UTC 
Test on RHEL-9.3.0-20230621.0(kernel-5.14.0-325.el9+rt)+ openvswitch3.1-3.1.0-33.el9fdp and didn't hit this issue.
https://beaker.engineering.redhat.com/jobs/8002286