Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
The FDP team is no longer accepting new bugs in Bugzilla. Please report your issues under FDP project in Jira. Thanks.

Bug 1776994

Summary: OCP IPv6 - Packets dropped on upcall from kernel after SNAT - Failed to acquire udpif_key corresponding to unexpected flow
Product: Red Hat Enterprise Linux Fast Datapath Reporter: Mark Michelson <mmichels>
Component: ovn2.12Assignee: Open vSwitch development team <ovs-team>
Status: CLOSED ERRATA QA Contact: Jianlin Shi <jishi>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: RHEL 8.0CC: ctrautma, dcbw, fleitner, jhsiao, jishi, kfida, markmc, mmichels, nusiddiq, ovs-qe, ovs-team, qding, ralongi
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1773605 Environment:
Last Closed: 2019-12-11 12:19:53 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1773605    
Bug Blocks: 1775160, 1776969, 1776973    

Comment 2 Jianlin Shi 2019-11-28 06:38:03 UTC
reproduced on ovn2.12-2.12.0-14:

[root@dell-per740-12 ~]# rpm -qa | grep -E "openvswitch|ovn"                                          
ovn2.12-host-2.12.0-14.el8fdp.x86_64                                                                  
openvswitch2.12-2.12.0-4.el8fdp.x86_64                                                                
ovn2.12-central-2.12.0-14.el8fdp.x86_64                                                               
openvswitch-selinux-extra-policy-1.0-19.el8fdp.noarch                                                 
ovn2.12-2.12.0-14.el8fdp.x86_64

[root@dell-per740-12 ~]# ip netns exec server0 ping6 3000::1 -c 1                                     
PING 3000::1(3000::1) 56 data bytes                                                                   

--- 3000::1 ping statistics ---                                                                       
1 packets transmitted, 0 received, 100% packet loss, time 0ms                                         
                                                                                                      
[root@dell-per740-12 ~]# grep ofproto_dpif_upcall /var/log/openvswitch/ovs-vswitchd.log               
2019-11-28T06:35:12.363Z|00001|ofproto_dpif_upcall(handler51)|INFO|received packet on unassociated datapath port 4294967295
2019-11-28T06:35:12.376Z|00001|ofproto_dpif_upcall(revalidator85)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (Invalid argument): ufid:6988794e-04ad-447d-a32e-96aa780075b5
<==== error message in log

[root@dell-per740-12 bz1776994]# ip netns exec server1 tcpdump -i any -nnle
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
^C                                                                                                    
0 packets captured                                                                                    
0 packets received by filter                                                                          
0 packets dropped by kernel

<=== no NS received

Comment 3 Jianlin Shi 2019-11-28 06:39:03 UTC
no ovn2.12-2.12.0-15 for rhel8 version, numan, please help to check

Comment 4 Numan Siddique 2019-11-28 10:08:35 UTC
Triggered the build for RHEL 8- https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=25010753

Comment 5 Jianlin Shi 2019-11-29 01:18:07 UTC
Verified on ovn2.12.0-15:

[root@dell-per740-12 bz1773598]# ip netns exec server0 ping6 3000::1 -c 2
PING 3000::1(3000::1) 56 data bytes
64 bytes from 3000::1: icmp_seq=2 ttl=63 time=3.38 ms

--- 3000::1 ping statistics ---
2 packets transmitted, 1 received, 50% packet loss, time 22ms
rtt min/avg/max/mdev = 3.376/3.376/3.376/0.000 ms

<==== ping passed
                                                     
[root@dell-per740-12 bz1773598]# grep ofproto_dpif_upcall /var/log/openvswitch/ovs-vswitchd.log
2019-11-29T01:15:31.717Z|00001|ofproto_dpif_upcall(handler50)|INFO|received packet on unassociated datapath port 4294967295
2019-11-29T01:15:31.718Z|00001|ofproto_dpif_upcall(revalidator85)|WARN|Failed to acquire udpif_key corresponding to unexpected flow (Invalid argument): ufid:d1bfea1f-f07f-4ade-ad32-57fe344199f1

<==== error message

[root@dell-per740-12 bz1773598]# rpm -qa | grep -E "openvswitch|ovn"                                  
ovn2.12-central-2.12.0-15.el8fdp.x86_64
openvswitch2.12-2.12.0-8.el8fdp.x86_64
ovn2.12-2.12.0-15.el8fdp.x86_64
ovn2.12-host-2.12.0-15.el8fdp.x86_64
openvswitch-selinux-extra-policy-1.0-19.el8fdp.noarch

[root@dell-per740-12 bz1776994]# ip netns exec server1 tcpdump -i any -nnle
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode                            
listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
20:15:31.704626   M 00:de:ad:ff:01:03 ethertype IPv6 (0x86dd), length 88: 2001::1 > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has 3000::1, length 32

<==== NS received

20:15:31.704670 Out f2:ce:2e:1b:37:80 ethertype IPv6 (0x86dd), length 88: 3000::1 > 2001::1: ICMP6, neighbor advertisement, tgt is 3000::1, length 32
20:15:32.659571 Out f2:ce:2e:1b:37:80 ethertype IPv6 (0x86dd), length 72: fe80::f0ce:2eff:fe1b:3780 > ff02::2: ICMP6, router solicitation, length 16
20:15:32.659915   M 0a:aa:07:c9:21:4a ethertype IPv6 (0x86dd), length 72: fe80::8aa:7ff:fec9:214a > ff02::2: ICMP6, router solicitation, length 16
20:15:32.724362  In 00:de:ad:ff:01:03 ethertype IPv6 (0x86dd), length 120: 3000::55 > 3000::1: ICMP6, echo request, seq 2, length 64
20:15:32.724420 Out f2:ce:2e:1b:37:80 ethertype IPv6 (0x86dd), length 88: 3000::1 > ff02::1:ff00:55: ICMP6, neighbor solicitation, who has 3000::55, length 32
20:15:32.726194  In 00:de:ad:ff:01:03 ethertype IPv6 (0x86dd), length 88: 3000::55 > 3000::1: ICMP6, neighbor advertisement, tgt is 3000::55, length 32
20:15:32.726228 Out f2:ce:2e:1b:37:80 ethertype IPv6 (0x86dd), length 120: 3000::1 > 3000::55: ICMP6, echo reply, seq 2, length 64


set VERIFIED based on https://bugzilla.redhat.com/show_bug.cgi?id=1773605#c11

Comment 7 errata-xmlrpc 2019-12-11 12:19:53 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2019:4209