Bug 2066611

Summary: exempted_ext_ips for ipv6 nat doesn't take effect
Product: Red Hat Enterprise Linux Fast Datapath Reporter: Jianlin Shi <jishi>
Component: ovn-2021Assignee: lorenzo bianconi <lorenzo.bianconi>
Status: CLOSED ERRATA QA Contact: Jianlin Shi <jishi>
Severity: medium Docs Contact:
Priority: medium    
Version: FDP 22.BCC: ctrautma, jiji, lorenzo.bianconi
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-06-30 17:59:57 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jianlin Shi 2022-03-22 07:45:01 UTC
Description of problem:
exempted_ext_ips doesn't take effect

Version-Release number of selected component (if applicable):
ovn-2021-21.12.0-32

How reproducible:
Always

Steps to Reproduce:
systemctl start openvswitch                          
systemctl start ovn-northd                                                                            
ovn-nbctl set-connection ptcp:6641                                                                    
ovn-sbctl set-connection ptcp:6642
ovs-vsctl set open . external_ids:system-id=hv1 external_ids:ovn-remote=tcp:1.1.178.25:6642 external_ids:ovn-encap-type=geneve external_ids:ovn-encap-ip=1.1.178.25
systemctl restart ovn-controller

ovn-nbctl lr-add R1

ovn-nbctl ls-add sw0
ovn-nbctl ls-add sw1
ovn-nbctl ls-add public

ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 192.168.1.1/24 2001::1/64
ovn-nbctl lrp-add R1 rp-sw1 00:00:03:01:02:03 192.168.2.1/24 2002::1/64
ovn-nbctl lrp-add R1 rp-public 00:00:02:01:02:03 172.16.1.1/24 1000::a/64 \
	    -- lrp-set-gateway-chassis rp-public hv1

ovs-vsctl add-br br-ext
ovn-nbctl lsp-add sw0 sw0-rp -- set Logical_Switch_Port sw0-rp \
	    type=router options:router-port=rp-sw0 \
	        -- lsp-set-addresses sw0-rp router
ovn-nbctl lsp-add sw1 sw1-rp -- set Logical_Switch_Port sw1-rp \
	    type=router options:router-port=rp-sw1 \
	        -- lsp-set-addresses sw1-rp router

ovn-nbctl lsp-add public public-rp -- set Logical_Switch_Port public-rp \
	    type=router options:router-port=rp-public \
	        -- lsp-set-addresses public-rp router

ovs-vsctl add-port br-int sw01 -- set interface sw01 type=internal external_ids:iface-id=sw01
ip netns add sw01
ip link set sw01 netns sw01
ip netns exec sw01 ip link set sw01 address f0:00:00:01:02:03
ip netns exec sw01 ip link set sw01 up
ip netns exec sw01 ip addr add 192.168.1.2/24 dev sw01
ip netns exec sw01 ip route add default via 192.168.1.1 dev sw01
ip netns exec sw01 ip addr add 2001::2/64 dev sw01
ip netns exec sw01 ip -6 route add default via 2001::1
ovn-nbctl lsp-add sw0 sw01 \
	    -- lsp-set-addresses sw01 "f0:00:00:01:02:03 192.168.1.2 2001::2"

ovs-vsctl add-port br-int sw11 -- set interface sw11 type=internal external_ids:iface-id=sw11
ip netns add sw11
ip link set sw11 netns sw11
ip netns exec sw11 ip link set sw11 address f0:00:00:02:02:03
ip netns exec sw11 ip link set sw11 up
ip netns exec sw11 ip addr add 192.168.2.2/24 dev sw11
ip netns exec sw11 ip route add default via 192.168.2.1 dev sw11
ip netns exec sw11 ip addr add 2002::2/64 dev sw11
ip netns exec sw11 ip -6 route add default via 2002::1
ovn-nbctl lsp-add sw1 sw11 \
	    -- lsp-set-addresses sw11 "f0:00:00:02:02:03 192.168.2.2 2002::2"

sleep 2

ovs-vsctl add-port br-ext server -- set interface server type=internal
ip netns add server
ip netns exec server ip link set lo up
ip link set server netns server
ip netns exec server ip link set server up
ip netns exec server ip addr add 172.16.1.50/24 dev server
ip netns exec server ip addr add 1000::50/64 dev server

ovs-vsctl set Open_vSwitch . external-ids:ovn-bridge-mappings=phynet:br-ext
ovn-nbctl lsp-add public public1 \
		-- lsp-set-addresses public1 unknown \
			-- lsp-set-type public1 localnet \
				-- lsp-set-options public1 network_name=phynet 

ovn-nbctl lr-nat-add R1 snat 1000::12 2001::2
set1_uuid=$(ovn-nbctl create address_set name=set1 addresses='1000\:\:50')
ovn-nbctl list nat 
nat_uuid=$(ovn-nbctl list nat | awk '/_uuid/{print $3}')
ovn-nbctl set nat $nat_uuid exempted_ext_ips=$set1_uuid
ip netns exec server tcpdump -i server -w server.pcap &
sleep 1
ip netns exec sw01 ping6 1000::50 -c 3
sleep 1
pkill tcpdump
sleep 1
tcpdump -r server.pcap -nnle -v host 1000::50

Actual results:
+ ip netns exec sw01 ping6 1000::50 -c 3
PING 1000::50(1000::50) 56 data bytes
64 bytes from 1000::50: icmp_seq=1 ttl=63 time=1068 ms
64 bytes from 1000::50: icmp_seq=2 ttl=63 time=4.62 ms
64 bytes from 1000::50: icmp_seq=3 ttl=63 time=1.34 ms

--- 1000::50 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2066ms
rtt min/avg/max/mdev = 1.340/358.066/1068.240/502.170 ms, pipe 2
+ sleep 1
+ pkill tcpdump
17 packets captured
17 packets received by filter
0 packets dropped by kernel
+ sleep 1
+ tcpdump -r server.pcap -nnle -v host 1000::50
reading from file server.pcap, link-type EN10MB (Ethernet)
dropped privs to tcpdump
03:44:35.646085 4a:b0:28:09:a3:18 > 00:00:02:01:02:03, ethertype IPv6 (0x86dd), length 86: (hlim 255, next-header ICMPv6 (58) payload length: 32) 1000::50 > fe80::200:2ff:fe01:203: [icmp6 sum ok] ICMP6, neighbor advertisement, length 32, tgt is 1000::50, Flags [solicited, override]
          destination link-address option (2), length 8 (1): 4a:b0:28:09:a3:18
03:44:35.647910 00:00:02:01:02:03 > 4a:b0:28:09:a3:18, ethertype IPv6 (0x86dd), length 118: (flowlabel 0x4f063, hlim 63, next-header ICMPv6 (58) payload length: 64) 1000::12 > 1000::50: [icmp6 sum ok] ICMP6, echo request, seq 1
03:44:35.647980 4a:b0:28:09:a3:18 > 33:33:ff:00:00:12, ethertype IPv6 (0x86dd), length 86: (hlim 255, next-header ICMPv6 (58) payload length: 32) 1000::50 > ff02::1:ff00:12: [icmp6 sum ok] ICMP6, neighbor solicitation, length 32, who has 1000::12
          source link-address option (1), length 8 (1): 4a:b0:28:09:a3:18
03:44:35.648220 00:00:02:01:02:03 > 4a:b0:28:09:a3:18, ethertype IPv6 (0x86dd), length 118: (flowlabel 0x4f063, hlim 63, next-header ICMPv6 (58) payload length: 64) 1000::12 > 1000::50: [icmp6 sum ok] ICMP6, echo request, seq 2
03:44:35.649143 00:00:02:01:02:03 > 4a:b0:28:09:a3:18, ethertype IPv6 (0x86dd), length 86: (hlim 255, next-header ICMPv6 (58) payload length: 32) 1000::12 > 1000::50: [icmp6 sum ok] ICMP6, neighbor advertisement, length 32, tgt is 1000::12, Flags [solicited, override]
          destination link-address option (2), length 8 (1): 00:00:02:01:02:03
03:44:35.649176 4a:b0:28:09:a3:18 > 00:00:02:01:02:03, ethertype IPv6 (0x86dd), length 118: (flowlabel 0x9c3fc, hlim 64, next-header ICMPv6 (58) payload length: 64) 1000::50 > 1000::12: [icmp6 sum ok] ICMP6, echo reply, seq 1
03:44:35.649297 4a:b0:28:09:a3:18 > 00:00:02:01:02:03, ethertype IPv6 (0x86dd), length 118: (flowlabel 0x9c3fc, hlim 64, next-header ICMPv6 (58) payload length: 64) 1000::50 > 1000::12: [icmp6 sum ok] ICMP6, echo reply, seq 2
03:44:36.647849 00:00:02:01:02:03 > 4a:b0:28:09:a3:18, ethertype IPv6 (0x86dd), length 118: (flowlabel 0x4f063, hlim 63, next-header ICMPv6 (58) payload length: 64) 1000::12 > 1000::50: [icmp6 sum ok] ICMP6, echo request, seq 3

<=== packet is snated

03:44:36.647884 4a:b0:28:09:a3:18 > 00:00:02:01:02:03, ethertype IPv6 (0x86dd), length 118: (flowlabel 0x9c3fc, hlim 64, next-header ICMPv6 (58) payload length: 64) 1000::50 > 1000::12: [icmp6 sum ok] ICMP6, echo reply, seq 3

Expected results:
the packet should not be nated, the ping should fail

Additional info:


[root@wsfd-advnetlab16 nat_test]# rpm -qa | grep -E "openvswitch2.15|ovn-2021"
ovn-2021-21.12.0-32.el8fdp.x86_64
openvswitch2.15-2.15.0-84.el8fdp.x86_64
ovn-2021-host-21.12.0-32.el8fdp.x86_64
ovn-2021-central-21.12.0-32.el8fdp.x86_64
python3-openvswitch2.15-2.15.0-84.el8fdp.x86_64

Comment 5 Jianlin Shi 2022-06-06 08:06:44 UTC
Verified on ovn-2021-21.12.0-73.el8fdp:

+ ovn-nbctl set nat e6424cd5-6155-4b5f-80d5-f7fbe228551c exempted_ext_ips=a8de008d-cc41-4aa0-9336-3d8833caf94d
+ sleep 1
+ ip netns exec server tcpdump -i server -w server.pcap                                               
dropped privs to tcpdump                                                                              
tcpdump: listening on server, link-type EN10MB (Ethernet), capture size 262144 bytes                  
+ ip netns exec sw01 ping6 1000::50 -c 3
PING 1000::50(1000::50) 56 data bytes                                                                 

--- 1000::50 ping statistics ---                                                                      
3 packets transmitted, 0 received, 100% packet loss, time 2039ms                                      

+ sleep 1
+ pkill tcpdump                                                                                       
18 packets captured
18 packets received by filter
0 packets dropped by kernel                                                                           
+ sleep 1
+ tcpdump -r server.pcap -nnle -v host 1000::50
reading from file server.pcap, link-type EN10MB (Ethernet)
dropped privs to tcpdump
04:05:44.289192 66:ee:d4:fe:38:bd > 00:00:02:01:02:03, ethertype IPv6 (0x86dd), length 86: (hlim 255, next-header ICMPv6 (58) payload length: 32) 1000::50 > fe80::200:2ff:fe01:203: [icmp6 sum ok] ICMP6, neighbor advertisement, length 32, tgt is 1000::50, Flags [solicited, override]
          destination link-address option (2), length 8 (1): 66:ee:d4:fe:38:bd                        
04:05:44.291059 00:00:02:01:02:03 > 66:ee:d4:fe:38:bd, ethertype IPv6 (0x86dd), length 118: (flowlabel 0x85a08, hlim 63, next-header ICMPv6 (58) payload length: 64) 2001::2 > 1000::50: [icmp6 sum ok] ICMP6, echo request, seq 1
04:05:44.291320 00:00:02:01:02:03 > 66:ee:d4:fe:38:bd, ethertype IPv6 (0x86dd), length 118: (flowlabel 0x85a08, hlim 63, next-header ICMPv6 (58) payload length: 64) 2001::2 > 1000::50: [icmp6 sum ok] ICMP6, echo request, seq 2
04:05:45.312890 00:00:02:01:02:03 > 66:ee:d4:fe:38:bd, ethertype IPv6 (0x86dd), length 118: (flowlabel 0x85a08, hlim 63, next-header ICMPv6 (58) payload length: 64) 2001::2 > 1000::50: [icmp6 sum ok] ICMP6, echo request, seq 3                                                                                
[root@dell-per740-12 bz2066611]# rpm -qa | grep -E "openvswitch2.15|ovn-2021"                         
ovn-2021-21.12.0-73.el8fdp.x86_64
ovn-2021-host-21.12.0-73.el8fdp.x86_64
openvswitch2.15-2.15.0-104.el8fdp.x86_64
ovn-2021-central-21.12.0-73.el8fdp.x86_64

Comment 6 Jianlin Shi 2022-06-06 08:08:57 UTC
also Verified on ovn22.03-22.03.0-52.el8fdp:

+ nat_uuid=a4a2c882-ce62-4de5-97db-7ed2967e867a                                                       
+ ovn-nbctl set nat a4a2c882-ce62-4de5-97db-7ed2967e867a exempted_ext_ips=5adcd91b-0a56-46fc-9ba3-010a8b1ce16f
+ sleep 1
+ ip netns exec server tcpdump -i server -w server.pcap
dropped privs to tcpdump
tcpdump: listening on server, link-type EN10MB (Ethernet), capture size 262144 bytes                  
+ ip netns exec sw01 ping6 1000::50 -c 3                                                              
PING 1000::50(1000::50) 56 data bytes                                                                 

--- 1000::50 ping statistics ---                                                                      
3 packets transmitted, 0 received, 100% packet loss, time 2061ms                                      

+ sleep 1
+ pkill tcpdump
18 packets captured
19 packets received by filter                                                                         
0 packets dropped by kernel
+ sleep 1
+ tcpdump -r server.pcap -nnle -v host 1000::50                                                       
reading from file server.pcap, link-type EN10MB (Ethernet)
dropped privs to tcpdump
04:07:51.521160 c6:fa:4e:de:dd:ef > 00:00:02:01:02:03, ethertype IPv6 (0x86dd), length 86: (hlim 255, next-header ICMPv6 (58) payload length: 32) 1000::50 > fe80::200:2ff:fe01:203: [icmp6 sum ok] ICMP6, neighbor advertisement, length 32, tgt is 1000::50, Flags [solicited, override]                        
          destination link-address option (2), length 8 (1): c6:fa:4e:de:dd:ef
04:07:51.522615 00:00:02:01:02:03 > c6:fa:4e:de:dd:ef, ethertype IPv6 (0x86dd), length 118: (flowlabel 0x85a08, hlim 63, next-header ICMPv6 (58) payload length: 64) 2001::2 > 1000::50: [icmp6 sum ok] ICMP6, echo request, seq 1                                                                                
04:07:51.522811 00:00:02:01:02:03 > c6:fa:4e:de:dd:ef, ethertype IPv6 (0x86dd), length 118: (flowlabel 0x85a08, hlim 63, next-header ICMPv6 (58) payload length: 64) 2001::2 > 1000::50: [icmp6 sum ok] ICMP6, echo request, seq 2
04:07:52.544904 00:00:02:01:02:03 > c6:fa:4e:de:dd:ef, ethertype IPv6 (0x86dd), length 118: (flowlabel 0x85a08, hlim 63, next-header ICMPv6 (58) payload length: 64) 2001::2 > 1000::50: [icmp6 sum ok] ICMP6, echo request, seq 3
[root@dell-per740-12 bz2066611]# rpm -qa | grep -E "openvswitch2.15|ovn22.03"
ovn22.03-22.03.0-52.el8fdp.x86_64                                                                     
ovn22.03-central-22.03.0-52.el8fdp.x86_64
openvswitch2.15-2.15.0-104.el8fdp.x86_64                                                              
ovn22.03-host-22.03.0-52.el8fdp.x86_64

Comment 7 Jianlin Shi 2022-06-07 07:24:10 UTC
Verified on ovn22.03-22.03.0-52.el9:

[root@wsfd-advnetlab18 bz2066611]# rpm -qa | grep -E "openvswitch|ovn"
ovn22.03-22.03.0-52.el9fdp.x86_64
openvswitch-selinux-extra-policy-1.0-31.el9fdp.noarch
openvswitch2.17-2.17.0-21.el9fdp.x86_64
ovn22.03-central-22.03.0-52.el9fdp.x86_64
ovn22.03-host-22.03.0-52.el9fdp.x86_64

+ ovn-nbctl set nat fd40716a-815c-4ff9-9988-3cce15055e9f exempted_ext_ips=71798ccc-b3be-4c9b-896c-59ae6928b80f
+ sleep 1
+ ip netns exec server tcpdump -i server -w server.pcap
dropped privs to tcpdump
tcpdump: listening on server, link-type EN10MB (Ethernet), snapshot length 262144 bytes
+ ip netns exec sw01 ping6 1000::50 -c 3
PING 1000::50(1000::50) 56 data bytes

--- 1000::50 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2026ms

+ sleep 1
+ pkill tcpdump
17 packets captured
17 packets received by filter
0 packets dropped by kernel
+ sleep 1
+ tcpdump -r server.pcap -nnle -v host 1000::50
reading from file server.pcap, link-type EN10MB (Ethernet), snapshot length 262144
dropped privs to tcpdump
03:22:16.115621 ea:56:ac:48:b8:3a > 00:00:02:01:02:03, ethertype IPv6 (0x86dd), length 86: (hlim 255, next-header ICMPv6 (58) payload length: 32) 1000::50 > fe80::200:2ff:fe01:203: [icmp6 sum ok] ICMP6, neighbor advertisement, length 32, tgt is 1000::50, Flags [solicited, override]
          destination link-address option (2), length 8 (1): ea:56:ac:48:b8:3a
03:22:16.117121 00:00:02:01:02:03 > ea:56:ac:48:b8:3a, ethertype IPv6 (0x86dd), length 118: (flowlabel 0x134d6, hlim 63, next-header ICMPv6 (58) payload length: 64) 2001::2 > 1000::50: [icmp6 sum ok] ICMP6, echo request, id 2654, seq 1
03:22:16.117302 00:00:02:01:02:03 > ea:56:ac:48:b8:3a, ethertype IPv6 (0x86dd), length 118: (flowlabel 0x134d6, hlim 63, next-header ICMPv6 (58) payload length: 64) 2001::2 > 1000::50: [icmp6 sum ok] ICMP6, echo request, id 2654, seq 2
03:22:17.139134 00:00:02:01:02:03 > ea:56:ac:48:b8:3a, ethertype IPv6 (0x86dd), length 118: (flowlabel 0x134d6, hlim 63, next-header ICMPv6 (58) payload length: 64) 2001::2 > 1000::50: [icmp6 sum ok] ICMP6, echo request, id 2654, seq 3

Comment 9 errata-xmlrpc 2022-06-30 17:59:57 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (ovn bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:5446