Description of problem: When a port is multichassis (requested-chassis is a comma separated list), if the LS that the port belongs to has a localnet port, traffic to and from the multichassis port is tunneled anyway. (This is done to guarantee delivery of packets destined to the port MAC address to all its locations.) This enforced tunneling may be a problem if the effective MTU for the ports becomes different from the theoretical MTU of the physical network that underlies the LS (defined by MTU of localnet port in the same switch). In this case, the port should not communicate with the outside world using the max MTU. The proposal here is for OVN controller to set up ICMP Path Discovery replies to oversized packets received from a multichassis port, so that the port owner is aware of the change in circumstances and can adequately adjust their effective MTU. The problem was originally discussed in ovs-dev ML: https://www.mail-archive.com/ovs-dev@openvswitch.org/msg68204.html Implementation was proposed at: https://mail.openvswitch.org/pipermail/ovs-dev/2022-November/398981.html This bug is to take the patch over and get it tested / merged in OVN. The bug affects OSP live migration scenario for VMs attached to physical networks (=switches with localnet port). I expect this bug to become a blocker for OSP 17.1 because of its effect on live migration scenario.
The upstream series to fix the issue was merged in main and 23.06. I've posted backports for review down to 22.09. We'll need 22.12 series downstream to close the bug: https://patchwork.ozlabs.org/project/ovn/list/?series=357540
Reproduced on: ovn22.12-22.12.0-79.el8fdp.x86_64 ovn22.12-host-22.12.0-79.el8fdp.x86_64 ovn22.12-central-22.12.0-79.el8fdp.x86_64 openvswitch2.17-2.17.0-102.el8fdp.x86_64 openvswitch-selinux-extra-policy-1.0-30.el8fdp.noarch ###### Here is the reproducer ##### ############# HV1 ############ systemctl start ovn-northd ovn-nbctl set-connection ptcp:6641 ovn-sbctl set-connection ptcp:6642 systemctl start openvswitch ovs-vsctl set open . external_ids:system-id=hv1 ovs-vsctl set open . external_ids:ovn-remote=tcp:1.1.101.25:6642 ovs-vsctl set open . external_ids:ovn-encap-type=geneve ovs-vsctl set open . external_ids:ovn-encap-ip=1.1.101.25 systemctl start ovn-controller ovs-vsctl add-br br-phys ip link set br-phys up ovs-vsctl set open . external-ids:ovn-bridge-mappings=phys:br-phys ovn-nbctl ls-add ls ovn-nbctl lsp-add ls lsp1 ovn-nbctl lsp-set-addresses lsp1 "00:00:00:01:00:02 172.16.1.1 3001::1" ovn-nbctl lsp-add ls lsp2 ovn-nbctl lsp-set-addresses lsp2 "00:00:00:02:00:02 172.16.1.2 3001::2" ovn-nbctl lsp-add ls lsp_multi1 ovn-nbctl lsp-set-addresses lsp_multi1 "00:00:00:03:00:02 172.16.1.3 3001::3" ovn-nbctl lsp-add ls lsp_multi2 ovn-nbctl lsp-set-addresses lsp_multi2 "00:00:00:04:00:02 172.16.1.4 3001::4" ovn-nbctl lsp-add ls public ovn-nbctl lsp-set-type public localnet ovn-nbctl lsp-set-addresses public unknown ovn-nbctl lsp-set-options public network_name=phys ovn-nbctl lsp-set-options lsp1 requested-chassis=hv1 ovn-nbctl lsp-set-options lsp2 requested-chassis=hv2 ovn-nbctl lsp-set-options lsp_multi1 requested-chassis=hv1,hv2 ovn-nbctl lsp-set-options lsp_multi2 requested-chassis=hv1,hv2 ovs-vsctl add-port br-int lsp1 -- set interface lsp1 type=internal ip netns add lsp1 ip link set lsp1 netns lsp1 ip netns exec lsp1 ip link set lsp1 address 00:00:00:01:00:02 ip netns exec lsp1 ip addr add 172.16.1.1/24 dev lsp1 ip netns exec lsp1 ip -6 addr add 3001::1/64 dev lsp1 ip netns exec lsp1 ip link set lsp1 up ovs-vsctl set interface lsp1 external_ids:iface-id=lsp1 ovs-vsctl add-port br-int lsp_multi1 -- set interface lsp_multi1 type=internal ip netns add lsp_multi1 ip link set lsp_multi1 netns lsp_multi1 ip netns exec lsp_multi1 ip link set lsp_multi1 address 00:00:00:03:00:02 ip netns exec lsp_multi1 ip addr add 172.16.1.3/24 dev lsp_multi1 ip netns exec lsp_multi1 ip -6 addr add 3001::3/64 dev lsp_multi1 ip netns exec lsp_multi1 ip link set lsp_multi1 up ovs-vsctl set interface lsp_multi1 external_ids:iface-id=lsp_multi1 ####### HV2 ########## ovs-vsctl set open . external_ids:system-id=hv2 ovs-vsctl set open . external_ids:ovn-remote=tcp:1.1.101.25:6642 ovs-vsctl set open . external_ids:ovn-encap-type=geneve ovs-vsctl set open . external_ids:ovn-encap-ip=1.1.101.26 systemctl start ovn-controller ovs-vsctl add-br br-phys ip link set br-phys up ovs-vsctl set open . external-ids:ovn-bridge-mappings=phys:br-phys ovs-vsctl add-port br-int lsp2 -- set interface lsp2 type=internal ip netns add lsp2 ip link set lsp2 netns lsp2 ip netns exec lsp2 ip link set lsp2 address 00:00:00:02:00:02 ip netns exec lsp2 ip addr add 172.16.1.2/24 dev lsp2 ip netns exec lsp2 ip -6 addr add 3001::2/64 dev lsp2 ip netns exec lsp2 ip link set lsp2 up ovs-vsctl set interface lsp2 external_ids:iface-id=lsp2 ovs-vsctl add-port br-int lsp_multi1 -- set interface lsp_multi1 type=internal ip netns add lsp_multi1 ip link set lsp_multi1 netns lsp_multi1 ip netns exec lsp_multi1 ip link set lsp_multi1 address 00:00:00:03:00:02 ip netns exec lsp_multi1 ip addr add 172.16.1.3/24 dev lsp_multi1 ip netns exec lsp_multi1 ip -6 addr add 3001::3/64 dev lsp_multi1 ip netns exec lsp_multi1 ip link set lsp_multi1 up ovs-vsctl set interface lsp_multi1 external_ids:iface-id=lsp_multi1 ###### RESULTS ######### [root@---bz2180955]# ip netns exec lsp_multi1 ping 172.16.1.2 -c 3 -s 3000 PING 172.16.1.2 (172.16.1.2) 3000(3028) bytes of data. --- 172.16.1.2 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2049ms ##### Verified on: ###### ovn22.12-host-22.12.0-91.el8fdp.x86_64 openvswitch2.17-2.17.0-102.el8fdp.x86_64 ovn22.12-22.12.0-91.el8fdp.x86_64 ovn22.12-central-22.12.0-91.el8fdp.x86_64 openvswitch-selinux-extra-policy-1.0-30.el8fdp.noarch [root@---bz2180955]# ip netns exec lsp_multi1 ping 172.16.1.2 -c 3 -s 3000 PING 172.16.1.2 (172.16.1.2) 3000(3028) bytes of data. From 172.16.1.2 icmp_seq=1 Frag needed and DF set (mtu = 1424) 3008 bytes from 172.16.1.2: icmp_seq=3 ttl=64 time=0.382 ms 3008 bytes from 172.16.1.2: icmp_seq=2 ttl=64 time=0.284 ms 3008 bytes from 172.16.1.2: icmp_seq=3 ttl=64 time=0.271 ms --- 172.16.1.2 ping statistics --- 3 packets transmitted, 3 received, +1 errors, 0% packet loss, time 2006ms rtt min/avg/max/mdev = 0.271/0.625/1.321/0.492 ms <=============== ICPM path discovery ######## Also verified on: ######## openvswitch-selinux-extra-policy-1.0-34.el9fdp.noarch openvswitch2.17-2.17.0-95.el9fdp.x86_64 ovn22.12-22.12.0-91.el9fdp.x86_64 ovn22.12-central-22.12.0-91.el9fdp.x86_64 ovn22.12-host-22.12.0-91.el9fdp.x86_64