Description of problem: Let's say we do two route add commands: /usr/bin/ovn-nbctl --timeout=15 --may-exist --bfd --policy=src-ip --ecmp-symmetric-reply lr-route-add GR_ovn-worker 10.244.0.5/32 172.18.0.4 rtoe-GR_ovn-worker /usr/bin/ovn-nbctl --timeout=15 --may-exist --bfd --policy=src-ip --ecmp-symmetric-reply lr-route-add GR_ovn-worker2 10.244.2.5/32 172.18.0.4 rtoe-GR_ovn-worker2 We'd expect two sessions: one from 0.5 to 0.4 and another from 2.5 to 0.4. Currently the way OVN creates a bfd session is: Introduce the --bfd option to lr-route-add command. If the BFD session UUID is provided, it will be used for the OVN route otherwise the next-hop will be used to perform a lookup in the OVN BFD table. If the lookup fails and outport is specified, a new entry in the BFD table will be created using the nexthop as dst_ip and outport as logical_port. How reproducible: Always Steps to Reproduce: 1. Create two routes one from x1 to y1 and another from x2 to y1. 2. 3. Actual results: Only one session from x1 to y1 will be created. x2 to y1 will fail because OVN checks that y1 already has a bfd session and won't create a new one. Expected results: We should have two bfd sessions.
(In reply to Surya Seetharaman from comment #0) > Description of problem: > > Let's say we do two route add commands: > > /usr/bin/ovn-nbctl --timeout=15 --may-exist --bfd --policy=src-ip > --ecmp-symmetric-reply lr-route-add GR_ovn-worker 10.244.0.5/32 172.18.0.4 > rtoe-GR_ovn-worker > /usr/bin/ovn-nbctl --timeout=15 --may-exist --bfd --policy=src-ip > --ecmp-symmetric-reply lr-route-add GR_ovn-worker2 10.244.2.5/32 172.18.0.4 > rtoe-GR_ovn-worker2 we need to consider output interface as well looking for an existing bfd session. For the time being we can create 2 different bfd seesions with different outports and then link them to the static routes with --bfd uuid > > We'd expect two sessions: one from 0.5 to 0.4 and another from 2.5 to 0.4. > Currently the way OVN creates a bfd session is: > > Introduce the --bfd option to lr-route-add command. > If the BFD session UUID is provided, it will be used for the OVN route > otherwise the next-hop will be used to perform a lookup in the OVN BFD > table. > If the lookup fails and outport is specified, a new entry in the BFD table > will be created using the nexthop as dst_ip and outport as logical_port. > > > > How reproducible: Always > > > Steps to Reproduce: > 1. Create two routes one from x1 to y1 and another from x2 to y1. > 2. > 3. > > Actual results: > Only one session from x1 to y1 will be created. x2 to y1 will fail because > OVN checks that y1 already has a bfd session and won't create a new one. > > Expected results: > We should have two bfd sessions.
upstream fix: http://patchwork.ozlabs.org/project/ovn/patch/b07f581f46ef50befd5a22875ab9a12ff540c0e9.1632491529.git.lorenzo.bianconi@redhat.com/
Tested this PR on an OCP OVN-K cluster and it works as expected. I created two pods on different nodes having the same nexthop and OVN created two BFD sessions one from each of the GW routers on those nodes towards the nexthop/dst_ip. ======= sh-4.4# ovn-nbctl lr-route-list GR_ci-ln-xwgizmt-f76d1-f8gv9-worker-b-vshc4 IPv4 Routes 10.128.8.12 10.0.128.4 src-ip rtoe-GR_ci-ln-xwgizmt-f76d1-f8gv9-worker-b-vshc4 ecmp-symmetric-reply bfd 10.128.0.0/16 100.64.0.1 dst-ip 0.0.0.0/0 10.0.128.1 dst-ip rtoe-GR_ci-ln-xwgizmt-f76d1-f8gv9-worker-b-vshc4 sh-4.4# ovn-nbctl lr-route-list GR_ci-ln-xwgizmt-f76d1-f8gv9-worker-c-rrq85 IPv4 Routes 10.128.10.13 10.0.128.4 src-ip rtoe-GR_ci-ln-xwgizmt-f76d1-f8gv9-worker-c-rrq85 ecmp-symmetric-reply bfd 10.128.0.0/16 100.64.0.1 dst-ip 0.0.0.0/0 10.0.128.1 dst-ip rtoe-GR_ci-ln-xwgizmt-f76d1-f8gv9-worker-c-rrq85 sh-4.4# ovn-nbctl list bfd _uuid : a646da00-f201-4048-923b-82bac426c91a detect_mult : [] dst_ip : "10.0.128.4" external_ids : {} logical_port : rtoe-GR_ci-ln-xwgizmt-f76d1-f8gv9-worker-b-vshc4 min_rx : [] min_tx : [] options : {} status : down _uuid : 00c14ced-6c0c-4b9e-8772-4eff9411bad6 detect_mult : [] dst_ip : "10.0.128.4" external_ids : {} logical_port : rtoe-GR_ci-ln-xwgizmt-f76d1-f8gv9-worker-c-rrq85 min_rx : [] min_tx : [] options : {} status : down Would be good if we could move this patch along.
tested with following script: systemctl start openvswitch systemctl start ovn-northd ovn-nbctl set-connection ptcp:6641 ovn-sbctl set-connection ptcp:6642 ovs-vsctl set open . external_ids:system-id=hv1 external_ids:ovn-remote=tcp:20.0.40.25:6642 external_ids:ovn-encap-type=geneve external_ids:ovn-encap-ip=20.0.40.25 systemctl restart ovn-controller ovn-nbctl lr-add r1 ovn-nbctl lrp-add r1 r1-net1 00:00:00:01:ff:01 192.168.100.1/24 ovn-nbctl lr-add r2 ovn-nbctl lrp-add r2 r2-net2 00:00:00:02:ff:01 192.168.100.1/24 ovn-nbctl lr-add r3 ovn-nbctl lrp-add r3 r3-net3 00:00:00:03:ff:01 192.168.200.1/24 ovn-nbctl ls-add pub ovn-nbctl lrp-add r1 r1-pub 00:00:00:11:ff:01 172.18.1.124/24 ovn-nbctl lrp-set-gateway-chassis r1-pub hv1 ovn-nbctl lsp-add pub pub-r1 ovn-nbctl lsp-set-type pub-r1 router ovn-nbctl lsp-set-addresses pub-r1 router ovn-nbctl lsp-set-options pub-r1 router-port=r1-pub ovn-nbctl lrp-add r2 r2-pub 00:00:00:12:ff:01 172.18.2.110/24 ovn-nbctl lrp-set-gateway-chassis r2-pub hv1 ovn-nbctl lsp-add pub pub-r2 ovn-nbctl lsp-set-type pub-r2 router ovn-nbctl lsp-set-addresses pub-r2 router ovn-nbctl lsp-set-options pub-r2 router-port=r2-pub ovn-nbctl lrp-add r3 r3-pub 00:00:00:13:ff:01 172.18.1.173/24 ovn-nbctl lrp-set-gateway-chassis r3-pub hv1 ovn-nbctl lsp-add pub pub-r3 ovn-nbctl lsp-set-type pub-r3 router ovn-nbctl lsp-set-addresses pub-r3 router ovn-nbctl lsp-set-options pub-r3 router-port=r3-pub ovn-nbctl --bfd lr-route-add r1 0.0.0.0/0 172.18.1.1 r1-pub ovn-nbctl --bfd lr-route-add r3 0.0.0.0/0 172.18.1.1 r3-pub #ovn-nbctl lr-route-add r2 172.18.2.10 172.18.2.1 r2-pub #ovn-nbctl lr-route-add r2 172.18.2.12 172.18.2.1 r2-pub ovs-vsctl add-br br-provider ovs-vsctl set open . external-ids:ovn-bridge-mappings=provider:br-provider ovn-nbctl lsp-add pub ln ovn-nbctl lsp-set-options ln network_name=provider ovn-nbctl lsp-set-type ln localnet ovn-nbctl lsp-set-addresses ln unknown ovs-vsctl add-port br-provider ext1 -- set interface ext1 type=internal ip netns add ext1 ip link set ext1 netns ext1 ip netns exec ext1 ip link set ext1 up ip netns exec ext1 ip addr add 172.18.1.1/24 dev ext1 ip netns exec ext1 ip addr add 172.18.2.1/24 dev ext1 ip netns exec ext1 sysctl -w net.ipv4.conf.all.rp_filter=0 ip netns exec ext1 sysctl -w net.ipv4.conf.all.forwarding=1 ovn-nbctl list bfd ovn-nbctl lr-route-list r1 ovn-nbctl lr-route-list r3 reproduced on ovn-2021-21.06.0-24: + ovn-nbctl list bfd _uuid : 62852b59-9235-4195-8886-ee607c80c0fd detect_mult : [] dst_ip : "172.18.1.1" external_ids : {} logical_port : r1-pub min_rx : [] min_tx : [] options : {} status : down + ovn-nbctl lr-route-list r1 IPv4 Routes 0.0.0.0/0 172.18.1.1 dst-ip r1-pub bfd + ovn-nbctl lr-route-list r3 IPv4 Routes 0.0.0.0/0 172.18.1.1 dst-ip r3-pub bfd Verified on ovn-2021-21.09.0-12: [root@dell-per740-12 bz2007549]# rpm -qa | grep -E "openvswitch2.16|ovn-2021" ovn-2021-21.09.0-12.el8fdp.x86_64 ovn-2021-central-21.09.0-12.el8fdp.x86_64 ovn-2021-host-21.09.0-12.el8fdp.x86_64 python3-openvswitch2.16-2.16.0-16.el8fdp.x86_64 openvswitch2.16-2.16.0-16.el8fdp.x86_64 openvswitch2.16-test-2.16.0-16.el8fdp.noarch + ovn-nbctl list bfd _uuid : 486b6b5a-81eb-4d5e-935f-c64ec0496052 detect_mult : [] dst_ip : "172.18.1.1" external_ids : {} logical_port : r3-pub min_rx : [] min_tx : [] options : {} status : down _uuid : 53c65703-25a6-43e4-a29a-eb1d0591148f detect_mult : [] dst_ip : "172.18.1.1" external_ids : {} logical_port : r1-pub min_rx : [] min_tx : [] options : {} status : down + ovn-nbctl lr-route-list r1 IPv4 Routes 0.0.0.0/0 172.18.1.1 dst-ip r1-pub bfd + ovn-nbctl lr-route-list r3 IPv4 Routes 0.0.0.0/0 172.18.1.1 dst-ip r3-pub bfd
also verified on ovn2.13-20.12.0-185.el8: [root@dell-per740-12 bz2007549]# rpm -qa | grep -E "openvswitch2.16|ovn2.13" ovn2.13-central-20.12.0-185.el8fdp.x86_64 ovn2.13-host-20.12.0-185.el8fdp.x86_64 python3-openvswitch2.16-2.16.0-16.el8fdp.x86_64 ovn2.13-20.12.0-185.el8fdp.x86_64 openvswitch2.16-2.16.0-16.el8fdp.x86_64 openvswitch2.16-test-2.16.0-16.el8fdp.noarch + ovn-nbctl list bfd _uuid : 4291183c-1baa-4b5d-9d6b-0913846cb516 detect_mult : [] dst_ip : "172.18.1.1" external_ids : {} logical_port : r1-pub min_rx : [] min_tx : [] options : {} status : down _uuid : b953e8ba-e4d9-4e64-af5c-32ad32e8e245 detect_mult : [] dst_ip : "172.18.1.1" external_ids : {} logical_port : r3-pub min_rx : [] min_tx : [] options : {} status : down + ovn-nbctl lr-route-list r1 IPv4 Routes 0.0.0.0/0 172.18.1.1 dst-ip r1-pub bfd + ovn-nbctl lr-route-list r3 IPv4 Routes 0.0.0.0/0 172.18.1.1 dst-ip r3-pub bfd
set VERIFIED per comment 4
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (ovn bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2021:5059