Description of problem: Currently we require that if you specify an external gateway that corresponds to a multus network, then you must do so as a pod update to add the exgw annotations. This is due to the fact that multus will not add the network attachment definition until after OVN CNI runs, and therefore OVN will not be able to locate the exgw information. The current workflow is this: 1. Create exgw pod (OVN will add pod, multus will add network attachment def) 2. Once created, edit the pod and annotate it with the exgw annotations that link to a multus network: k8s.ovn.org/routing-namespaces: exgw k8s.ovn.org/routing-network: blah However, we should be able to handle including the routing annotations on the pod during pod add. When multus runs after OVN and annotates the network attachment definition, OVN-Kube will get an update event from k8s and should be able to then reconcile the routes.
Verified on 4.8.0-0.nightly-2021-04-08-005413 on vSphere route is updated after annotation change. sh-4.4# ovn-nbctl --no-leader-only lr-route-list GR_compute-0 IPv4 Routes 10.128.2.119 172.31.249.143 src-ip ecmp-symmetric-reply 10.128.0.0/14 100.64.0.1 dst-ip 0.0.0.0/0 172.31.248.1 dst-ip rtoe-GR_compute-0 2021-04-08T15:57:22.842Z|14445|nbctl|INFO|Running command run --may-exist --policy=src-ip --ecmp-symmetric-reply -- lr-route-add GR_compute-0 10.128.2.119/32 172.31.249.168 2021-04-08T15:57:22.867Z|14446|nbctl|INFO|Running command run --may-exist --policy=src-ip --ecmp-symmetric-reply -- lr-route-add GR_compute-0 10.128.2.119/32 172.31.249.168 E0408 15:57:22.868472 1 ovn.go:519] unable to add src-ip route to GR router, stderr:"ovn-nbctl: duplicate nexthop for the same ECMP route\n", err:OVN command '/usr/bin/ovn-nbctl --timeout=15 --may-exist --policy=src-ip --ecmp-symmetric-reply lr-route-add GR_compute-0 10.128.2.119/32 172.31.249.168' failed: exit status 1 I0408 15:57:22.868636 1 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"default", Name:"testpod1", UID:"0fa79daf-d39e-412c-bd38-8e5e567a80b0", APIVersion:"v1", ResourceVersion:"83697", FieldPath:""}): type: 'Warning' reason: 'ErrorAddingLogicalPort' unable to add src-ip route to GR router, stderr:"ovn-nbctl: duplicate nexthop for the same ECMP route\n", err:OVN command '/usr/bin/ovn-nbctl --timeout=15 --may-exist --policy=src-ip --ecmp-symmetric-reply lr-route-add GR_compute-0 10.128.2.119/32 172.31.249.168' failed: exit status 1 2021-04-08T15:57:56.706Z|14452|nbctl|INFO|Running command run --may-exist --policy=src-ip --ecmp-symmetric-reply -- lr-route-add GR_compute-0 10.128.2.119/32 172.31.249.168
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:2438