Bug 2242323

Description of problem:
In ml2/ovs with l3-agent and keepalived, there seems to be a race during static route deletion: if keepalived removes the route first, then the neutron-l3-agent "ip route del" command would delete the remaining route (which is the neutron-l3-agent route), but if events happen in the reverse order, then neutron-l3-agent deletes keepalived route and its own route is never deleted.

Version-Release number of selected component (if applicable):
RHOSP 16.2 (Train), standard neutron-l3-agent container


How reproducible:
1) Add static route in neutron with L3 HA in place:
$ openstack router set a37b45d8-49fb-4281-add7-6247686f3f0f \
--route destination=1.2.3.4/32,gateway=2.3.4.5
2) On l3 agent, check routes:
# ip netns  exec qrouter-a37b45d8-49fb-4281-add7-6247686f3f0f ip route
1.2.3.4 via 2.3.4.5 dev qr-34dcdcb0-85 proto keepalived 
1.2.3.4 via 2.3.4.5 dev qr-34dcdcb0-85 
2.3.4.0/24 dev qr-34dcdcb0-85 proto kernel scope link src 2.3.4.1 
169.254.0.0/24 dev ha-600a323b-13 proto kernel scope link src 169.254.0.241 
169.254.192.0/18 dev ha-600a323b-13 proto kernel scope link src 169.254.193.208 
3) Delete the static route
$ openstack router unset a37b45d8-49fb-4281-add7-6247686f3f0f \
--route destination=1.2.3.4/32,gateway=2.3.4.5
4) On l3-agent, chekc routes:
# ip netns  exec qrouter-a37b45d8-49fb-4281-add7-6247686f3f0f ip route
1.2.3.4 via 2.3.4.5 dev qr-34dcdcb0-85  <== this route SHOULD HAVE BEEN DELETED!
2.3.4.0/24 dev qr-34dcdcb0-85 proto kernel scope link src 2.3.4.1 
169.254.0.0/24 dev ha-600a323b-13 proto kernel scope link src 169.254.0.241 
169.254.192.0/18 dev ha-600a323b-13 proto kernel scope link src 169.254.193.208 
Actual results:
Static route (not the one with 'proto keepalived') remains intact.

Expected results:
Static route should be deleted.


Additional info:
The issue is similar to the one described in https://bugs.launchpad.net/neutron/+bug/1956846, but here it's not default routes affected, but the extra routes in the Router object.