+++ This bug was initially created as a clone of Bug #1651647 +++

Description of problem:

It is possible to create two subnets for single external network and it looks like we support such kind of setup. However, it doesn't work properly: every DVR will have a single on-link route for ONLY ONE external subnet and will not have on-link route for another one.

As a result, there are two scenarios when routing doesn't work as it should:

- two instances with FIPs from different external subnets will have asymmetric traffic flows: if there is an on-link route for DST packet, it will be routed directly to another DVR. If not, packet will be routed to external router. As a result, if there is a stateful firewall on external router, this communication will be blocked.
- two instances with FIPs from the same external subnet could have suboptimal traffic flows if there are no on-link routes for this subnet. As a result, traffic will always go through external router, which could cause issues if router has protection against such kind of flows.

Here is an example output of routing table in fip-* namespace on compute node where single external network has two subnets: 10.0.0.0/24 and 10.0.1.0/24:

default via 10.0.0.1 dev fg-83ec5f16-be table 2852022899 
10.0.0.0/24 dev fg-83ec5f16-be proto kernel scope link src 10.0.0.225 
10.0.0.218 via 169.254.106.114 dev fpr-789f245b-1 
10.0.0.219 via 169.254.106.114 dev fpr-789f245b-1 
10.0.1.15 via 169.254.106.114 dev fpr-789f245b-1

As we can see, there are host routes for local FIPs, a single on-link route for 10.0.0.0/24 subnet and default route.


Additional information:

An upstream bug for legacy routers [1] was solved long time ago.

[1] https://bugs.launchpad.net/neutron/+bug/1312467