Description of problem:
It is possible to create two subnets for single external network and it looks like we support such kind of setup. However, it doesn't work properly: every DVR will have a single on-link route for ONLY ONE external subnet and will not have on-link route for another one.
As a result, there are two scenarios when routing doesn't work as it should:
- two instances with FIPs from different external subnets will have asymmetric traffic flows: if there is an on-link route for DST packet, it will be routed directly to another DVR. If not, packet will be routed to external router. As a result, if there is a stateful firewall on external router, this communication will be blocked.
- two instances with FIPs from the same external subnet could have suboptimal traffic flows if there are no on-link routes for this subnet. As a result, traffic will always go through external router, which could cause issues if router has protection against such kind of flows.
Here is an example output of routing table in fip-* namespace on compute node where single external network has two subnets: 10.0.0.0/24 and 10.0.1.0/24:
default via 10.0.0.1 dev fg-83ec5f16-be table 2852022899
10.0.0.0/24 dev fg-83ec5f16-be proto kernel scope link src 10.0.0.225
10.0.0.218 via 169.254.106.114 dev fpr-789f245b-1
10.0.0.219 via 169.254.106.114 dev fpr-789f245b-1
10.0.1.15 via 169.254.106.114 dev fpr-789f245b-1
As we can see, there are host routes for local FIPs, a single on-link route for 10.0.0.0/24 subnet and default route.
An upstream bug for legacy routers  was solved long time ago.