in kernel, the onlink flag (RTNH_F_ONLINK) is associated with each next hop (and not for the route alone).

Unlike for NetworkManager currently, where there is only one route-wide NMPlatformIP4Route.r_rtm_flags field.

---

1)

Note that the onlink flag is part of the identity of a route, that means, you can add two routes that only differ in their next-hop, like:

  # ip route append 1.2.3.4 nexthop via 192.168.9.1 dev v nexthop via 192.168.9.1 dev v
  # ip route append 1.2.3.4 nexthop via 192.168.9.1 dev v nexthop via 192.168.9.1 dev v onlink

gives:

  1.2.3.4 
          nexthop via 192.168.9.1 dev v weight 1 
          nexthop via 192.168.9.1 dev v weight 1 
  1.2.3.4 
          nexthop via 192.168.9.1 dev v weight 1 
          nexthop via 192.168.9.1 dev v weight 1 onlink 


NetworkManager currentlty cannot express that (see also NM_PLATFORM_IP_ROUTE_CMP_TYPE_ID). The effect is, that to NetworkManager those two routes look identical and they get combined as one entry in the platform cache. This needs fixing, by tracking the next hop flags per next-hop. Otherwise, cache inconsistencies can happen.


--

2) Also note that for single hop routes without gateway, kernel (pointlessly) does not allow setting the onlink flag:

  # ip route append 1.2.3.4 dev v onlink
  Error: Invalid flags for nexthop - PERVASIVE and ONLINK can not be set.
  # ip route append 1.2.3.4 nexthop dev v onlink
  Error: Invalid flags for nexthop - PERVASIVE and ONLINK can not be set.
  # ip route append 1.2.3.4 nexthop via 192.168.9.1 dev v onlink nexthop dev v onlink
  Error: Invalid flags for nexthop - PERVASIVE and ONLINK can not be set.

NetworkManager works around that by ignoring the "onlink" flag for IPv4 routes which don't have a gateway: https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/commit/58011fe88dd8e2601e0919bb9eb8549908957d63

---

3)

Also, NetworkManager can merge ECMP routes "weight". For that, it uses NM_PLATFORM_IP_ROUTE_CMP_TYPE_ECMP_ID. Note that two routes can only be merged to an ECMP route, if they agree in the onlink flag.

With point 2), this also means that a route without gateway cannot merge to an ECMP route with a route that has a gateway and the onlink flag set. That is a big limitation.



Points 1) and 3) need fixing.