Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
in kernel, the onlink flag (RTNH_F_ONLINK) is associated with each next hop (and not for the route alone).
Unlike for NetworkManager currently, where there is only one route-wide NMPlatformIP4Route.r_rtm_flags field.
---
1)
Note that the onlink flag is part of the identity of a route, that means, you can add two routes that only differ in their next-hop, like:
# ip route append 1.2.3.4 nexthop via 192.168.9.1 dev v nexthop via 192.168.9.1 dev v
# ip route append 1.2.3.4 nexthop via 192.168.9.1 dev v nexthop via 192.168.9.1 dev v onlink
gives:
1.2.3.4
nexthop via 192.168.9.1 dev v weight 1
nexthop via 192.168.9.1 dev v weight 1
1.2.3.4
nexthop via 192.168.9.1 dev v weight 1
nexthop via 192.168.9.1 dev v weight 1 onlink
NetworkManager currentlty cannot express that (see also NM_PLATFORM_IP_ROUTE_CMP_TYPE_ID). The effect is, that to NetworkManager those two routes look identical and they get combined as one entry in the platform cache. This needs fixing, by tracking the next hop flags per next-hop. Otherwise, cache inconsistencies can happen.
--
2) Also note that for single hop routes without gateway, kernel (pointlessly) does not allow setting the onlink flag:
# ip route append 1.2.3.4 dev v onlink
Error: Invalid flags for nexthop - PERVASIVE and ONLINK can not be set.
# ip route append 1.2.3.4 nexthop dev v onlink
Error: Invalid flags for nexthop - PERVASIVE and ONLINK can not be set.
# ip route append 1.2.3.4 nexthop via 192.168.9.1 dev v onlink nexthop dev v onlink
Error: Invalid flags for nexthop - PERVASIVE and ONLINK can not be set.
NetworkManager works around that by ignoring the "onlink" flag for IPv4 routes which don't have a gateway: https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/commit/58011fe88dd8e2601e0919bb9eb8549908957d63
---
3)
Also, NetworkManager can merge ECMP routes "weight". For that, it uses NM_PLATFORM_IP_ROUTE_CMP_TYPE_ECMP_ID. Note that two routes can only be merged to an ECMP route, if they agree in the onlink flag.
With point 2), this also means that a route without gateway cannot merge to an ECMP route with a route that has a gateway and the onlink flag set. That is a big limitation.
Points 1) and 3) need fixing.
Comment 2RHEL Program Management
2023-08-17 07:27:45 UTC
Issue migration from Bugzilla to Jira is in process at this time. This will be the last message in Jira copied from the Bugzilla bug.
in kernel, the onlink flag (RTNH_F_ONLINK) is associated with each next hop (and not for the route alone). Unlike for NetworkManager currently, where there is only one route-wide NMPlatformIP4Route.r_rtm_flags field. --- 1) Note that the onlink flag is part of the identity of a route, that means, you can add two routes that only differ in their next-hop, like: # ip route append 1.2.3.4 nexthop via 192.168.9.1 dev v nexthop via 192.168.9.1 dev v # ip route append 1.2.3.4 nexthop via 192.168.9.1 dev v nexthop via 192.168.9.1 dev v onlink gives: 1.2.3.4 nexthop via 192.168.9.1 dev v weight 1 nexthop via 192.168.9.1 dev v weight 1 1.2.3.4 nexthop via 192.168.9.1 dev v weight 1 nexthop via 192.168.9.1 dev v weight 1 onlink NetworkManager currentlty cannot express that (see also NM_PLATFORM_IP_ROUTE_CMP_TYPE_ID). The effect is, that to NetworkManager those two routes look identical and they get combined as one entry in the platform cache. This needs fixing, by tracking the next hop flags per next-hop. Otherwise, cache inconsistencies can happen. -- 2) Also note that for single hop routes without gateway, kernel (pointlessly) does not allow setting the onlink flag: # ip route append 1.2.3.4 dev v onlink Error: Invalid flags for nexthop - PERVASIVE and ONLINK can not be set. # ip route append 1.2.3.4 nexthop dev v onlink Error: Invalid flags for nexthop - PERVASIVE and ONLINK can not be set. # ip route append 1.2.3.4 nexthop via 192.168.9.1 dev v onlink nexthop dev v onlink Error: Invalid flags for nexthop - PERVASIVE and ONLINK can not be set. NetworkManager works around that by ignoring the "onlink" flag for IPv4 routes which don't have a gateway: https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/commit/58011fe88dd8e2601e0919bb9eb8549908957d63 --- 3) Also, NetworkManager can merge ECMP routes "weight". For that, it uses NM_PLATFORM_IP_ROUTE_CMP_TYPE_ECMP_ID. Note that two routes can only be merged to an ECMP route, if they agree in the onlink flag. With point 2), this also means that a route without gateway cannot merge to an ECMP route with a route that has a gateway and the onlink flag set. That is a big limitation. Points 1) and 3) need fixing.