RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1337855 - [RFE] support an identifier for routes
Summary: [RFE] support an identifier for routes
Keywords:
Status: CLOSED DEFERRED
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: kernel
Version: 7.3
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: rc
: ---
Assignee: Sabrina Dubroca
QA Contact: Jianlin Shi
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-05-20 09:16 UTC by Thomas Haller
Modified: 2021-06-10 10:06 UTC (History)
12 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-09-11 18:48:01 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Thomas Haller 2016-05-20 09:16:20 UTC
In kernel you can add routes to the same destination that only differ by any fields:

    unshare -n
    ip link add name L type dummy
    ip link set L up
    ip addr add 192.168.100.5/24 dev L

    ip route add    192.168.105.0/24 dev L
    ip route append 192.168.105.0/24 dev L advmss 100

yields:

    # ip route
    192.168.100.0/24 dev L  proto kernel  scope link  src 192.168.100.5 
    192.168.105.0/24 dev L  scope link 
    192.168.105.0/24 dev L  scope link  advmss 100


in this case, there are two routes that are identical except their advmss (RTAX_ADVMSS in RTA_METRICS).



A user-space application like NetworkManager or sd-networkd might want to maintain a cache of the kernel routes and only update them based on netlink events (RTM_NEW_ROUTE, RTM_DEL_ROUTE).

For example, the when user-space receives a 
  RTM_DEL_ROUTE(192.168.105.0/24 dev L advmss 100)
event, it must know to delete exactly that route from the cache, but keeping (192.168.105.0/24 dev L).


(1) That means, user-space must consider all (or just some? which?) fields of a route. If it uses a list of currently known fields, a new kernel version could break that by adding a new field. That is broken and requires constant adjustment in user-space. At least kernel should document *which* fields must be considered.

(2) Maybe another solution for user-space would be to consider the entire netlink message as an opaque ID. E.g. would kernel guarantee that the entire blob can be compared via memcmp()? If not, it seems hard to implement a equal-operator only using netlink paradigms without understanding what the attributes mean. Does kernel guarantee a stable format so that a netlink_cmp() function is reliable?

(3) A much nicer solution would be for kernel to add some int64 ID to each route.



This would greatly help NetworkManager. In fact, I doubt that there is *any* user-space application that gets "caching of routes based on netlink events" right. E.g. broken are:
  - NetworkManger: https://cgit.freedesktop.org/NetworkManager/NetworkManager/tree/src/platform/nmp-object.c?id=57c001200db44ef7077381d10f3c12f6a64e3690#n806
  - sd-networkd: https://github.com/systemd/systemd/blob/35b132e8ad3c50614605e00cf8ff20988094d21c/src/network/networkd-route.c#L134
  - libnl3: https://github.com/thom311/libnl/blob/648338a2170854952e605175af141ee27cc8dd2b/lib/route/route_obj.c#L1310


(for libnl3 there were patches on the mailing list to implement (1), but apart from being complicated they also don't fix the problem. It's really that hard to use the netlink API for routes).



Please int64 IDs.

Comment 2 Tom Gundersen 2016-05-20 14:47:38 UTC
I would love something like this! Currently it is really hard to get this right in userspace, and just adding an identifier (that is guaranteed to be stable, unique and never reused) to each message would solve the problem very nicely for us.

I spent some time trying to figure out how to do this in networkd (currently it is broken as Thomas points out). For LINK the IFINDEX gives us exactly what we want, for ADDRESS it is fine, as mimicking the equality implemented by the kernel is more or less trivial, but four ROUTE it is really complicated figuring out what exactly the notion of equality the kernel uses (exactly which fields needs to be compared in which circumstances).

If an ID is not possible, then at least some documentation exactly when two {NEW,DEL}ROUTE messages refer to the same, and when to different routes would be nice to have.

Comment 3 Jianlin Shi 2016-12-01 01:26:54 UTC
As the feature needs updates of both kernel-space and user-space. if kernel-space and user-space all support the feature described in Description, then we can test with the user-space tools: add routes to same destination with different fields, then check if user-space tool can show the all routes.

if user-space not support the feature yet when testing, then we would only check the codes of kernel-space.

set ACK+

Comment 4 Anant Deepak 2016-12-07 04:21:26 UTC
This would be very helpful. 

A simple case of adding the same prefix on difference devices 

Example: 
ff00::/8 dev eth0
ff00::/8 dev eth1

cacheManagers like libnl cannot differentiate these 2 routes in its cache. 

As Thomas pointed out, the userspace application does not consider the MULTIPATH attribute, so the 2 routes appear similar to libnl route cache.

Comment 5 Marcelo Ricardo Leitner 2019-07-03 15:11:05 UTC
Mass-moving bugs RHEL <= 7.6.0 to 7.7.0.
As we are past RFE deadline for 7.7.0 and we should have no new features on 7.8.0, please evaluate if it's still wanted on RHEL7 and contact PM for exception. You may also move it to RHEL8 if that's wanted. Thanks!

Comment 7 Marcelo Ricardo Leitner 2020-09-11 18:48:01 UTC
RHEL7.. dropping this one in favor of the RHEL8 one, https://bugzilla.redhat.com/show_bug.cgi?id=1722728.

Comment 8 Thomas Haller 2021-06-10 10:06:20 UTC
btw, for IPv6 routes there would be something special to consider:


Kernel allows adding two (otherwise identical) routes that only differ by their next-hop.
So one might think that the nexthop is part of the identity of an IPv6 route.

However, look at:


# ip route monitor &
# ip route append a:b:f::/64 nexthop via 1:2:3:4::120 dev v nexthop via 1:2:3:4::121 dev v 
a:b:f::/64 metric 1024 pref medium
        nexthop via 1:2:3:4::120 dev v weight 1 
        nexthop via 1:2:3:4::121 dev v weight 1 
# ip route append a:b:f::/64 nexthop via 1:2:3:4::122
a:b:f::/64 metric 1024 pref medium
        nexthop via 1:2:3:4::120 dev v weight 1 
        nexthop via 1:2:3:4::121 dev v weight 1 
        nexthop via 1:2:3:4::122 dev v weight 1 
# ip route delete a:b:f::/64 nexthop via 1:2:3:4::120
Deleted a:b:f::/64 via 1:2:3:4::120 dev v metric 1024 pref medium
# ip route append a:b:f::/64 nexthop via 1:2:3:4::125 dev v nexthop via 1:2:3:4::121 dev v 
RTNETLINK answers: File exists
a:b:f::/64 metric 1024 pref medium
        nexthop via 1:2:3:4::121 dev v weight 1 
        nexthop via 1:2:3:4::122 dev v weight 1 
        nexthop via 1:2:3:4::125 dev v weight 1 
Deleted a:b:f::/64 via 1:2:3:4::125 dev v metric 1024 pref medium



note that on netlink, kernel exposes one RTM_NEWROUTE instance (with muliple next hops), but
when you append/delete routes, then this route gets mangled. That means, the nexthop is
and is not part of the identifying properties of a IPv6 route.


I think for IPv6 routes, it would be more apt to see all multihop routes a several singlehop
routes, with the twist, that on the netlink API these singlehop routes a notified together.



For this rhbz that would mean, that the identifier for IPv6 is not a toplevel attribute of
the RTM_NEWROUTE, but instead each RTA_MULTIPATH has it's own identifier.

For IPv4 that is not the case, this merging/subtracting does not seem to happen and one RTM_NEWROUTE
message can have one identifier.


Note You need to log in before you can comment on or make changes to this bug.