Bug 1470930 - NetworkManager clears IPv6 PMTU cache when ICMPv6 Router Advertisement is received
NetworkManager clears IPv6 PMTU cache when ICMPv6 Router Advertisement is rec...
Status: MODIFIED
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: NetworkManager (Show other bugs)
7.3
x86_64 Linux
low Severity low
: rc
: ---
Assigned To: Beniamino Galvani
Desktop QE
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-07-13 23:26 EDT by Jeremy Visser
Modified: 2017-08-29 08:09 EDT (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Reproducer script (1.93 KB, application/x-shellscript)
2017-07-31 15:02 EDT, Beniamino Galvani
no flags Details
[PATCH] default-route: skip addition when the route already exists (3.62 KB, patch)
2017-08-01 03:16 EDT, Beniamino Galvani
no flags Details | Diff
[PATCH v2] default-route: skip addition when the route already exists (4.60 KB, patch)
2017-08-28 10:23 EDT, Beniamino Galvani
no flags Details | Diff

  None (edit)
Description Jeremy Visser 2017-07-13 23:26:00 EDT
Description of problem:

If NetworkManager is configured to manage IPv6 (i.e. IPV6INIT=yes) on an interface where Router Advertisements (RA) are used, each time an RA is received, the PMTU cache is flushed.

The impact of flushing the PMTU cache means if the Linux machine is actively communicating with another machine behind an MTU bottleneck, each time an RA is received it will start sending too-large packets, and needs to rely on another ICMPv6 Packet Too Big message before it can send again.

Depending on how frequent RAs are sent on the network or the ICMP rate-limiting behaviour of the intermediate router, the right circumstances can cause several-second hangs in TCP connections.

Version-Release number of selected component (if applicable):

NetworkManager-1.4.0-20.el7_3.x86_64

How reproducible:

This is easily reproducible on EL7 when NetworkManager is set to IPV6INIT=yes and an MTU bottleneck exists on an intermediate network (not the EL machine's directly connected network) and ICMP Router Advertisements are sent frequently.

Steps to Reproduce:

1. Set up EL7 with an IPv6 connection using Router Advertisements (not stateful DHCPv6) and NetworkManager managing the network interface (this is the default configuration).

2. To make the issue easier to reproduce, configure the router next to the EL7 box to sent RAs every 5 seconds.

3. Configure an MTU bottleneck on the far side of the connection, such that when the EL7 box tries to send traffic, it receives an ICMP Packet Too Big message in response.

4. Send some data from the EL7 box to the other box:

    el7$       nc otherhost 9999 < /dev/urandom
    otherhost$ nc -l -p 9999     > /dev/null

5. Perform a packet capture to observe what the resulting behaviour is from receiving an ICMP Packet Too Big, and an ICMP Router Advertisement.

6. Run this command to view the route cache entry for the remote host:

     $ remote_host=2001:db8::1337
     $ while [ 1 ] ; do ip -6 route get $remote_host ; sleep 0.2 ; done

7. Now try again by reconfiguring NetworkManager to not manage IPv6. (i.e. change IPV6INIT=no).

Actual results:

The output of "ip -6 route get $remote_host" will flap between these two values depending on whether an ICMP Packet Too Big or an ICMP Router Advertisement was received most recently:

  good)  2001:db8::1337 via fe80::1  dev ens192  cache  expiry 599  mtu 1398
  bad)   2001:db8::1337 via fe80::1  dev ens192  cache

Expected results:

I expect receiving an ICMP Router Advertisement to be an idempotent operation. Sure, if a route changes, I fully expect the route cache to be cleared.

However, if the RA contains no new information, and the kernel already has the correct table, it should NOT be touched.

Poking the kernel route table when no changes are necessary is considered harmful.
Comment 2 Beniamino Galvani 2017-07-31 15:02 EDT
Created attachment 1307217 [details]
Reproducer script
Comment 3 Beniamino Galvani 2017-08-01 03:16 EDT
Created attachment 1307398 [details]
[PATCH] default-route: skip addition when the route already exists
Comment 4 Thomas Haller 2017-08-18 11:36:00 EDT
(In reply to Beniamino Galvani from comment #3)
> Created attachment 1307398 [details]
> [PATCH] default-route: skip addition when the route already exists

patch (and approach) locks good to me for nm-1-8 branch.

But there are conflicts with master.
Comment 5 Thomas Haller 2017-08-18 11:37:07 EDT
ok, one thing: 

+                   _LOGt (AF_INET6, "update platform route: %s",
+                          nm_platform_ip6_route_to_string (plat_rt, NULL, 0));
+                   _LOGt (AF_INET6, "                 with: %s",
+                          nm_platform_ip6_route_to_string (&rt, NULL, 0));

maybe not log two lines? :) I know, it's only --with-more-logging, but our logging is already so verbose, please combine the lines.
Comment 6 Beniamino Galvani 2017-08-28 10:23 EDT
Created attachment 1319087 [details]
[PATCH v2] default-route: skip addition when the route already exists

Patch rebased on master.

Note You need to log in before you can comment on or make changes to this bug.