Bug 588560

Summary: Bogus IPv6 default route added when connecting to a IPv4-only network using IPv6 mode auto
Product: [Fedora] Fedora Reporter: Tore Anderson <tore>
Component: NetworkManagerAssignee: Dan Williams <dcbw>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: low    
Version: 12CC: dcbw, i.grok
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: NetworkManager-0.8.1-0.1.git20100510.fc12 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-06-10 15:08:04 EDT Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Bug Depends On:    
Bug Blocks: 538499    
Attachments:
Description Flags
/var/log/messages
none
NM debug output none

Description Tore Anderson 2010-05-03 18:54:49 EDT
Created attachment 411155 [details]
/var/log/messages

NetworkManager-0.8.0-11.git20100503.fc12.x86_64

When connecting to a IPv4-only network (no radvd, no dhcpd6), using IPv6 mode "automatic", a bogus default IPv6 route is added:

default dev eth0  proto static  metric 1024  mtu 1500 advmss 1440 hoplimit 4294967295

This was added at 00:42:48, see attached log.  This is potentially _very_ problematic - if a globally scoped IPv6 address is present on the system for some reason (e.g. if there's DHCPv6 service on the LAN but no RAs, or if dhclient falls back on an earlier recorded lease), getaddrinfo() will sort IPv6 addresses before IPv4 ones due to this bogus route making them look "reachable".  This will in turn make dual-stacked sites like for instance www.fedoraproject.org appear unreachable or extremely slow as all initial connection attempts will be routed into an IPv6 blackhole and have to time out before a fallback to IPv4 is done.

BTW:  I'm not sure if you're supposed to start DHCPv6 transactions at all if there's no ICMPv6 RA with OtherConfig or Managed set.  I'm not familiar with the standards here.  What I do know, though, is that DHCPv6 can (presently) not deliver a standard gateway address - ICMPv6 RA is the _only_ way that can do this at the moment.  So if there's no ICMPv6 RAs on the network, no routes should be added (except link-locals).

Tore
Comment 1 Dan Williams 2010-05-04 17:24:56 EDT
The logs indicate that RA *did* request other/managed:

May  4 00:42:44 wrath NetworkManager: <info> Activation (eth0) Stage 3 of 5 (IP Configure Start) starting DHCPv6 as requested by IPv6 router...
May  4 00:42:44 wrath NetworkManager: <info> Activation (eth0) Beginning DHCPv6 transaction (timeout in 45 seconds)

So I'd expect that based on that you'd get an IPv6 default route. Something is running RA on your network...

DHCPv6 only gets started if:

1) RA requests it via Managed/Other and you have "Automatic" mode set
2) Your ifcfg file has "IPV6_AUTOCONF" set and RA requests Managed/Other
3) You use "Automatic (DHCP only)" method in the connection editor
4) Your ifcfg file has "DHCP6C" set

Looks like here you're hitting #1 or #2...

Are you sure you're not running RA on this network?
Comment 2 Tore Anderson 2010-05-04 22:34:14 EDT
(In reply to comment #1)

> The logs indicate that RA *did* request other/managed:
> 
> May  4 00:42:44 wrath NetworkManager: <info> Activation (eth0) Stage 3 of 5 (IP
> Configure Start) starting DHCPv6 as requested by IPv6 router...
> May  4 00:42:44 wrath NetworkManager: <info> Activation (eth0) Beginning DHCPv6
> transaction (timeout in 45 seconds)
> 
> So I'd expect that based on that you'd get an IPv6 default route. Something is
> running RA on your network...
> 
> DHCPv6 only gets started if:
> 
> 1) RA requests it via Managed/Other and you have "Automatic" mode set
> 2) Your ifcfg file has "IPV6_AUTOCONF" set and RA requests Managed/Other
> 3) You use "Automatic (DHCP only)" method in the connection editor
> 4) Your ifcfg file has "DHCP6C" set
> 
> Looks like here you're hitting #1 or #2...
> 
> Are you sure you're not running RA on this network?

Yep, just reproduced it.  radvd is stopped on the router, and tcpdump on eth0 while activating the interface did not show any RAs being received at all.

Also, the fact that the bogus route do not have a next-hop is quite telling.  A RA-learned default route would have had a next-hop set (to the link-local address of the router originating the RA).

But now I have to run to catch a plane..  Won't be able to help out with more debugging until after a week.

Tore
Comment 3 Dan Williams 2010-05-05 01:47:03 EDT
Hmm, cannot reproduce this issue with radvd stopped on the router and an Auto/Auto connection.  Whenever you get a chance to test this again, please:

1) service NetworkManager stop
2) /usr/sbin/NetworkManager --no-daemon --log-level=debug

and try to reproduce the problem.  It'll give us a lot more information about what netlink events are coming from the kernel.
Comment 4 Tore Anderson 2010-05-07 00:31:56 EDT
Created attachment 412220 [details]
NM debug output

I managed to reproduce it remotely.  I stopped radvd and dhcpd6 on the router, and then ran the following script on the host:

service NetworkManager stop
rm /var/lib/dhclient/*.lease
ip a | tee -a log.txt
ip -6 r | tee -a log.txt
(sleep 60s; ip a; ip -6 r) | tee -a log.txt &
/usr/sbin/NetworkManager --no-daemon --log-level=debug 2>&1 | tee -a log.txt

I'm attaching the resulting log.txt.  As you can see, a bogus default route with no next-hop has been added sometime during those sixty seconds.

Tore
Comment 5 Dan Williams 2010-05-10 18:21:30 EDT
Thanks for the logs; I think I see what's going on here.  Upstream fix is 3e68d3358311cd04ae85884ef593da7554f4b3e3
Comment 6 Fedora Update System 2010-05-11 02:49:14 EDT
NetworkManager-0.8.1-0.1.git20100510.fc13 has been submitted as an update for Fedora 13.
http://admin.fedoraproject.org/updates/NetworkManager-0.8.1-0.1.git20100510.fc13
Comment 7 Fedora Update System 2010-05-11 02:55:18 EDT
NetworkManager-0.8.1-0.1.git20100510.fc12 has been submitted as an update for Fedora 12.
http://admin.fedoraproject.org/updates/NetworkManager-0.8.1-0.1.git20100510.fc12
Comment 8 Tore Anderson 2010-05-11 14:44:20 EDT
Seems to work much better, thanks!

Tore
Comment 9 Fedora Update System 2010-05-11 15:42:19 EDT
NetworkManager-0.8.1-0.1.git20100510.fc12 has been pushed to the Fedora 12 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update NetworkManager'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/NetworkManager-0.8.1-0.1.git20100510.fc12
Comment 10 Fedora Update System 2010-05-12 08:22:08 EDT
NetworkManager-0.8.1-0.1.git20100510.fc13 has been pushed to the Fedora 13 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update NetworkManager'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/NetworkManager-0.8.1-0.1.git20100510.fc13
Comment 11 Fedora Update System 2010-06-10 15:06:58 EDT
NetworkManager-0.8.1-0.1.git20100510.fc12 has been pushed to the Fedora 12 stable repository.  If problems still persist, please make note of it in this bug report.