Bug 243526

Summary: IPv6 default route does not work
Product: Red Hat Enterprise Linux 5 Reporter: Sander Steffann <s.steffann>
Component: kernelAssignee: Thomas Graf <tgraf>
Status: CLOSED ERRATA QA Contact: Martin Jenner <mjenner>
Severity: medium Docs Contact:
Priority: low    
Version: 5.0CC: berni+rh, cra, cward, davem, jfeeney, pasteur, ralph+rh-bugzilla, redhat-bugzilla, rkhan, seanos, tao, vfalico, wwlinuxengineering
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2009-01-20 20:26:01 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 409971    
Attachments:
Description Flags
upstream Linux changeset: [IPV6] ROUTE: No longer handle ::/0 specially
none
proposed patch none

Description Sander Steffann 2007-06-09 12:47:39 UTC
Description of problem:
The default IPv6 route, as configured in /etc/sysconfig/network with the
IPV6_DEFAULTGW option, does not work. When manually adding a route to 2000::/3
routing does work.

Version-Release number of selected component (if applicable):
RHEL5

How reproducible:
Add the IPV6_DEFAULTGW to /etc/sysconfig/network, and try to reach a non-local
network/host.

Steps to Reproduce:
1.Add the IPV6_DEFAULTGW, in our case IPV6_DEFAULTGW=2001:4038:0:100::1
2.'service network restart'
3.Check the routing table with 'ip -6 route'. Here it shows:

unreachable ::/96 dev lo  metric 1024  expires 21330552sec error -101 mtu 16436
advmss 16376 hoplimit 4294967295
unreachable ::ffff:0.0.0.0/96 dev lo  metric 1024  expires 21330552sec error
-101 mtu 16436 advmss 16376 hoplimit 4294967295
2001:4038:0:100::/64 dev eth0  metric 256  expires 21330542sec mtu 1500 advmss
1440 hoplimit 4294967295
2001:4038:0:101::/64 dev eth1  metric 256  expires 21330550sec mtu 1500 advmss
1440 hoplimit 4294967295
unreachable 2002:a00::/24 dev lo  metric 1024  expires 21330552sec error -101
mtu 16436 advmss 16376 hoplimit 4294967295
unreachable 2002:7f00::/24 dev lo  metric 1024  expires 21330552sec error -101
mtu 16436 advmss 16376 hoplimit 4294967295
unreachable 2002:a9fe::/32 dev lo  metric 1024  expires 21330552sec error -101
mtu 16436 advmss 16376 hoplimit 4294967295
unreachable 2002:ac10::/28 dev lo  metric 1024  expires 21330552sec error -101
mtu 16436 advmss 16376 hoplimit 4294967295
unreachable 2002:c0a8::/32 dev lo  metric 1024  expires 21330552sec error -101
mtu 16436 advmss 16376 hoplimit 4294967295
unreachable 2002:e000::/19 dev lo  metric 1024  expires 21330552sec error -101
mtu 16436 advmss 16376 hoplimit 4294967295
unreachable 3ffe:ffff::/32 dev lo  metric 1024  expires 21330552sec error -101
mtu 16436 advmss 16376 hoplimit 4294967295
fe80::/64 dev eth0  metric 256  expires 21330536sec mtu 1500 advmss 1440
hoplimit 4294967295
fe80::/64 dev eth1  metric 256  expires 21330545sec mtu 1500 advmss 1440
hoplimit 4294967295
default via 2001:4038:0:100::1 dev eth0  metric 1  expires 21334355sec mtu 1500
advmss 1440 hoplimit 4294967295

4.Ping a non-local IPv6 address with 'ping6 www.sixxs.net', which gives:
connect: Network is unreachable

5.Try routing 2000::/3 instead of default ::/0
'ip -6 route add 2000::/3 via 2001:4038:0:100::1 metric 1'

6.Check the new routing table with 'ip -6 route'. We get:

unreachable ::/96 dev lo  metric 1024  expires 21330410sec error -101 mtu 16436
advmss 16376 hoplimit 4294967295
unreachable ::ffff:0.0.0.0/96 dev lo  metric 1024  expires 21330410sec error
-101 mtu 16436 advmss 16376 hoplimit 4294967295
2001:4038:0:100::/64 dev eth0  metric 256  expires 21330400sec mtu 1500 advmss
1440 hoplimit 4294967295
2001:4038:0:101::/64 dev eth1  metric 256  expires 21330408sec mtu 1500 advmss
1440 hoplimit 4294967295
unreachable 2002:a00::/24 dev lo  metric 1024  expires 21330410sec error -101
mtu 16436 advmss 16376 hoplimit 4294967295
unreachable 2002:7f00::/24 dev lo  metric 1024  expires 21330410sec error -101
mtu 16436 advmss 16376 hoplimit 4294967295
unreachable 2002:a9fe::/32 dev lo  metric 1024  expires 21330410sec error -101
mtu 16436 advmss 16376 hoplimit 4294967295
unreachable 2002:ac10::/28 dev lo  metric 1024  expires 21330410sec error -101
mtu 16436 advmss 16376 hoplimit 4294967295
unreachable 2002:c0a8::/32 dev lo  metric 1024  expires 21330410sec error -101
mtu 16436 advmss 16376 hoplimit 4294967295
unreachable 2002:e000::/19 dev lo  metric 1024  expires 21330410sec error -101
mtu 16436 advmss 16376 hoplimit 4294967295
unreachable 3ffe:ffff::/32 dev lo  metric 1024  expires 21330410sec error -101
mtu 16436 advmss 16376 hoplimit 4294967295
2000::/3 via 2001:4038:0:100::1 dev eth0  metric 1  expires 21334334sec mtu 1500
advmss 1440 hoplimit 4294967295
fe80::/64 dev eth0  metric 256  expires 21330394sec mtu 1500 advmss 1440
hoplimit 4294967295
fe80::/64 dev eth1  metric 256  expires 21330402sec mtu 1500 advmss 1440
hoplimit 4294967295
default via 2001:4038:0:100::1 dev eth0  metric 1  expires 21334213sec mtu 1500
advmss 1440 hoplimit 4294967295

7.Try to ping again 'ping6 www.sixxs.net':
PING www.sixxs.net(noc.sixxs.net) 56 data bytes
64 bytes from noc.sixxs.net: icmp_seq=0 ttl=60 time=9.01 ms
64 bytes from noc.sixxs.net: icmp_seq=1 ttl=60 time=8.60 ms

Actual results:
See reproduction step 4

Expected results:
See reproduction step 7

Additional info:
Might be related to bug #142308

Comment 1 Charles R. Anderson 2008-01-29 20:40:45 UTC
What is the status of this bug?


Comment 3 Charles R. Anderson 2008-01-29 20:52:25 UTC
Created attachment 293338 [details]
upstream Linux changeset: [IPV6] ROUTE: No longer handle ::/0 specially

Comment 4 Charles R. Anderson 2008-01-30 18:47:53 UTC
Bug is still in RHEL 5.1.

kernel-2.6.18-53.1.6.el5PAE


Comment 5 Charles R. Anderson 2008-01-30 19:05:04 UTC
Example of how to workaround this bug by creating a static route to 2000::/3
that is persistent across network restarts and reboots:

/etc/sysconfig/static-routes-ipv6:

  eth0    2000::/3                               3ffe:ffff:0:1::1


Comment 6 Narendra K 2008-03-10 13:34:51 UTC
I have a setup where one of the machines acts as a router with IPv6 forwarding 
enabled. This machine has two nics and they have ips
       nic1 ip = 2001:4038:0:100::1/64 ( vlan 1 )
       nic2 ip = 2000:4038:0:100::1/64 ( vlan 2 )and are on two different 
subnets ( i have two vlans on the switch ).

I have connected a machine say machine-1 with ip 2001:4038:0:100::25/64 to 
vlan 1. It's /etc/sysconfig/network looks like

         NETWORKING=yes
         NETWORKING_IPV6=yes
         IPV6_DEFAULTGW=2001:4038:0:100::1
         HOSTNAME=localhost.localdomain

The other machine say machine-2 has ip 2000:4038:0:100::25/64 and is connected 
to vlan 2.

ip -6 route on machine-1 looks like this :

unreachable ::/96 dev lo  metric 1024  expires 21332077sec error -101 mtu 
16436 advmss 16376 hoplimit 4294967295
unreachable ::ffff:0.0.0.0/96 dev lo  metric 1024  expires 21332077sec error -
101 mtu 16436 advmss 16376 hoplimit 4294967295
2001:4038:0:100::/64 dev eth0  metric 256  expires 21332077sec mtu 1500 advmss 
1440 hoplimit 4294967295
unreachable 2002:a00::/24 dev lo  metric 1024  expires 21332077sec error -101 
mtu 16436 advmss 16376 hoplimit 4294967295
unreachable 2002:7f00::/24 dev lo  metric 1024  expires 21332077sec error -101 
mtu 16436 advmss 16376 hoplimit 4294967295
unreachable 2002:a9fe::/32 dev lo  metric 1024  expires 21332077sec error -101 
mtu 16436 advmss 16376 hoplimit 4294967295
unreachable 2002:ac10::/28 dev lo  metric 1024  expires 21332077sec error -101 
mtu 16436 advmss 16376 hoplimit 4294967295
unreachable 2002:c0a8::/32 dev lo  metric 1024  expires 21332077sec error -101 
mtu 16436 advmss 16376 hoplimit 4294967295
unreachable 2002:e000::/19 dev lo  metric 1024  expires 21332077sec error -101 
mtu 16436 advmss 16376 hoplimit 4294967295
unreachable 3ffe:ffff::/32 dev lo  metric 1024  expires 21332077sec error -101 
mtu 16436 advmss 16376 hoplimit 4294967295
fe80::/64 dev eth0  metric 256  expires 21332075sec mtu 1500 advmss 1440 
hoplimit 4294967295
default via 2001:4038:0:100::1 dev eth0  metric 1  expires 21332077sec mtu 
1500 advmss 1440 hoplimit 4294967295

and i can ping from machine-1 to machine-2 using ip's. I don't have a dns 
server. ping6  2000:4038:0:100::25 looks like 

PING 2000:4038:0:100::25(2000:4038:0:100::25) 56 data bytes
64 bytes from 2000:4038:0:100::25: icmp_seq=0 ttl=63 time=3.69 ms
64 bytes from 2000:4038:0:100::25: icmp_seq=1 ttl=63 time=0.202 ms
64 bytes from 2000:4038:0:100::25: icmp_seq=2 ttl=63 time=0.187 ms
64 bytes from 2000:4038:0:100::25: icmp_seq=3 ttl=63 time=0.240 ms
64 bytes from 2000:4038:0:100::25: icmp_seq=4 ttl=63 time=0.189 ms
64 bytes from 2000:4038:0:100::25: icmp_seq=5 ttl=63 time=0.216 ms
64 bytes from 2000:4038:0:100::25: icmp_seq=6 ttl=63 time=0.187 ms
64 bytes from 2000:4038:0:100::25: icmp_seq=7 ttl=63 time=0.201 ms

I see a "connect: Network is unreachable" only when i have commented 
IPV6_DEFAULTGW in machine-1.

It would be helpful if you can share configuration details of your setup for 
me to reproduce the issue.




Comment 7 Narendra K 2008-03-10 13:53:39 UTC
The traceroute output for the setup mentioned in comment #6 looks like this: 
 
traceroute to 2000:4038:0:100::25 (2000:4038:0:100::25), 30 hops max, 40 byte 
packets
 1   (2001:4038:0:100::1)  0.220 ms  0.156 ms  0.124 ms
 2   (2000:4038:0:100::25)  0.235 ms  0.223 ms  0.197 ms


Comment 8 Sander Steffann 2008-03-10 14:50:20 UTC
The machine where this problem occured has been replaced. It was an IPv6 router
between two subnets, and has been replaced with a 'real' router. Other IPv6
client machines don't have this problem anymore.

Since the problem only occured on our router, I suspect that the problem is
related to one or more of the following settings:
  IPV6_ROUTER
  IPV6_AUTOCONF
  IPV6FORWARDING

I am sorry I can't give any more details at this time. If you need me to perform
more tests, please let me know and I will schedule them.

Comment 9 Narendra K 2008-03-10 15:09:15 UTC
I enabled IPv6 forwarding on the machine that acted as a router by adding a 
line which looks like "net.ipv6.conf.all.forwarding=1" to /etc/sysctl.conf and 
executing sysctl -e -p /etc/sysctl.conf to dynamically enable forwarding. This 
might be useful.

Comment 10 Bernhard Schmidt 2008-03-11 23:51:30 UTC
First of all, let me emphasize that I'm neither a RHEL nor a CentOS user myself.
I hit this bug when it was introduced into 2.6.21-stable vanilla kernel series
and had several encounters with it later on, both with a customer running CentOS
and several people running RHEL or CentOS in #ipv6 on Freenode.

The bug itself is legit, it is in the kernel bugzille (
http://bugzilla.kernel.org/show_bug.cgi?id=8349 ), it has been tracked,
discussed and fixed on lkml by Simon Arlott, Chris Wright and David Miller (
http://lkml.org/lkml/2007/5/28/299 ). As you can read in these documents it is
caused by 

commit f11e6659ce9058928d73ff440f9b40a818d628ab
Author: David S. Miller <davem.net>
Date:   Sat Mar 24 20:36:25 2007 -0700
    [IPV6]: Fix routing round-robin locking.

during the 2.6.22 release cycle from where it was backported to 2.6.20-stable
and 2.6.21-stable. During this process it must have been picked up by RHEL
maintainers and integrated into the RHEL kernel sources (this patch is
integrated in http://centos.bio.lmu.de/5.1/os/SRPMS/kernel-2.6.18-53.el5.src.rpm
by the name "linux-2.6-ipv6-fix-routing-regression.patch").

As one can read on lkml this causes issues with manual defaultroutes. Apparently
not always, I've had cases where it just worked and cases where it didn't as
well. The proper fix, as visible on lkml as well, is

commit b407fdbabb6c630660e1e9c9c2d212a03b746775
Author: YOSHIFUJI Hideaki <yoshfuji>
Date:   Wed Jun 6 22:42:58 2007 -0700

    [PATCH] IPV6 ROUTE: No longer handle ::/0 specially.

I'm currently installing CentOS in VMware, but since this is a known bug we need
someone from RH to finally wake up and import the fix (for the "fix") into the
kernel sources as well.


Comment 11 Matt Domsch 2008-03-13 03:22:02 UTC
DaveM, as you were involved in the patches that invoked and fixed this problem
in mainline, can you persuade your colleagues on the RHEL kernel team to apply
the fix as well?

Thanks,
Matt

Comment 12 Thomas Graf 2008-06-18 21:25:48 UTC
Created attachment 309792 [details]
proposed patch

Comment 13 RHEL Program Management 2008-06-18 22:11:53 UTC
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
release.

Comment 15 Don Zickus 2008-07-18 20:06:29 UTC
in kernel-2.6.18-98.el5
You can download this test kernel from http://people.redhat.com/dzickus/el5

Comment 17 Chris Ward 2008-11-28 07:11:49 UTC
Partners, this bug should be fixed in the latest RHEL 5.3 Snapshot. We believe that you have some interest in its correct functionality, so we're making a friendly request to send us some testing feedback. 

If you have a chance to test it, please share with us your findings. If you have successfully VERIFIED the fix, please add PartnerVerified to the Bugzilla keywords, along with a description of the results. Thanks!

Comment 18 Jens Kuehnel 2008-12-11 13:05:00 UTC
Hi,

the kernel 2.6.18-120.x86_64 from RHEL5.3beta fixed this bug for me. 

I now can use aiccu from sixxs, without the the static route for 2000::/3 and only the default route.

I'm not a official Partner, but longtime Trainer for RedHat and RHCA#8 
If that is helping :-)

Greeting 
Jens

Comment 19 Chris Ward 2008-12-11 13:08:54 UTC
Works! Thanks Jens. Appreciate the support. Added CustomerVerified keyword.

Comment 22 errata-xmlrpc 2009-01-20 20:26:01 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2009-0225.html