Bug 567978

Summary: Unable to activate network in loader with [*] Enable IPv6 support
Product: [Fedora] Fedora Reporter: James Laska <jlaska>
Component: NetworkManagerAssignee: Dan Williams <dcbw>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: high    
Version: 13CC: awilliam, dbuggzie, dcbw, dhorak, franta, jklimes, jonathan, jturner, petersen, sascha, tore, vanmeeuwen+fedora, walters
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: NetworkManager-0.8.1-0.1.git20100510.fc12 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-05-04 23:51:12 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 507681, 578179    
Attachments:
Description Flags
anaconda.log
none
syslog
none
/tmp/*log after deselecting IPV6 and continuing into stage2
none
wireshark dump.pcap
none
/tmp/syslog none

Description James Laska 2010-02-24 14:32:32 UTC
Created attachment 396071 [details]
anaconda.log

Description of problem:

I'm unable to activate networking whenever the default selected option:

  [*] Enable IPv6 support

is used.  If I unselect this option, I'm able to activate the network, and proceed with the install.

Version-Release number of selected component (if applicable):
 * anaconda-13.29 (13.28, 13.27)

How reproducible:
 * everytime on both i386 and x86_64 (bare metal or virt)


Steps to Reproduce:
1. Boot the F-13-Alpha-RC2 vmlinuz/initrd.img
2. When prompted for networking, accept defaults, and select OK

                ┌─────────────┤ Configure TCP/IP ├─────────────┐                
                │                                              │                
                │ [*] Enable IPv4 support                      │                
                │        (*) Dynamic IP configuration (DHCP)   │                
                │        ( ) Manual configuration              │                
                │                                              │                
                │ [*] Enable IPv6 support                      │                
                │        (*) Automatic neighbor discovery      │                
                │        ( ) Dynamic IP configuration (DHCPv6) │                
                │        ( ) Manual configuration              │                
                │                                              │                
                │        ┌────┐               ┌──────┐         │                
                │        │ OK │               │ Back │         │                
                │        └────┘               └──────┘         │                
                │                                              │                
                │                                              │                
                └──────────────────────────────────────────────┘                

Actual results:

           ┌───────────────────────────────────────────────────────┐            
           │                                                       │            
           │ Waiting for NetworkManager to configure eth0.         │            
           │                                                       │            
           └───────────────────────────────────────────────────────┘            
                       ┌───────┤ Network Error ├────────┐                       
                       │                                │                       
                       │ There was an error configuring │                       
                       │ your network interface.        │                       
                       │                                │                       
                       │           ┌───────┐            │                       
                       │           │ Retry │            │                       
                       │           └───────┘            │                       
                       │                                │                       
                       │                                │                       
                       └────────────────────────────────┘                       

Expected results:

 * It should "just work" with or without IPV6 (Automatic neighbor discovery) selected.

Additional info:

 * Attaching /tmp/anaconda.log and /tmp/syslog from right after the "| Network Error |" above.

Comment 1 James Laska 2010-02-24 14:33:40 UTC
Created attachment 396072 [details]
syslog

Comment 2 James Laska 2010-02-24 14:35:06 UTC
Created attachment 396073 [details]
/tmp/*log after deselecting IPV6 and continuing into stage2

Attaching /tmp/*log after I deselect IPV6 and proceed with IPV4 networking into stage#2

-rw-r--r-- root/root      4294 2010-02-24 09:31 tmp/anaconda.log
-rw-r--r-- root/root       586 2010-02-24 09:31 tmp/program.log
-rw-r--r-- root/root      2379 2010-02-24 09:31 tmp/storage.log
-rw-r--r-- root/root     49133 2010-02-24 09:31 tmp/syslog
-rwxr-xr-x root/root       558 2010-02-24 09:31 tmp/vncserver.log

Comment 3 Chris Lumens 2010-02-24 16:32:51 UTC
From the bottom of your syslog:

14:25:30,003 NOTICE NetworkManager: (eth0): DHCPv6 request timed out.
14:25:30,005 WARN kernel:dhclient used greatest stack depth: 5732 bytes left
14:25:30,204 NOTICE NetworkManager: (eth0): canceled DHCP transaction, dhcp client pid 269
14:25:30,205 NOTICE NetworkManager: <info>  Activation (eth0) Stage 4 of 5 (IP6 Configure Timeout) scheduled...
14:25:30,205 NOTICE NetworkManager: <info>  Activation (eth0) Stage 4 of 5 (IP6 Configure Timeout) started...
14:25:30,205 NOTICE NetworkManager: <info>  (eth0): device state change: 7 -> 9 (reason 5)
14:25:30,206 NOTICE NetworkManager: <info>  Marking connection 'System eth0' invalid.
14:25:30,206 NOTICE NetworkManager: <info>  Activation (eth0) failed.
14:25:30,206 NOTICE NetworkManager: <info>  Activation (eth0) Stage 4 of 5 (IP6 Configure Timeout) complete.
14:25:30,208 NOTICE NetworkManager: <info>  (eth0): device state change: 9 -> 3 (reason 0)
14:25:30,208 NOTICE NetworkManager: <info>  (eth0): deactivating device (reason: 0).
14:25:30,210 NOTICE NetworkManager: (eth0): canceled DHCP transaction, dhcp client pid 267

Comment 4 James Laska 2010-03-12 13:34:19 UTC
This appears to be impacting automated RATS installs as well.

Dan, any thoughts on further isolating this failure?

Comment 5 Jirka Klimes 2010-03-12 14:21:57 UTC
There is a bug 569192 on this.

Basically, if a connection is configured both for IPv4 and IPv6, NM runs IPv6 configuration. However when a SLAAC or DHCPv6 fails (network is not IPv6 enabled), NM marks the connection as invalid.

Comment 6 James Laska 2010-03-23 12:28:00 UTC
Still a problem using F-13-Beta-TC1 test images at http://serverbeach1.fedoraproject.org/pub/alt/stage/13-Beta.TC1/

Comment 7 Dan Williams 2010-03-23 18:27:39 UTC
So the thing providing router advertisements is telling you to get a DHCPv6 address too:

14:24:44,358 NOTICE NetworkManager: <info>  Activation (eth0) Stage 3 of 5 (IP Configure Start) starting DHCPv6 as requested by IPv6 router...
...
14:25:30,003 NOTICE NetworkManager: (eth0): DHCPv6 request timed out.

which them times out.  Is the network configured incorrectly such that dhcpv6 is not available on the link despite the IPv6 router advertisement indicating that it should be?  Or do we have an iptables issue here?

Comment 8 James Laska 2010-03-23 19:32:35 UTC
Created attachment 402127 [details]
wireshark dump.pcap

Comment 9 Colin Walters 2010-03-31 17:11:17 UTC
I can reproduce this here reliably on Fedora 12, using virt-install (default NAT networking).  Shouldn't dnsmasq be answering requests in this situation?  (time passes)

From "man dnsmasq":

Dnsmasq supports IPv6 for DNS, but not DHCP.

So my guess is what's happening here is that it's passing through the request, and then we're subject to the host router.

Comment 10 Dan Williams 2010-04-08 22:59:59 UTC
Yeah, you need ISC dhcpd for DHCP6.  dnsmasq doesn't cut it.

So what's the resolution?  Mark as notabug and chalk it up to user awareness?

One option is to add a "try for IPvX but if it fails that's OK" option.

Comment 11 Adam Williamson 2010-04-16 20:19:42 UTC
Discussed at today's blocker review meeting. We all agreed we'd like to do something about this for F13, as it'll affect all virt cases where the loader needs to bring up the network, and may affect bare metal cases depending on router capabilities.

If we can come up with something that'll improve this without making big invasive changes that would be good. We like Dan's option of making the default be to try IPv6 but fall back gracefully if it fails, rather than throwing the toys out of the pram...if that could be implemented without too much trouble it'd be ideal. Otherwise it may make sense to default to IPv4, especially since stage2 doesn't even offer IPv6 yet, as far as we know.



-- 
Fedora Bugzappers volunteer triage team
https://fedoraproject.org/wiki/BugZappers

Comment 12 Dan Horák 2010-04-19 08:43:35 UTC
*** Bug 583609 has been marked as a duplicate of this bug. ***

Comment 13 Adam Williamson 2010-04-23 19:09:08 UTC
Discussed again at this week's blocker review meeting. Dan, are you waiting for approval / advice from anyone else on this? We'd like to remove any roadblocks so we can get a fix ASAP, final release deadlines start creeping up on us :)



-- 
Fedora Bugzappers volunteer triage team
https://fedoraproject.org/wiki/BugZappers

Comment 14 Dan Williams 2010-04-29 16:28:12 UTC
(In reply to comment #13)
> Discussed again at this week's blocker review meeting. Dan, are you waiting for
> approval / advice from anyone else on this? We'd like to remove any roadblocks
> so we can get a fix ASAP, final release deadlines start creeping up on us :)

Now that the IPv6 rework bits have landed this is my top issue.

Comment 15 Dan Williams 2010-04-30 00:33:55 UTC
*** Bug 586678 has been marked as a duplicate of this bug. ***

Comment 16 Adam Williamson 2010-04-30 19:20:37 UTC
Discussed again at the blocker meeting today, no new info. Dan, just to be sure, please be aware we have a target date of Tuesday 2010/05/04 to work with, as that's when we should start building RCs - are you likely to be able to get the fix in by then? Thanks.



-- 
Fedora Bugzappers volunteer triage team
https://fedoraproject.org/wiki/BugZappers

Comment 17 Dan Williams 2010-04-30 19:28:57 UTC
Yeah, hopefully landing this tonight and pushing the testing updates.

Comment 18 Dan Williams 2010-05-03 08:35:38 UTC
Support committed upstream as of 13e1aaa423e3ad65a849b55d65f273272b454336 (nm) and a2113fe42c19ab7ea3cc7ff06f37a9acc4f68f63 (nm-applet).

Comment 19 Dan Williams 2010-05-03 10:05:02 UTC
*** Bug 588051 has been marked as a duplicate of this bug. ***

Comment 20 Fedora Update System 2010-05-03 12:21:57 UTC
NetworkManager-0.8.0-11.git20100503.fc13 has been submitted as an update for Fedora 13.
http://admin.fedoraproject.org/updates/NetworkManager-0.8.0-11.git20100503.fc13

Comment 21 Fedora Update System 2010-05-03 12:26:26 UTC
NetworkManager-0.8.0-11.git20100503.fc12 has been submitted as an update for Fedora 12.
http://admin.fedoraproject.org/updates/NetworkManager-0.8.0-11.git20100503.fc12

Comment 22 Dan Williams 2010-05-03 12:33:28 UTC
James et al; please give these latest builds a shot; they should allow for IPv6 configuration to fail by default and still have networking up and running.  Also if you have the chance, bang on them for anything you can think of since the code churned a bit with the fix for this bug.  Things to look for after you get installed: spinning green balls where NM never connects, failures where you'd expect success, etc.

Comment 23 James Laska 2010-05-03 14:37:01 UTC
(In reply to comment #22)
> James et al; please give these latest builds a shot; they should allow for IPv6
> configuration to fail by default and still have networking up and running. 
> Also if you have the chance, bang on them for anything you can think of since
> the code churned a bit with the fix for this bug.  Things to look for after you
> get installed: spinning green balls where NM never connects, failures where
> you'd expect success, etc.    

Since this occurs during install time, I'll need to ask rel-eng for custom install images with those builds from comment#20.  Will reply asap, thanks!

Comment 24 Adam Williamson 2010-05-03 23:14:57 UTC
New NetworkManager successfully connects to my wireless network, can't easily test the actual bug at present.



-- 
Fedora Bugzappers volunteer triage team
https://fedoraproject.org/wiki/BugZappers

Comment 25 James Laska 2010-05-04 00:24:18 UTC
Created attachment 411162 [details]
/tmp/syslog

Release Engineering provided test install images for verifying this issue inside the installer (http://serverbeach1.fedoraproject.org/pub/alt/stage/13.nmtest/Fedora/i386/os/).  Using the images provided, I'm able to activate networking for virt guests, using the defaults.  I have attached /tmp/syslog output for confirmation.

I'll run this on a few more systems in the morning, but this appears to resolve the reported issue.

Comment 26 Fedora Update System 2010-05-04 06:13:01 UTC
NetworkManager-0.8.0-11.git20100503.fc13 has been pushed to the Fedora 13 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update NetworkManager'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/NetworkManager-0.8.0-11.git20100503.fc13

Comment 27 Fedora Update System 2010-05-04 06:14:54 UTC
NetworkManager-0.8.0-11.git20100503.fc12 has been pushed to the Fedora 12 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update NetworkManager'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/NetworkManager-0.8.0-11.git20100503.fc12

Comment 28 Fedora Update System 2010-05-04 23:50:33 UTC
NetworkManager-0.8.0-11.git20100503.fc13 has been pushed to the Fedora 13 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 29 Fedora Update System 2010-05-05 09:16:05 UTC
NetworkManager-0.8.0-12.git20100504.fc13 has been submitted as an update for Fedora 13.
http://admin.fedoraproject.org/updates/NetworkManager-0.8.0-12.git20100504.fc13

Comment 30 Fedora Update System 2010-05-05 09:18:48 UTC
NetworkManager-0.8.0-12.git20100504.fc12 has been submitted as an update for Fedora 12.
http://admin.fedoraproject.org/updates/NetworkManager-0.8.0-12.git20100504.fc12

Comment 31 Fedora Update System 2010-05-06 07:02:08 UTC
NetworkManager-0.8.0-12.git20100504.fc13 has been pushed to the Fedora 13 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 32 Fedora Update System 2010-05-11 06:54:49 UTC
NetworkManager-0.8.1-0.1.git20100510.fc12 has been submitted as an update for Fedora 12.
http://admin.fedoraproject.org/updates/NetworkManager-0.8.1-0.1.git20100510.fc12

Comment 33 Fedora Update System 2010-06-10 19:06:26 UTC
NetworkManager-0.8.1-0.1.git20100510.fc12 has been pushed to the Fedora 12 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 34 Dan Williams 2010-08-17 16:31:18 UTC
*** Bug 569024 has been marked as a duplicate of this bug. ***