Bug 1364281 - Network not brought up correctly since 20160714.n.0 nightlies
Summary: Network not brought up correctly since 20160714.n.0 nightlies
Keywords:
Status: CLOSED DUPLICATE of bug 1358416
Alias: None
Product: Fedora
Classification: Fedora
Component: anaconda
Version: rawhide
Hardware: x86_64
OS: Linux
unspecified
urgent
Target Milestone: ---
Assignee: Anaconda Maintenance Team
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-08-04 23:49 UTC by Adam Williamson (Red Hat non-Fedora)
Modified: 2016-08-05 15:09 UTC (History)
4 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2016-08-05 15:09:47 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
journal from boot of Fedora-Server-dvd-x86_64-Rawhide-20160708.n.0.iso on KVM with libvirt NAT networking (197.86 KB, text/plain)
2016-08-05 00:24 UTC, Adam Williamson (Red Hat non-Fedora)
no flags Details
journal from boot of Fedora-Server-dvd-x86_64-Rawhide-20160715.n.0.iso on KVM with libvirt NAT networking (203.33 KB, text/plain)
2016-08-05 00:25 UTC, Adam Williamson (Red Hat non-Fedora)
no flags Details
`nmcli conn show ens3` output from boot of Fedora-Server-dvd-x86_64-Rawhide-20160708.n.0.iso on KVM with libvirt NAT networking (5.87 KB, text/plain)
2016-08-05 00:42 UTC, Adam Williamson (Red Hat non-Fedora)
no flags Details
`nmcli conn show ens3` output from boot of Fedora-Server-dvd-x86_64-Rawhide-20160715.n.0.iso on KVM with libvirt NAT networking (4.75 KB, text/plain)
2016-08-05 00:44 UTC, Adam Williamson (Red Hat non-Fedora)
no flags Details

Description Adam Williamson (Red Hat non-Fedora) 2016-08-04 23:49:43 UTC
In several configurations it seems that the network connection is not correctly brought up by anaconda, since Fedora-Rawhide-20160714.n.0.

It's easy to observe this bug, if you're affected: just boot any installer image with a working ethernet connection and either try to ping 8.8.8.8 (it won't work) or go to the NETWORK & HOST NAME spoke (it'll show 'Connected', but no 'Speed', 'Default Route' or 'DNS'). If you use a netinst image, it'll show 'Error setting up base repository' at the hub, also.

I'm observing this effect in three different configurations here:

1. KVM with standard libvirt NAT networking
2. KVM with networking via a bridge
3. Bare metal system (on the same network)

If I boot a 20160708.n.0 image, the network works fine. If I boot a 20160715.n.0 image (I don't have a 20160714.n.0 image lying around locally and they've been garbage-collected from https://kojipkgs.fedoraproject.org/compose/rawhide ) the bug appears.

I believe it's also what's causing the install_kickstart_nfs openQA tests to fail; they started failing exactly with the 20160714.n.0 compose (which is why I identify that as the first broken compose). You can see this at https://openqa.stg.fedoraproject.org/tests/30327?limit_previous=20#previous .

Other openQA tests are not affected, though; the kickstart_nfs test uses a somewhat different network configuration from the default openQA config, suggesting the default openQA config (which uses qemu's default networking, IIRC) is not affected but the network config used by the kickstart_nfs test (which is more or less a bridged configuration set up via openvswitch) is affected.

There were no images for the composes between 20160708.n.0 and 20160714.n.0 so we can't pin down precisely where the breakage occurred between those two composes, but there are some network-related changes in anaconda-25.20-1.fc25 which landed on 20160709. I've tried reverting each of those and haven't yet found that any reversion avoids the bug. It's possible the bug isn't in anaconda, but if not anaconda, I don't know where; NetworkManager was not changed in the affected time frame.

Comment 1 Adam Williamson (Red Hat non-Fedora) 2016-08-05 00:06:40 UTC
The issue does seem to be in the installer environment somehow, though - live images do not seem to be affected.

Comment 2 Adam Williamson (Red Hat non-Fedora) 2016-08-05 00:24:54 UTC
Created attachment 1187681 [details]
journal from boot of Fedora-Server-dvd-x86_64-Rawhide-20160708.n.0.iso on KVM with libvirt NAT networking

Attaching journals from 20160708.n.0 (working case) and 20160715.n.0 (broken case) on the same VM. The NetworkManager messages seem to diverge, at least, at the end. In the 20160708.n.0 case, we see this:

Aug 05 00:07:48 localhost NetworkManager[1573]: <info>  [1470355668.1997] manager: NetworkManager state is now CONNECTED_LOCAL
Aug 05 00:07:48 localhost NetworkManager[1573]: <debug> [1470355668.1999] active-connection[0x55a3fe6f34f0]: check-master-ready: not signalling (state activated, no master)
Aug 05 00:07:48 localhost NetworkManager[1573]: <debug> [1470355668.2001] device[0x55a3fe71e4e0] (ens3): remove_pending_action (2): 'activation::0x55a3fe6f34f0'
Aug 05 00:07:48 localhost NetworkManager[1573]: <debug> [1470355668.2003] dns-mgr: (device_state_changed): queueing DNS updates (1)
Aug 05 00:07:48 localhost NetworkManager[1573]: <debug> [1470355668.2006] dns-mgr: (update_routing_and_dns): queueing DNS updates (2)
Aug 05 00:07:48 localhost NetworkManager[1573]: <info>  [1470355668.2007] manager: NetworkManager state is now CONNECTED_GLOBAL
Aug 05 00:07:48 localhost NetworkManager[1573]: <info>  [1470355668.2009] policy: set 'Wired connection 1' (ens3) as default for IPv4 routing and DNS
Aug 05 00:07:48 localhost NetworkManager[1573]: <debug> [1470355668.2010] manager: PrimaryConnection now Wired connection 1
Aug 05 00:07:48 localhost NetworkManager[1573]: <debug> [1470355668.2014] dns-mgr: (update_routing_and_dns): DNS configuration changed
Aug 05 00:07:48 localhost NetworkManager[1573]: <debug> [1470355668.2026] dns-mgr: (update_routing_and_dns): no DNS changes to commit (1)
Aug 05 00:07:48 localhost NetworkManager[1573]: <debug> [1470355668.2027] dns-mgr: (device_state_changed): DNS configuration changed
Aug 05 00:07:48 localhost NetworkManager[1573]: <debug> [1470355668.2027] dns-mgr: (device_state_changed): committing DNS changes (0)
Aug 05 00:07:48 localhost NetworkManager[1573]: <debug> [1470355668.2027] dns-mgr: update-dns: updating resolv.conf
Aug 05 00:07:48 localhost NetworkManager[1573]: <debug> [1470355668.2033] policy: restarting reverse-lookup thread for address 192.168.122.176
Aug 05 00:07:48 localhost NetworkManager[1573]: <info>  [1470355668.2034] device (ens3): Activation: successful, device activated.

but in the 20160715 case, only this:

Aug 05 00:13:23 localhost NetworkManager[1597]: <info>  [1470356003.3241] manager: NetworkManager state is now CONNECTED_LOCAL
Aug 05 00:13:23 localhost NetworkManager[1597]: <debug> [1470356003.3246] active-connection[0x55fbc96974f0]: check-master-ready: not signalling (state activated, no master)
Aug 05 00:13:23 localhost NetworkManager[1597]: <debug> [1470356003.3247] device[0x55fbc96c25f0] (ens3): remove_pending_action (2): 'activation::0x55fbc96974f0'
Aug 05 00:13:23 localhost NetworkManager[1597]: <debug> [1470356003.3248] dns-mgr: (device_state_changed): queueing DNS updates (1)
Aug 05 00:13:23 localhost NetworkManager[1597]: <debug> [1470356003.3249] dns-mgr: (update_routing_and_dns): queueing DNS updates (2)
Aug 05 00:13:23 localhost NetworkManager[1597]: <debug> [1470356003.3249] dns-mgr: (update_routing_and_dns): DNS configuration did not change
Aug 05 00:13:23 localhost NetworkManager[1597]: <debug> [1470356003.3249] dns-mgr: (update_routing_and_dns): no DNS changes to commit (1)
Aug 05 00:13:23 localhost NetworkManager[1597]: <debug> [1470356003.3249] dns-mgr: (device_state_changed): DNS configuration did not change
Aug 05 00:13:23 localhost NetworkManager[1597]: <debug> [1470356003.3249] dns-mgr: (device_state_changed): no DNS changes to commit (0)
Aug 05 00:13:23 localhost NetworkManager[1597]: <info>  [1470356003.3250] device (ens3): Activation: successful, device activated.

Comment 3 Adam Williamson (Red Hat non-Fedora) 2016-08-05 00:25:30 UTC
Created attachment 1187682 [details]
journal from boot of Fedora-Server-dvd-x86_64-Rawhide-20160715.n.0.iso on KVM with libvirt NAT networking

Comment 4 Adam Williamson (Red Hat non-Fedora) 2016-08-05 00:42:04 UTC
Created attachment 1187696 [details]
`nmcli conn show ens3` output from boot of Fedora-Server-dvd-x86_64-Rawhide-20160708.n.0.iso on KVM with libvirt NAT networking

Comment 5 Adam Williamson (Red Hat non-Fedora) 2016-08-05 00:44:04 UTC
Created attachment 1187697 [details]
`nmcli conn show ens3` output from boot of Fedora-Server-dvd-x86_64-Rawhide-20160715.n.0.iso on KVM with libvirt NAT networking

Here's the output from `nmcli conn show ens3` for both images. Note the difference in GENERAL.DEFAULT as well as all the DHCP4.OPTION values.

Also for the record just running 'dhclient ens3' brings up the network on the affected image. I've not yet figured out what nmcli commands or NetworkManager service changes will cause the connection to come up properly, though.

Comment 6 Adam Williamson (Red Hat non-Fedora) 2016-08-05 15:09:47 UTC

*** This bug has been marked as a duplicate of bug 1358416 ***


Note You need to log in before you can comment on or make changes to this bug.