Bug 1063800

Summary: After the recent updates on F20 NM does not start br0 on boot.
Product: [Fedora] Fedora Reporter: Cristian Sava <cristis53>
Component: NetworkManagerAssignee: Dan Williams <dcbw>
Status: CLOSED DUPLICATE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 20CC: cristis53, dcbw, dinesy91+rhbug, jklimes, thaller
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: NetworkManager-0.9.9.0-30.git20131003.fc20 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-02-12 12:05:24 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
The relevant part from /var/log/messages for br0 not starting. none

Description Cristian Sava 2014-02-11 13:08:26 UTC 
Created attachment 861781 [details]
The relevant part from /var/log/messages for br0 not starting.

Description of problem:
My servers & desktops were working ok with F19 and F20 (new or
upgrade).
After the recent updates on F20 NM does not start br0 on boot.
I succeed to connect to the net after manually restarting NM.
(I know I can restart NM in rc.local but ...)
I did not tested it on F19 with latest updates.
https://lists.fedoraproject.org/pipermail/users/2014-February/446703.html

Version-Release number of selected component (if applicable):
NetworkManager-0.9.9.0-28.git20131003.fc20.x86_64


How reproducible:
Allways

Steps to Reproduce:
1.
After boot/reboot:
[root@localhost ~]# route
Kernel IP routing table
Destination    Gateway   Genmask         Flags Metric Ref Use Iface
192.168.122.0  0.0.0.0   255.255.255.0   U     0      0   0   virbr0

2.
After "sysctl restart NetworkManager.service"
[root@localhost ~]# route
Kernel IP routing table
Destination   Gateway         Genmask         Flags Metric Ref Use Iface
default       fw.central.ucv. 0.0.0.0         UG    1024   0     0 br0
192.168.122.0 0.0.0.0         255.255.255.0   U     0      0     0
virbr0
193.231.x.r   0.0.0.0         255.255.255.240 U     0      0     0 br0

3.

Actual results:
The bridged interface does not start at boot/reboot but start after restarting NM.

Expected results:
The bridged interfaces have to start at boot, they were working ok before the latest updates.

Additional info:
After boot/reboot:
[root@localhost ~]# route
Kernel IP routing table
Destination    Gateway   Genmask         Flags Metric Ref Use Iface
192.168.122.0  0.0.0.0   255.255.255.0   U     0      0   0   virbr0
[root@localhost ~]# ls /etc/sysconfig/network-scripts
ifcfg-br0
ifcfg-em1
ifcfg-lo
ifcfg-p3p1
...
[root@localhost ~]# cat /etc/sysconfig/network-scripts/ifcfg-em1
TYPE="Ethernet"
NAME="em1"
DEVICE="em1"
ONBOOT="yes"
UUID="093691ed-5ec9-4417-b7f2-8fa7d4f663fc"
HWADDR="00:25:90:87:16:72"
BRIDGE="br0"
[root@localhost ~]# cat /etc/sysconfig/network-scripts/ifcfg-br0
TYPE="Bridge"
NAME="Bridge"
DEVICE="br0"
BOOTPROTO="none"
ONBOOT="yes"
DEFROUTE="yes"
DOMAIN="central.ucv.ro"
IPADDR0="193.231.x.y"
PREFIX0="28"
GATEWAY0="193.231.x.z"
DNS1="193.231.x.w"
IPV4_FAILURE_FATAL=yes
IPV6INIT="no"
PEERNTP="yes"
STP=no
UUID=d2d68553-f97e-7549-7a26-b34a26f29318
[root@localhost ~]# cat /etc/sysconfig/network-scripts/ifcfg-p3p1
TYPE=Ethernet
BOOTPROTO=dhcp
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_FAILURE_FATAL=no
NAME=p3p1
UUID=a6f68419-9c54-4a58-b3fb-2e424eb12d39
ONBOOT=no
HWADDR=00:25:90:87:16:73
PEERDNS=yes
PEERROUTES=yes
[root@localhost ~]# systemctl status NetworkManager.service
NetworkManager.service - Network Manager
   Loaded: loaded (/usr/lib/systemd/system/NetworkManager.service;
enabled)
   Active: active (running) since Tue 2014-02-11 10:16:15 EET; 6min ago
 Main PID: 525 (NetworkManager)
   CGroup: /system.slice/NetworkManager.service
           └─525 /usr/sbin/NetworkManager --no-daemon
Feb 11 10:16:17 localhost.localdomain NetworkManager[525]: <info>
(vnet0): link connected
Feb 11 10:16:17 localhost.localdomain NetworkManager[525]: <error>
[1392106577.228161] [platform/nm-linux-platform.c:1252]
event_notification(): netlink cache error: Object exists
Feb 11 10:16:17 localhost.localdomain NetworkManager[525]: <error>
[1392106577.228435] [platform/nm-linux-platform.c:1252]
event_notification(): netlink cache error: Object exists
Feb 11 10:16:17 localhost.localdomain NetworkManager[525]: <error>
[1392106577.921547] [platform/nm-linux-platform.c:1252]
event_notification(): netlink cache error: Object exists
Feb 11 10:16:19 localhost.localdomain NetworkManager[525]: <info> (em1):
link connected
Feb 11 10:16:19 localhost.localdomain NetworkManager[525]: <info> (em1):
device state change: unavailable -> disconnected (reason
'carrier-changed') [20 30 40]
Feb 11 10:16:19 localhost.localdomain NetworkManager[525]: <info>
Auto-activating connection 'em1'.
Feb 11 10:16:19 localhost.localdomain NetworkManager[525]: <info>
Connection 'em1' auto-activation failed: (2) Master device br0 unmanaged
or not available for activation
Feb 11 10:16:21 localhost.localdomain NetworkManager[525]: <info>
startup complete
Feb 11 10:16:26 localhost.localdomain NetworkManager[525]: <error>
[1392106586.339564] [platform/nm-linux-platform.c:1252]
event_notification(): netlink cache error: Object exists
[root@localhost ~]# systemctl restart NetworkManager.service
[root@localhost ~]# systemctl status NetworkManager.service
NetworkManager.service - Network Manager
   Loaded: loaded (/usr/lib/systemd/system/NetworkManager.service;
enabled)
   Active: active (running) since Tue 2014-02-11 10:23:11 EET; 9s ago
 Main PID: 896 (NetworkManager)
   CGroup: /system.slice/NetworkManager.service
           └─896 /usr/sbin/NetworkManager --no-daemon
Feb 11 10:23:15 localhost.localdomain NetworkManager[896]: <info> (br0):
attached bridge port em1
Feb 11 10:23:15 localhost.localdomain NetworkManager[896]: <info>
Activation (em1) connection 'em1' enslaved, continuing activation
Feb 11 10:23:15 localhost.localdomain NetworkManager[896]: <info>
Activation (em1) Stage 3 of 5 (IP Configure Start) complete.
Feb 11 10:23:15 localhost.localdomain NetworkManager[896]: <error>
[1392106995.13071] [platform/nm-linux-platform.c:1252]
event_notification(): netlink cache error: Object exists
Feb 11 10:23:15 localhost.localdomain NetworkManager[896]: <error>
[1392106995.13359] [platform/nm-linux-platform.c:1252]
event_notification(): netlink cache error: Object exists
Feb 11 10:23:15 localhost.localdomain NetworkManager[896]: <info> (em1):
device state change: ip-config -> secondaries (reason 'none') [70 90 0]
Feb 11 10:23:15 localhost.localdomain NetworkManager[896]: <info> (em1):
device state change: secondaries -> activated (reason 'none') [90 100 0]
Feb 11 10:23:15 localhost.localdomain NetworkManager[896]: <info>
Activation (em1) successful, device activated.
Feb 11 10:23:15 localhost.localdomain NetworkManager[896]: <warn>
Dispatcher script timed out: Script
'/etc/NetworkManager/dispatcher.d/20-chrony' timed out.
Feb 11 10:23:17 localhost.localdomain NetworkManager[896]: <info>
startup complete
[root@localhost ~]# route
Kernel IP routing table
Destination   Gateway         Genmask         Flags Metric Ref Use Iface
default       fw.central.ucv. 0.0.0.0         UG    1024   0     0 br0
192.168.122.0 0.0.0.0         255.255.255.0   U     0      0     0
virbr0
193.231.x.r   0.0.0.0         255.255.255.240 U     0      0     0 br0
Comment 1 Jirka Klimes 2014-02-11 15:56:38 UTC 
It might be the same as bug 1063545. Does it help to downgrade NetworkManager?
What libnl3 version do you have? It may influence the behaviour.
Comment 2 poma 2014-02-12 03:22:35 UTC 
Maybe we should consider file a bug report upstream in addition.
https://bugzilla.gnome.org/
And "NetworkManager discussions" does not hurt either. :)
https://mail.gnome.org/mailman/listinfo/networkmanager-list
What do you think about that?

e.g.
https://bugzilla.gnome.org/buglist.cgi?bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=NEEDINFO&content=bridge&product=NetworkManager&query_format=specific&order=bug_id%20DESC&query_based_on=
Comment 3 Cristian Sava 2014-02-12 07:37:15 UTC 
(In reply to Jirka Klimes from comment #1)
> It might be the same as bug 1063545. Does it help to downgrade
> NetworkManager?
> What libnl3 version do you have? It may influence the behaviour.

libnl3-3.2.24-1.fc20.x86_64
Cause of dependencies:
yum downgrade NetworkManager NetworkManager-glib nm-connection-editor libnm-gtk

Same as before.
After downgrade I had to manually restart NM to start bridgeing.
Comment 4 Cristian Sava 2014-02-12 11:32:58 UTC 
(In reply to poma from comment #2)
> Maybe we should consider file a bug report upstream in addition.
> https://bugzilla.gnome.org/
> And "NetworkManager discussions" does not hurt either. :)
> https://mail.gnome.org/mailman/listinfo/networkmanager-list
> What do you think about that?
> 
> e.g.
> https://bugzilla.gnome.org/buglist.
> cgi?bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOP
> ENED&bug_status=NEEDINFO&content=bridge&product=NetworkManager&query_format=s
> pecific&order=bug_id%20DESC&query_based_on=

Yes, good idea.
https://bugzilla.gnome.org/show_bug.cgi?id=724225
Comment 5 Thomas Haller 2014-02-12 12:05:24 UTC 
This is (most likely) a duplicate of bug 1063290.

(supposing the analysis is correct...) the bug is already present in older NM version, but it does hit preferably with the new libnl3-3.2.24-1 version. So, as a mitigation you can downgrade libnl3.


(it is *not* a duplicate of bug 1063545 -- which itself is a clone of bug 1063290 but for another issue)

*** This bug has been marked as a duplicate of bug 1063290 ***