Bug 1048046 - NetLink errors during IPv6 initialization
Summary: NetLink errors during IPv6 initialization
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: NetworkManager
Version: 20
Hardware: All
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Dan Williams
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1057441 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-01-03 01:47 UTC by David Strauss
Modified: 2014-01-30 09:01 UTC (History)
8 users (show)

Fixed In Version: NetworkManager-0.9.9.0-27.git20131003.fc20
Clone Of:
Environment:
Last Closed: 2014-01-30 03:41:23 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description David Strauss 2014-01-03 01:47:29 UTC
Description of problem:
IPv6 networks, on initialization, are returning errors. The connection seems to work properly, though, including routing and resolv.conf for IPv6.

Jan 02 17:42:28 titan.office.getpantheon.com NetworkManager[777]: :: inet6 dev 2 scope nowhere
Jan 02 17:42:28 titan.office.getpantheon.com NetworkManager[777]: valid-lifetime forever preferred-lifetime forever
Jan 02 17:42:28 titan.office.getpantheon.com NetworkManager[777]: created boot-time+��Xq� last-updated boot-time+��Xq�
Jan 02 17:42:32 titan.office.getpantheon.com dbus-daemon[672]: (geoclue:1573): Geoclue-WARNING **: Error fetching location from geoip server: SSL handshake failed
Jan 02 17:42:35 titan.office.getpantheon.com dbus-daemon[672]: (geoclue:1573): Geoclue-WARNING **: Error fetching location from geoip server: SSL handshake failed
Jan 02 17:42:46 titan.office.getpantheon.com NetworkManager[777]: <info> Activation (wlp3s0) Stage 5 of 5 (IPv6 Commit) scheduled...
Jan 02 17:42:46 titan.office.getpantheon.com NetworkManager[777]: <info> Activation (wlp3s0) Stage 5 of 5 (IPv6 Commit) started...
Jan 02 17:42:46 titan.office.getpantheon.com NetworkManager[777]: <error> [1388713366.79148] [platform/nm-linux-platform.c:1127] add_object(): Netlink error: Invalid address for specified address family
Jan 02 17:42:46 titan.office.getpantheon.com NetworkManager[777]: <info> Activation (wlp3s0) Stage 5 of 5 (IPv6 Commit) complete.

Here's the NetworkManager version:
Name        : NetworkManager
Arch        : x86_64
Epoch       : 1
Version     : 0.9.9.0
Release     : 22.git20131003.fc20

How reproducible:
Happens every time I join my office network, which uses IPv6 RA and stateless DHCPv6.

Comment 1 Shivaram Lingamneni 2014-01-09 06:03:25 UTC
I'm affected by this as well. My systemd journal is filling up with these messages, and the NetworkManager process also seems to be consuming a lot of CPU time (although not as much as in #1029213).

Jan 08 22:02:43 good-fortune.localdomain NetworkManager[400]: <info> Activation (p33p1) Stage 5 of 5 (IPv6 Commit) scheduled...
Jan 08 22:02:43 good-fortune.localdomain NetworkManager[400]: <info> Activation (p33p1) Stage 5 of 5 (IPv6 Commit) started...
Jan 08 22:02:43 good-fortune.localdomain NetworkManager[400]: <error> [1389247363.478787] [platform/nm-linux-platform.c:1127] add_object(): Netlink error: Invalid address for specified address family
Jan 08 22:02:43 good-fortune.localdomain NetworkManager[400]: :: inet6 dev 2 scope nowhere
Jan 08 22:02:43 good-fortune.localdomain NetworkManager[400]: valid-lifetime forever preferred-lifetime forever
Jan 08 22:02:43 good-fortune.localdomain NetworkManager[400]: [54B blob data]
Jan 08 22:02:43 good-fortune.localdomain NetworkManager[400]: <info> Activation (p33p1) Stage 5 of 5 (IPv6 Commit) complete.

Comment 2 Shivaram Lingamneni 2014-01-14 21:53:14 UTC
I also get an out-of-memory error sometimes:

Jan 14 13:52:53 good-fortune.localdomain NetworkManager[398]: <info> Activation (p33p1) Stage 5 of 5 (IPv6 Commit) scheduled...
Jan 14 13:52:53 good-fortune.localdomain NetworkManager[398]: <info> Activation (p33p1) Stage 5 of 5 (IPv6 Commit) started...
Jan 14 13:52:53 good-fortune.localdomain NetworkManager[398]: <info> Activation (p33p1) Stage 5 of 5 (IPv6 Commit) complete.
Jan 14 13:52:53 good-fortune.localdomain NetworkManager[398]: <error> [1389736373.500444] [platform/nm-linux-platform.c:2518] event_handler(): Failed to retrieve incoming events: Out of memory

Comment 3 Will Tisdale 2014-01-17 07:44:56 UTC
I hate to be the 'me too' guy, but I'm also seeing this. 

In fact, I've seen this issue for quite a while now on Fedora 20, since the 10th September 2013 according to journalctl, which means it *may* have been introduced in NetworkManager-0.9.9.0-14.git20131003.fc20, but I can't be sure.

I also used to find that my IPv6 connectivity would fail after an arbitrary amount of time, requiring a NetworkManager restart to regain connectivity. That particular bug was fixed with NetworkManager-0.9.9.0-22.git20131003.fc20, but the messages have persisted since, so that is clearly a further issue which needs resolving.

Comment 4 Dan Williams 2014-01-23 18:24:59 UTC
(In reply to David Strauss from comment #0)
> Description of problem:
> IPv6 networks, on initialization, are returning errors. The connection seems
> to work properly, though, including routing and resolv.conf for IPv6.
> 
> Jan 02 17:42:28 titan.office.getpantheon.com NetworkManager[777]: :: inet6
> dev 2 scope nowhere
> Jan 02 17:42:28 titan.office.getpantheon.com NetworkManager[777]:
> valid-lifetime forever preferred-lifetime forever
> Jan 02 17:42:28 titan.office.getpantheon.com NetworkManager[777]: created
> boot-time+��Xq� last-updated boot-time+��Xq�
> Jan 02 17:42:32 titan.office.getpantheon.com dbus-daemon[672]:
> (geoclue:1573): Geoclue-WARNING **: Error fetching location from geoip
> server: SSL handshake failed
> Jan 02 17:42:35 titan.office.getpantheon.com dbus-daemon[672]:
> (geoclue:1573): Geoclue-WARNING **: Error fetching location from geoip
> server: SSL handshake failed
> Jan 02 17:42:46 titan.office.getpantheon.com NetworkManager[777]: <info>
> Activation (wlp3s0) Stage 5 of 5 (IPv6 Commit) scheduled...
> Jan 02 17:42:46 titan.office.getpantheon.com NetworkManager[777]: <info>
> Activation (wlp3s0) Stage 5 of 5 (IPv6 Commit) started...
> Jan 02 17:42:46 titan.office.getpantheon.com NetworkManager[777]: <error>
> [1388713366.79148] [platform/nm-linux-platform.c:1127] add_object(): Netlink
> error: Invalid address for specified address family
> Jan 02 17:42:46 titan.office.getpantheon.com NetworkManager[777]: <info>
> Activation (wlp3s0) Stage 5 of 5 (IPv6 Commit) complete.
> 
> Here's the NetworkManager version:
> Name        : NetworkManager
> Arch        : x86_64
> Epoch       : 1
> Version     : 0.9.9.0
> Release     : 22.git20131003.fc20
> 
> How reproducible:
> Happens every time I join my office network, which uses IPv6 RA and
> stateless DHCPv6.

I have identified the source of this problem and fixed it.  However, this problem should not be consuming 100% CPU as the comment #2 indicates.  Is NM consuming too much CPU for you or not?

Comment 5 Dan Williams 2014-01-23 18:26:11 UTC
For those of you with the 100% CPU problem, would you please:

sudo nmcli g logging level debug domains core,hw,ip6,dhcp6

and then report the log output of NetworkManager.  Thanks!

Comment 6 David Strauss 2014-01-24 00:21:20 UTC
I've never seen any 100% CPU issue with this.

Comment 7 Thomas Haller 2014-01-24 07:22:05 UTC
For the record, this is the upstream commit by Dan that fixes this bug:

http://cgit.freedesktop.org/NetworkManager/NetworkManager/commit/?id=a9f1d121783a7f3c65c9f47d3419b5a8a255fe47

Comment 8 Dan Williams 2014-01-24 19:39:57 UTC
*** Bug 1057441 has been marked as a duplicate of this bug. ***

Comment 9 Ed Greshko 2014-01-28 13:38:57 UTC
Any idea when this fix will hit updates-testing?

Comment 10 Fedora Update System 2014-01-28 15:23:23 UTC
NetworkManager-0.9.9.0-27.git20131003.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/NetworkManager-0.9.9.0-27.git20131003.fc20

Comment 11 Shivaram Lingamneni 2014-01-28 22:31:49 UTC
Is there a way I can suppress the INFO-level lines as well?

Jan 28 14:19:03 good-fortune.localdomain NetworkManager[397]: <info> Activation (p33p1) Stage 5 of 5 (IPv6 Commit) complete.
Jan 28 14:19:12 good-fortune.localdomain NetworkManager[397]: <info> Activation (p33p1) Stage 5 of 5 (IPv6 Commit) scheduled...
Jan 28 14:19:12 good-fortune.localdomain NetworkManager[397]: <info> Activation (p33p1) Stage 5 of 5 (IPv6 Commit) started...

All my log levels are currently set to WARN:

# nmcli g log
LEVEL  DOMAINS
WARN   PLATFORM,RFKILL,ETHER,WIFI,BT,MB,DHCP4,DHCP6,PPP,WIFI_SCAN,IP4,IP6,AUTOIP4,DNS,VPN,SHARING,SUPPLICANT,AGENTS,SETTINGS,SUSPEND,CORE,DEVICE,OLPC,WIMAX,INFINIBAND,FIREWALL,ADSL,BOND,VLAN,BRIDGE,DBUS_PROPS,TEAM,CONCHECK

Comment 12 Ed Greshko 2014-01-28 23:49:39 UTC
NetworkManager-0.9.9.0-27.git20131003.fc20 fixes it for me.  karma given

Comment 13 Fedora Update System 2014-01-29 03:13:03 UTC
Package NetworkManager-0.9.9.0-27.git20131003.fc20:
* should fix your issue,
* was pushed to the Fedora 20 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing NetworkManager-0.9.9.0-27.git20131003.fc20'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2014-1733/NetworkManager-0.9.9.0-27.git20131003.fc20
then log in and leave karma (feedback).

Comment 14 Thomas Haller 2014-01-29 09:18:29 UTC
(In reply to Shivaram Lingamneni from comment #11)
> Is there a way I can suppress the INFO-level lines as well?

This too frequent logging is unrelated to this bug, it would be bug 1044757.
NetworkManager-0.9.9.0-27.git20131003.fc20 fixes that as well.

Comment 15 Shivaram Lingamneni 2014-01-29 11:36:49 UTC
Thanks for your help. I installed NetworkManager-1:0.9.9.0-26.git20131003.fc20 and it fixed the "Netlink error" lines (and also the excess logging described in #1044757).

Unfortunately, I still have the "Out of memory" lines:

Jan 29 03:32:57 good-fortune.localdomain NetworkManager[20674]: <error> [1390995177.812465] [platform/nm-linux-platform.c:2538] event_handler(): Failed to retrieve incoming events: Out of memory
Jan 29 03:33:01 good-fortune.localdomain NetworkManager[20674]: <error> [1390995181.431847] [platform/nm-linux-platform.c:2538] event_handler(): Failed to retrieve incoming events: Out of memory
Jan 29 03:33:05 good-fortune.localdomain NetworkManager[20674]: <error> [1390995185.388601] [platform/nm-linux-platform.c:2538] event_handler(): Failed to retrieve incoming events: Out of memory
Jan 29 03:33:14 good-fortune.localdomain NetworkManager[20674]: <error> [1390995194.131286] [platform/nm-linux-platform.c:2538] event_handler(): Failed to retrieve incoming events: Out of memory

Comment 16 Ed Greshko 2014-01-29 12:29:02 UTC
Thanks for pointing that out.  I too am still getting those "Out of memory" log entries.

Comment 17 Thomas Haller 2014-01-29 12:47:37 UTC
(In reply to Shivaram Lingamneni from comment #15)
> Thanks for your help. I installed
> NetworkManager-1:0.9.9.0-26.git20131003.fc20 and it fixed the "Netlink
> error" lines (and also the excess logging described in #1044757).

All of these issues are fixed in 0.9.9.0-27 and later, not 0.9.9.0-26.


> Unfortunately, I still have the "Out of memory" lines:
> 
> Jan 29 03:32:57 good-fortune.localdomain NetworkManager[20674]: <error>
> [1390995177.812465] [platform/nm-linux-platform.c:2538] event_handler():
> Failed to retrieve incoming events: Out of memory
> Jan 29 03:33:01 good-fortune.localdomain NetworkManager[20674]: <error>
> [1390995181.431847] [platform/nm-linux-platform.c:2538] event_handler():
> Failed to retrieve incoming events: Out of memory
> Jan 29 03:33:05 good-fortune.localdomain NetworkManager[20674]: <error>
> [1390995185.388601] [platform/nm-linux-platform.c:2538] event_handler():
> Failed to retrieve incoming events: Out of memory
> Jan 29 03:33:14 good-fortune.localdomain NetworkManager[20674]: <error>
> [1390995194.131286] [platform/nm-linux-platform.c:2538] event_handler():
> Failed to retrieve incoming events: Out of memory

These "Out of memory" lines are most likely not related to this bug. AFAIS, it's a different (open) issue. Could you please file a new bug and provide more details?

Comment 18 Shivaram Lingamneni 2014-01-29 13:02:33 UTC
Sorry, I copied the wrong version from my yum history. I installed 0.9.9.0-27.git20131003.fc20 from updates-testing and it fixed all the issues except the "Out of memory" lines, for which I will open a new bug.

Comment 19 Ed Greshko 2014-01-29 13:03:57 UTC
Please mention the bugzilla # here, or add me to the CC:, so I can follow.  Thanks.

Comment 20 Fedora Update System 2014-01-30 03:41:23 UTC
NetworkManager-0.9.9.0-27.git20131003.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 21 Shivaram Lingamneni 2014-01-30 09:01:30 UTC
Follow-up bug for the "out of memory" loglines: https://bugzilla.redhat.com/show_bug.cgi?id=1059597

Thanks everyone for your time and assistance!


Note You need to log in before you can comment on or make changes to this bug.