Bug 1211859

Summary:	_nl_get_vtable: assertion 'vtable.handle' failed
Product:	Red Hat Enterprise Linux 7	Reporter:	Qunfang Zhang <qzhang>
Component:	NetworkManager	Assignee:	Jirka Klimes <jklimes>
Status:	CLOSED ERRATA	QA Contact:	Desktop QE <desktop-qa-list>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	7.1	CC:	coli, dcbw, extras-qa, fedora, jklimes, jstodola, lrintel, msugaya, psimerda, rkhan, thaller, vbenes, wolf, xutian
Target Milestone:	rc
Target Release:	---
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:	Cause: NetworkManager loaded the libnm from a wrong location. Consequence: NetworkManager would fail to start if libnl3-devel package was installed. Fix: NetworkManager now uses the correct library location. Result: NetworkManager no longer crashes if libnl3-devel package is not installed.	Story Points:	---
Clone Of:	1205195	Environment:
Last Closed:	2015-11-19 11:01:40 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1205195
Bug Blocks:

Description Qunfang Zhang 2015-04-15 06:23:42 UTC

Also hit this bug for rhel7.1-z kernel:
kernel-3.10.0-229.2.1.el7


+++ This bug was initially created as a clone of Bug #1205195 +++

Description of problem:
NetworkManager displays following error when starting:

_nl_get_vtable: assertion 'vtable.handle' failed


[root@localhost ~]# systemctl -n 200 -l status NetworkManager
● NetworkManager.service - Network Manager
   Loaded: loaded (/usr/lib/systemd/system/NetworkManager.service; enabled; vendor preset: enabled)
   Active: active (running) since Tue 2015-03-24 12:25:34 CET; 47min ago
 Main PID: 559 (NetworkManager)
   CGroup: /system.slice/NetworkManager.service
           ├─559 /usr/sbin/NetworkManager --no-daemon
           └─882 /sbin/dhclient -d -q -sf /usr/libexec/nm-dhcp-helper -pf /var/run/dhclient-ens3.pid -lf /var/lib/NetworkManager/dhclient-8cfa7efd-e8a8-4020-9fe2-2b6896dd208c-ens3.lease -cf /var/lib/NetworkManager/dhclient-ens3.conf ens3

Mar 24 12:25:34 localhost.localdomain systemd[1]: Starting Network Manager...
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  NetworkManager (version 1.0.0-8.fc22) is starting...
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  Read config: /etc/NetworkManager/NetworkManager.conf
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  WEXT support is enabled
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  Acquired D-Bus service com.redhat.ifcfgrh1
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  Loaded plugin ifcfg-rh: (c) 2007 - 2013 Red Hat, Inc.  To report bugs please use the NetworkManager mailing list.
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  Loaded plugin keyfile: (c) 2007 - 2013 Red Hat, Inc.  To report bugs please use the NetworkManager mailing list.
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  parsing /etc/sysconfig/network-scripts/ifcfg-lo ...
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  parsing /etc/sysconfig/network-scripts/ifcfg-ens3 ...
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>      read connection 'ens3'
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  monitoring kernel firmware directory '/lib/firmware'.
Mar 24 12:25:34 localhost.localdomain systemd[1]: Started Network Manager.
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  WiFi enabled by radio killswitch; enabled by state file
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  WWAN enabled by radio killswitch; enabled by state file
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  WiMAX enabled by radio killswitch; enabled by state file
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  Networking is enabled by state file
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  (lo): link connected
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  (lo): carrier is ON
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  (lo): new Generic device (driver: 'unknown' ifindex: 1)
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  (lo): exported as /org/freedesktop/NetworkManager/Devices/0
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  (ens3): carrier is OFF
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  (ens3): new Ethernet device (driver: '8139cp' ifindex: 2)
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  (ens3): exported as /org/freedesktop/NetworkManager/Devices/1
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  (ens3): device state change: unmanaged -> unavailable (reason 'managed') [10 20 2]
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  (ens3): link connected
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  (ens3): preparing device
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  (ens3): device state change: unavailable -> disconnected (reason 'none') [20 30 0]
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  Auto-activating connection 'ens3'.
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  (ens3): Activation: starting connection 'ens3'
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  (ens3): Activation: Stage 1 of 5 (Device Prepare) scheduled...
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  (ens3): Activation: Stage 1 of 5 (Device Prepare) started...
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  (ens3): device state change: disconnected -> prepare (reason 'none') [30 40 0]
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  NetworkManager state is now CONNECTING
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  (ens3): Activation: Stage 2 of 5 (Device Configure) scheduled...
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  (ens3): Activation: Stage 1 of 5 (Device Prepare) complete.
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  (ens3): Activation: Stage 2 of 5 (Device Configure) starting...
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  (ens3): device state change: prepare -> config (reason 'none') [40 50 0]
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  (ens3): Activation: Stage 2 of 5 (Device Configure) successful.
Mar 24 12:25:34 localhost.localdomain NetworkManager[559]: <info>  (ens3): Activation: Stage 2 of 5 (Device Configure) complete.
Mar 24 12:25:35 localhost.localdomain NetworkManager[559]: <info>  (ens3): Activation: Stage 3 of 5 (IP Configure Start) scheduled.
Mar 24 12:25:35 localhost.localdomain NetworkManager[559]: <info>  (ens3): Activation: Stage 3 of 5 (IP Configure Start) started...
Mar 24 12:25:35 localhost.localdomain NetworkManager[559]: <info>  (ens3): device state change: config -> ip-config (reason 'none') [50 70 0]
Mar 24 12:25:35 localhost.localdomain NetworkManager[559]: <info>  Activation (ens3) Beginning DHCPv4 transaction (timeout in 45 seconds)
Mar 24 12:25:35 localhost.localdomain NetworkManager[559]: <info>  dhclient started with pid 882
Mar 24 12:25:35 localhost.localdomain NetworkManager[559]: <info>  (ens3): Activation: Stage 3 of 5 (IP Configure Start) complete.
Mar 24 12:25:35 localhost.localdomain dhclient[882]: DHCPREQUEST on ens3 to 255.255.255.255 port 67 (xid=0x1a292852)
Mar 24 12:25:35 localhost.localdomain dhclient[882]: DHCPACK from 192.168.122.1 (xid=0x1a292852)
Mar 24 12:25:35 localhost.localdomain NetworkManager[559]: <info>    address 192.168.122.201
Mar 24 12:25:35 localhost.localdomain NetworkManager[559]: <info>    plen 24 (255.255.255.0)
Mar 24 12:25:35 localhost.localdomain NetworkManager[559]: <info>    gateway 192.168.122.1
Mar 24 12:25:35 localhost.localdomain NetworkManager[559]: <info>    server identifier 192.168.122.1
Mar 24 12:25:35 localhost.localdomain NetworkManager[559]: <info>    lease time 3600
Mar 24 12:25:35 localhost.localdomain NetworkManager[559]: <info>    nameserver '192.168.122.1'
Mar 24 12:25:35 localhost.localdomain NetworkManager[559]: <info>  (ens3): DHCPv4 state changed unknown -> bound
Mar 24 12:25:35 localhost.localdomain NetworkManager[559]: <info>  (ens3): Activation: Stage 5 of 5 (IPv4 Configure Commit) scheduled...
Mar 24 12:25:35 localhost.localdomain NetworkManager[559]: <info>  (ens3): Activation: Stage 5 of 5 (IPv4 Commit) started...
Mar 24 12:25:35 localhost.localdomain NetworkManager[559]: _nl_get_vtable: assertion 'vtable.handle' failed
Mar 24 12:25:35 localhost.localdomain NetworkManager[559]: <info>  (ens3): device state change: ip-config -> ip-check (reason 'none') [70 80 0]
Mar 24 12:25:35 localhost.localdomain NetworkManager[559]: <info>  (ens3): Activation: Stage 5 of 5 (IPv4 Commit) complete.
Mar 24 12:25:35 localhost.localdomain NetworkManager[559]: <info>  (ens3): device state change: ip-check -> secondaries (reason 'none') [80 90 0]
Mar 24 12:25:35 localhost.localdomain NetworkManager[559]: <info>  (ens3): device state change: secondaries -> activated (reason 'none') [90 100 0]
Mar 24 12:25:35 localhost.localdomain NetworkManager[559]: <info>  NetworkManager state is now CONNECTED_LOCAL
Mar 24 12:25:35 localhost.localdomain NetworkManager[559]: <info>  NetworkManager state is now CONNECTED_GLOBAL
Mar 24 12:25:35 localhost.localdomain NetworkManager[559]: <info>  Policy set 'ens3' (ens3) as default for IPv4 routing and DNS.
Mar 24 12:25:35 localhost.localdomain NetworkManager[559]: <info>  (ens3): Activation: successful, device activated.
Mar 24 12:25:35 localhost.localdomain dhclient[882]: bound to 192.168.122.201 -- renewal in 1390 seconds.
Mar 24 12:25:40 localhost.localdomain NetworkManager[559]: <info>  startup complete
Mar 24 12:26:07 localhost.localdomain NetworkManager[559]: <info>  (ens3): Activation: Stage 4 of 5 (IPv6 Configure Timeout) scheduled...
Mar 24 12:26:07 localhost.localdomain NetworkManager[559]: <info>  (ens3): Activation: Stage 4 of 5 (IPv6 Configure Timeout) started...
Mar 24 12:26:07 localhost.localdomain NetworkManager[559]: <info>  (ens3): Activation: Stage 4 of 5 (IPv6 Configure Timeout) complete.
Mar 24 12:48:45 localhost.localdomain dhclient[882]: DHCPREQUEST on ens3 to 192.168.122.1 port 67 (xid=0x1a292852)
Mar 24 12:48:45 localhost.localdomain dhclient[882]: DHCPACK from 192.168.122.1 (xid=0x1a292852)
Mar 24 12:48:45 localhost.localdomain NetworkManager[559]: <info>    address 192.168.122.201
Mar 24 12:48:45 localhost.localdomain NetworkManager[559]: <info>    plen 24 (255.255.255.0)
Mar 24 12:48:45 localhost.localdomain NetworkManager[559]: <info>    gateway 192.168.122.1
Mar 24 12:48:45 localhost.localdomain NetworkManager[559]: <info>    server identifier 192.168.122.1
Mar 24 12:48:45 localhost.localdomain NetworkManager[559]: <info>    lease time 3600
Mar 24 12:48:45 localhost.localdomain NetworkManager[559]: <info>    nameserver '192.168.122.1'
Mar 24 12:48:45 localhost.localdomain NetworkManager[559]: <info>  (ens3): DHCPv4 state changed bound -> bound
Mar 24 12:48:45 localhost.localdomain dhclient[882]: bound to 192.168.122.201 -- renewal in 1639 seconds.

Version-Release number of selected component (if applicable):
NetworkManager-1.0.0-8.fc22

How reproducible:
always

Steps to Reproduce:
1. Install F22 and reboot
2. run "journalctl -b" and search for "_nl_get_vtable"

Actual results:
_nl_get_vtable: assertion 'vtable.handle' failed

Expected results:
no error

--- Additional comment from Jirka Klimes on 2015-03-25 10:38:54 EDT ---

libnl-3.so link is only present when libnl3-devel is installed.

--- Additional comment from Dan Williams on 2015-03-27 18:01:16 EDT ---

Looks OK to me, but obviously this will now break when libnl bumps it's major soname.  I guess at that point it's a major ABI break so we'll have to change NM too, but I wonder if there's a way to make sure we don't forget this bit.  Anyway, not a blocker for me right now.

--- Additional comment from Jirka Klimes on 2015-03-30 05:48:07 EDT ---

Committed to upstream master:
d767fb1 platform: load libnl-3.so.200, not libnl-3.so (rh #1205195)

(In reply to Dan Williams from comment #2)
> Looks OK to me, but obviously this will now break when libnl bumps it's
> major soname.  I guess at that point it's a major ABI break so we'll have to
> change NM too, but I wonder if there's a way to make sure we don't forget
> this bit.

libnl doesn't apparently change major soname often. But, of course, it would be nice to have an automatic check. Some configure.ac magic, perhaps?

Comment 2 Thomas Haller 2015-05-12 16:03:25 UTC

Upstream fix is http://cgit.freedesktop.org/NetworkManager/NetworkManager/commit/?id=d767fb160c36bd9dc339e343ebac58274204ad4f

Comment 3 Thomas Haller 2015-05-15 08:09:09 UTC

*** Bug 1221744 has been marked as a duplicate of this bug. ***

Comment 5 Vladimir Benes 2015-09-11 14:32:16 UTC

no errors in journal, link is libnl-3.so.200 is opened instead of libnl-3.so which was just present in libnl3-devel package. Both libnl-3.so and libnl-3.so.200 point to libnl-3.so.200.16.1.

Comment 6 errata-xmlrpc 2015-11-19 11:01:40 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-2315.html