RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1260920 - [abrt] [beaker] [master] Creating pipes for GWakeup: Too many open files
Summary: [abrt] [beaker] [master] Creating pipes for GWakeup: Too many open files
Keywords:
Status: CLOSED NEXTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: NetworkManager
Version: 7.3
Hardware: x86_64
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Beniamino Galvani
QA Contact: Desktop QE
URL:
Whiteboard: abrt_hash:0fd8057911309c9afeee68a75b0...
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-09-08 08:44 UTC by Lubomir Rintel
Modified: 2015-09-30 10:02 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-09-30 10:02:57 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
File: backtrace (29.49 KB, text/plain)
2015-09-08 08:44 UTC, Lubomir Rintel 		no flags Details
File: cgroup (199 bytes, text/plain)
2015-09-08 08:44 UTC, Lubomir Rintel 		no flags Details
File: core_backtrace (10.20 KB, text/plain)
2015-09-08 08:44 UTC, Lubomir Rintel 		no flags Details
File: dso_list (5.81 KB, text/plain)
2015-09-08 08:44 UTC, Lubomir Rintel 		no flags Details
File: environ (72 bytes, text/plain)
2015-09-08 08:44 UTC, Lubomir Rintel 		no flags Details
File: limits (1.29 KB, text/plain)
2015-09-08 08:44 UTC, Lubomir Rintel 		no flags Details
File: machineid (135 bytes, text/plain)
2015-09-08 08:44 UTC, Lubomir Rintel 		no flags Details
File: maps (26.92 KB, text/plain)
2015-09-08 08:44 UTC, Lubomir Rintel 		no flags Details
File: open_fds (54.50 KB, text/plain)
2015-09-08 08:44 UTC, Lubomir Rintel 		no flags Details
File: proc_pid_status (1.04 KB, text/plain)
2015-09-08 08:44 UTC, Lubomir Rintel 		no flags Details
File: var_log_messages (43.81 KB, text/plain)
2015-09-08 08:44 UTC, Lubomir Rintel 		no flags Details
[PATCH] core: fix file descriptor leak in bus manager (1.09 KB, patch)
2015-09-08 14:37 UTC, Beniamino Galvani 		no flags Details | Diff
[PATCH v2] core: fix file descriptor leak in bus manager (1.20 KB, patch)
2015-09-08 16:39 UTC, Beniamino Galvani 		no flags Details | Diff
Show Obsolete (1) View All


Links
System ID Private Priority Status Summary Last Updated
GNOME Bugzilla 754730 0 None None None Never

Description Lubomir Rintel 2015-09-08 08:44:27 UTC 
Description of problem:
Sep 07 16:16:02 myown.hostname NetworkManager[6845]: <debug> [1441656962.986592] [nm-exported-object.c:571] idle_emit_properties_changed(): PropertiesChanged NMSettings 0x7fabf5c4a160: {'Connections': <[objectpath '/org/freedesktop/NetworkManager/Settings/113', '/org/freedesktop/NetworkManager/Settings/114', '/org/freedesktop/NetworkManager/Settings/115', '/org/freedesktop/NetworkManager/Settings/116', '/org/freedesktop/NetworkManager/Settings/117', '/org/freedesktop/NetworkManager/Settings/118', '/org/freedesktop/NetworkManager/Settings/119', '/org/freedesktop/NetworkManager/Settings/120', '/org/freedesktop/NetworkManager/Settings/10', '/org/freedesktop/NetworkManager/Settings/11', '/org/freedesktop/NetworkManager/Settings/12', '/org/freedesktop/NetworkManager/Settings/13', '/org/freedesktop/NetworkManager/Settings/0', '/org/freedesktop/NetworkManager/Settings/1', '/org/freedesktop/NetworkManager/Settings/2', '/org/freedesktop/NetworkManager/Settings/3', '/org/freedesktop/NetworkManager/Settings/4', '/org/freedesktop/NetworkManager/Settings/5', '/org/freedesktop/NetworkManager/Settings/6', '/org/freedesktop/NetworkManager/Settings/7', '/org/freedesktop/NetworkManager/Settings/8', '/org/freedesktop/NetworkManager/Settings/9']>, 'Connections': <[objectpath '/org/freedesktop/NetworkManager/Settings/114', '/org/freedesktop/NetworkManager/Settings/115', '/org/freedesktop/NetworkManager/Settings/116', '/org/freedesktop/NetworkManager/Settings/117', '/org/freedesktop/NetworkManager/Settings/118', '/org/freedesktop/NetworkManager/Settings/119', '/org/freedesktop/NetworkManager/Settings/120', '/org/freedesktop/NetworkManager/Settings/10', '/org/freedesktop/NetworkManager/Settings/11', '/org/freedesktop/NetworkManager/Settings/12', '/org/freedesktop/NetworkManager/Settings/13', '/org/freedesktop/NetworkManager/Settings/0', '/org/freedesktop/NetworkManager/Settings/1', '/org/freedesktop/NetworkManager/Settings/2', '/org/freedesktop/NetworkManager/Settings/3', '/org/freedesktop/NetworkManager/Settings/4', '/org/freedesktop/NetworkManager/Settings/5', '/org/freedesktop/NetworkManager/Settings/6', '/org/freedesktop/NetworkManager/Settings/7', '/org/freedesktop/NetworkManager/Settings/8', '/org/freedesktop/NetworkManager/Settings/9']>}
Sep 07 16:16:02 myown.hostname NetworkManager[6845]: <debug> [1441656962.995517] [nm-exported-object.c:571] idle_emit_properties_changed(): PropertiesChanged NMDeviceEthernet 0x7fabf5d09eb0: {'AvailableConnections': <[objectpath '/org/freedesktop/NetworkManager/Settings/11']>}
Sep 07 16:16:03 myown.hostname NetworkManager[6845]: <debug> [1441656963.004245] [nm-bus-manager.c:126] private_server_closed(): (private) closed connection 0x7fabd413d900 on private socket.
Sep 07 16:16:03 myown.hostname sudo[24983]:     root : TTY=unknown ; PWD=/mnt/tests/NetworkManager ; USER=root ; COMMAND=/sbin/ip link del bridge0
Sep 07 16:16:03 myown.hostname sudo[24989]:     root : TTY=unknown ; PWD=/mnt/tests/NetworkManager ; USER=root ; COMMAND=/bin/nmcli con delete bridge-slave-eth3 bridge-slave-eth4 bridge-slave-eth5 bridge-slave-eth6 bridge-slave-eth7 bridge-slave-eth8 bridge-slave-eth9
Sep 07 16:16:03 myown.hostname NetworkManager[6845]: <debug> [1441656963.060551] [nm-bus-manager.c:153] private_server_new_connection(): (private) accepted connection 0x7fabd413da00 on private socket.
Sep 07 16:16:03 myown.hostname NetworkManager[6845]: <debug> [1441656963.170911] [nm-bus-manager.c:126] private_server_closed(): (private) closed connection 0x7fabd413da00 on private socket.
Sep 07 16:16:04 myown.hostname NetworkManager[6845]: <debug> [1441656964.465308] [platform/nm-linux-platform.c:2344] event_notification(): platform-linux: event-notification: NEWROUTE, seq 0: 2620:52:0:1040:5054:ff:fe38:5099/128 via fe80:52:0:1086::fc dev 2 metric 0 mss 0 src rtm-f-cloned
Sep 07 16:16:05 myown.hostname nmtui/./runtest.sh[25026]: Test nmtui_bridge_add_many_slaves finished with result PASS: 0
Sep 07 16:16:08 myown.hostname nmtui/./runtest.sh[25054]: Running test nmtui_bridge_over_ethernet_devices
Sep 07 16:16:09 myown.hostname sudo[25063]:     root : TTY=unknown ; PWD=/mnt/tests/NetworkManager ; USER=root ; COMMAND=/bin/pkill nmtui
Sep 07 16:16:10 myown.hostname NetworkManager[6845]: (NetworkManager:6845): GLib-ERROR **: Creating pipes for GWakeup: Too many open files
Sep 07 16:16:10 myown.hostname kernel: traps: NetworkManager[6845] trap int3 ip:7fabf139d8c3 sp:7ffe54704f70 error:0
Sep 07 16:16:10 myown.hostname systemd[1]: NetworkManager.service: main process exited, code=dumped, status=5/TRAP
Sep 07 16:16:10 myown.hostname systemd[1]: Unit NetworkManager.service entered failed state.
Sep 07 16:16:10 myown.hostname systemd[1]: NetworkManager.service failed.

Version-Release number of selected component:
NetworkManager-1.1.0-12937.e6c64af8be.el7.centos

Additional info:
reporter:       libreport-2.1.11
backtrace_rating: 4
cmdline:        /usr/sbin/NetworkManager --no-daemon
crash_function: g_wakeup_new
executable:     /usr/sbin/NetworkManager
kernel:         3.10.0-302.el7.x86_64
runlevel:       N 3
type:           CCpp
uid:            0

Truncated backtrace:
Thread no. 1 (6 frames)
 #2 g_wakeup_new at gwakeup.c:161
 #3 g_cancellable_make_pollfd at gcancellable.c:405
 #4 g_socket_condition_timed_wait at gsocket.c:3650
 #5 g_socket_accept at gsocket.c:2234
 #6 accept_ready at gsocketlistener.c:696
 #7 socket_source_dispatch at gsocket.c:3264
Comment 1 Lubomir Rintel 2015-09-08 08:44:30 UTC 
Created attachment 1071241 [details]
File: backtrace
Comment 2 Lubomir Rintel 2015-09-08 08:44:31 UTC 
Created attachment 1071242 [details]
File: cgroup
Comment 3 Lubomir Rintel 2015-09-08 08:44:32 UTC 
Created attachment 1071243 [details]
File: core_backtrace
Comment 4 Lubomir Rintel 2015-09-08 08:44:33 UTC 
Created attachment 1071244 [details]
File: dso_list
Comment 5 Lubomir Rintel 2015-09-08 08:44:34 UTC 
Created attachment 1071245 [details]
File: environ
Comment 6 Lubomir Rintel 2015-09-08 08:44:35 UTC 
Created attachment 1071246 [details]
File: limits
Comment 7 Lubomir Rintel 2015-09-08 08:44:36 UTC 
Created attachment 1071247 [details]
File: machineid
Comment 8 Lubomir Rintel 2015-09-08 08:44:37 UTC 
Created attachment 1071248 [details]
File: maps
Comment 9 Lubomir Rintel 2015-09-08 08:44:38 UTC 
Created attachment 1071249 [details]
File: open_fds
Comment 10 Lubomir Rintel 2015-09-08 08:44:39 UTC 
Created attachment 1071250 [details]
File: proc_pid_status
Comment 11 Lubomir Rintel 2015-09-08 08:44:40 UTC 
Created attachment 1071251 [details]
File: var_log_messages
Comment 13 Beniamino Galvani 2015-09-08 14:37:02 UTC 
Created attachment 1071384 [details]
[PATCH] core: fix file descriptor leak in bus manager

The attached patch resolves the problem, however I'm not quite sure of the fix. Can someone please have a look at it?

Steps to reproduce:

1. lsof -p `pidof NetworkManager`
2. activate a connection with ipv4.method = auto
3. lsof -p `pidof NetworkManager` # check if there are additional '/var/run/NetworkManager/private' unix sockets
Comment 14 Thomas Haller 2015-09-08 14:54:07 UTC 
(In reply to Beniamino Galvani from comment #13)
> Created attachment 1071384 [details]
> [PATCH] core: fix file descriptor leak in bus manager
> 
> The attached patch resolves the problem, however I'm not quite sure of the
> fix. Can someone please have a look at it?

I don't know, but it doesn't look wrong (if it fixes the problem).

Is it possible that we leak the GDBusConnection? (you could find out by attaching a weak-ref and logging it's destruction).
Because it seems to me that destroying the connection should also close the file handle...

CC-ing danw.
Comment 15 Dan Winship 2015-09-08 15:15:57 UTC 
I think this is a bug in GLib. In particular, if you change that to g_dbus_connection_close_sync(), then it will leak the fd in this case just like we were doing before.

It might be better to put this in private_server_closed(), with a FIXME comment?
Comment 16 Beniamino Galvani 2015-09-08 16:39:35 UTC 
Created attachment 1071412 [details]
[PATCH v2] core: fix file descriptor leak in bus manager

(In reply to Thomas Haller from comment #14)
> Is it possible that we leak the GDBusConnection? (you could find out by
> attaching a weak-ref and logging it's destruction).

Apparently the connection is not leaked, but a close() is required to free
the connection resources.

(In reply to Dan Winship from comment #15)
> I think this is a bug in GLib. In particular, if you change that to
> g_dbus_connection_close_sync(), then it will leak the fd in this case just
> like we were doing before.
>
> It might be better to put this in private_server_closed(), with a FIXME
> comment?

What about the attached patch?
Comment 17 Dan Winship 2015-09-08 18:20:28 UTC 
lgtm
Comment 18 Beniamino Galvani 2015-09-09 08:19:38 UTC 
Changed comment style to C89 and applied to master:

8318672fe997 core: fix file descriptor leak in bus manager
Comment 19 Thomas Haller 2015-09-30 10:02:57 UTC 
This bug was opened for a test-failure while testing upstream master.

nm-1-0 branch is not affected because it doesn't contain the code in question (NMBusManager). Especially, there are no affected RHEL-7 builds.


Closing BZ.
Note You need to log in before you can comment on or make changes to this bug.