Bug 470706

Summary: beagled uses up all inotify watches
Product: [Fedora] Fedora Reporter: Scott Doty <scott>
Component: beagleAssignee: Adel Gadllah <adel.gadllah>
Status: CLOSED WONTFIX QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: medium    
Version: 10CC: adel.gadllah, aliux.parra, aravind.parchuri, davidz, dknapp, john, mclasen, redhat-bugzilla, rhughes, sashang, scott, timosha
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2009-12-18 06:48:02 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
typescript of gdb session, attempting to run gpk-update-viewer
none
ps -axww output none

Description Scott Doty 2008-11-09 03:41:08 UTC
Created attachment 322988 [details]
typescript of gdb session, attempting to run gpk-update-viewer

+++ This bug was initially created as a clone of Bug #456513 +++

Description of problem:
After running a Gnome app based on PolicyKit a number of times, the app stops
working. Some affected apps:
- gpk-application
- gpk-update-viewer
- polkit-gnome-example

Version-Release number of selected component (if applicable):
PolicyKit-0.8-2.fc9.x86_64
PolicyKit-0.8-2.fc9.i386
PolicyKit-gnome-0.8-4.fc9.x86_64

How reproducible:
Run polkit-based app a number of times.

Steps to Reproduce:
1.
2.
3.
  
Actual results:
App crashes with following backtrace:

[WARN 22576] polkit-error.c:143:polkit_error_get_error_message(): error != NULL
 Not built with -rdynamic so unable to print a backtrace

** (polkit-gnome-example:22576): WARNING **: Failed to initialize PolicyKit
context: (null)
[WARN 22576] polkit-error.c:156:polkit_error_free(): error != NULL
 Not built with -rdynamic so unable to print a backtrace
[WARN 22576] polkit-error.c:143:polkit_error_get_error_message(): error != NULL
 Not built with -rdynamic so unable to print a backtrace

** (polkit-gnome-example:22576): WARNING **: Failed to initialize PolicyKit
context: (null)
[WARN 22576] polkit-error.c:156:polkit_error_free(): error != NULL
 Not built with -rdynamic so unable to print a backtrace

(polkit-gnome-example:22576): GLib-GObject-WARNING **: invalid (NULL) pointer
instance

(polkit-gnome-example:22576): GLib-GObject-CRITICAL **: g_signal_connect_data:
assertion `G_TYPE_CHECK_INSTANCE (instance)' failed

(polkit-gnome-example:22576): GLib-GObject-WARNING **: invalid (NULL) pointer
instance

(polkit-gnome-example:22576): GLib-GObject-CRITICAL **: g_signal_connect_data:
assertion `G_TYPE_CHECK_INSTANCE (instance)' failed


Expected results:
App runs the same as normally.

Additional info:

--- Additional comment from dennis.wagelaar.be on 2008-07-24 06:25:50 EDT ---

Workaround:
Log out and back in: PolicyKit-based apps are working again

--- Additional comment from aravind.parchuri on 2008-08-27 21:06:03 EDT ---

I can confirm this bug - apps like gnome-system-monitor, the calendar applet, etc start crashing after a while.

Here's the backtrace in case it helps:
** (gnome-system-monitor:13931): WARNING **: Failed to initialize PolicyKit context: (null)
[WARN 13931] polkit-error.c:156:polkit_error_free(): error != NULL
 Not built with -rdynamic so unable to print a backtrace

---Type <return> to continue, or q <return> to quit---
Program received signal SIGSEGV, Segmentation fault.
0x02fbc5f2 in ?? () from /usr/lib/libpolkit-gnome.so.0
(gdb) bt
#0  0x02fbc5f2 in ?? () from /usr/lib/libpolkit-gnome.so.0
#1  0x02fbe938 in ?? () from /usr/lib/libpolkit-gnome.so.0
#2  0x00369991 in ?? () from /lib/libgobject-2.0.so.0
#3  0x02fbe99b in ?? () from /usr/lib/libpolkit-gnome.so.0
#4  0x0036a0de in g_object_newv () from /lib/libgobject-2.0.so.0
#5  0x0036acd2 in g_object_new_valist () from /lib/libgobject-2.0.so.0
#6  0x0036ae1e in g_object_new () from /lib/libgobject-2.0.so.0
#7  0x02fbcfe8 in polkit_gnome_action_new_default ()
   from /usr/lib/libpolkit-gnome.so.0
#8  0x08055577 in gtk_toggle_button_set_active ()
#9  0x08054528 in gtk_toggle_button_set_active ()
#10 0x009b15d6 in __libc_start_main () from /lib/libc.so.6
#11 0x08052bb1 in gtk_toggle_button_set_active ()
(gdb)

--- Additional comment from aravind.parchuri on 2008-08-27 21:07:41 EDT ---

BTW, I'm on i386, not x86_64.

--- Additional comment from rhughes on 2008-08-28 04:17:21 EDT ---

Have you got a 100% way of reproducing this? I can't see to reproduce on i386 or x64.

--- Additional comment from aravind.parchuri on 2008-08-28 15:18:53 EDT ---

I don't really have a step-by-step procedure to reproduce this. I'm pretty sure that a few hours after logging into a gnome session, some apps start crashing with this error, consistently.

I'd be happy to help debug. Is there some policykit backend that I should monitor? Any locks that might be stuck? Maybe I'll just snapshot /var/lib/PolicyKit* before and after the problem and attach them if they don't contain any personal info.

--- Additional comment from dennis.wagelaar.be on 2008-09-01 10:21:27 EDT ---

This does happen 100% of the time for me on both of my Fedora machines at home. Both of those machines were upgraded from Fedora 8 using yum (http://fedoraproject.org/wiki/YumUpgradeFaq).

--- Additional comment from john.uk on 2008-10-15 12:47:26 EDT ---

This is happening on my f9 x86_64 box that I upgraded from fc6 x86_64 too.

Again, I can't run various things that use PolicyKit, including gnome-system-monitor, polkit-gnome-example and gpk-update-icon.

A few lines from an strace of polikit-gnome-example that also occur when the other apps are run:

inotify_init()                          = 6
inotify_add_watch(6, "/etc/PolicyKit/PolicyKit.conf", IN_MODIFY|IN_ATTRIB|IN_CREATE) = -1 ENOSPC (No space left on device)
write(2, "[WARN 20006] polkit-error.c:143:"..., 80[WARN 20006] polkit-error.c:143:polkit_error_get_error_message(): error != NULL
) = 80
write(2, " Not built with -rdynamic so una"..., 57 Not built with -rdynamic so unable to print a backtrace
) = 57
write(2, "\n** (polkit-gnome-example:20006)"..., 93
** (polkit-gnome-example:20006): WARNING **: Failed to initialize PolicyKit context: (null)


(and, no, my disc _isn't_ full)

--- Additional comment from john.uk on 2008-10-15 12:50:09 EDT ---

Oh, and why is this severity "low"? An important part of the system stops working and the only way is to fix it is to log out of your whole desktop session?

Where "low" is meant to be:

Problem is minor loss of function, or other problem where easy workaround is present.

Not severity "low" in my mind.

--- Additional comment from rhughes on 2008-10-15 13:58:41 EDT ---

(In reply to comment #8)
> Oh, and why is this severity "low"?

Because it's low priority for me, at the moment.

Does this happen on a fresh install, or just from upgrades? What is the difference in packages installed from an F8->F9 upgrade and a fresh F9 install?

Richard.

--- Additional comment from aravind.parchuri on 2008-10-15 15:47:30 EDT ---

I found the bug on a fresh F9 install.

--- Additional comment from john.uk on 2008-10-15 17:47:15 EDT ---

On my upgraded an fully updated system:

# rpm -qa|grep -i policy
PolicyKit-gnome-0.8-4.fc9.x86_64
PolicyKit-gnome-libs-0.8-4.fc9.x86_64
PolicyKit-gnome-demo-0.8-4.fc9.x86_64
selinux-policy-3.3.1-95.fc9.noarch
PolicyKit-0.8-3.fc9.x86_64
selinux-policy-targeted-3.3.1-95.fc9.noarch
policycoreutils-2.0.52-8.fc9.x86_64
checkpolicy-2.0.16-3.fc9.x86_64

# rpm -qa|grep -i dbus
ndesk-dbus-glib-0.4.1-3.fc9.x86_64
dbus-glib-devel-0.74-8.fc9.x86_64
dbus-debuginfo-1.2.1-1.fc9.x86_64
dbus-libs-1.2.4-1.fc9.x86_64
dbus-glib-0.74-8.fc9.i386
dbus-1.2.4-1.fc9.x86_64
dbus-devel-1.2.4-1.fc9.x86_64
dbus-glib-debuginfo-0.74-8.fc9.x86_64
dbus-libs-1.2.4-1.fc9.i386
dbus-sharp-0.63-9.fc9.x86_64
dbus-glib-0.74-8.fc9.x86_64
dbus-python-0.83.0-2.fc9.x86_64
dbus-x11-1.2.4-1.fc9.x86_64
dbus-qt-0.70-4.fc9.x86_64
ndesk-dbus-0.6.1a-2.fc9.x86_64

I was about to paste a list of packages on a clean install that had been fully updated, but I tried to get such a system up to date, and got to a point where gpk-update-icon stopped working in a different way, namely:

process 3394: arguments to dbus_message_new_method_call() were incorrect, assertion "_dbus_check_is_valid_path (path)" failed in file dbus-message.c line 1074. This is normally a bug in some application using the D-Bus library.
 D-Bus not built with -rdynamic so unable to print a backtrace

At this point:

# rpm -qa |grep -i policy
PolicyKit-0.8-2.fc9.x86_64
checkpolicy-2.0.14-1.fc9.x86_64
policycoreutils-2.0.46-5.fc9.x86_64
selinux-policy-targeted-3.3.1-42.fc9.noarch
PolicyKit-gnome-libs-0.8-4.fc9.x86_64
PolicyKit-gnome-0.8-4.fc9.x86_64
policycoreutils-gui-2.0.46-5.fc9.x86_64
selinux-policy-3.3.1-42.fc9.noarch


# rpm -qa |grep -i dbus
dbus-libs-1.2.1-1.fc9.x86_64
dbus-1.2.1-1.fc9.x86_64
dbus-x11-1.2.1-1.fc9.x86_64
ndesk-dbus-glib-0.4.1-3.fc9.x86_64
dbus-glib-0.74-6.fc9.x86_64
dbus-python-0.82.4-2.fc9.x86_64
dbus-qt-0.70-4.fc9.x86_64
ndesk-dbus-0.6.1a-2.fc9.x86_64


After a 'yum upgrade' :

# rpm -qa|grep -i policy
PolicyKit-0.8-3.fc9.x86_64
checkpolicy-2.0.16-3.fc9.x86_64
PolicyKit-gnome-0.8-4.fc9.x86_64
selinux-policy-targeted-3.3.1-95.fc9.noarch
selinux-policy-3.3.1-95.fc9.noarch
selinux-policy-devel-3.3.1-95.fc9.noarch
policycoreutils-gui-2.0.52-8.fc9.x86_64
PolicyKit-gnome-libs-0.8-4.fc9.x86_64
policycoreutils-2.0.52-8.fc9.x86_64

# rpm -qa|grep -i dbus
dbus-x11-1.2.4-1.fc9.x86_64
dbus-glib-0.74-8.fc9.x86_64
dbus-qt-0.70-4.fc9.x86_64
ndesk-dbus-0.6.1a-2.fc9.x86_64
dbus-1.2.4-1.fc9.x86_64
dbus-python-0.83.0-2.fc9.x86_64
ndesk-dbus-glib-0.4.1-3.fc9.x86_64
dbus-libs-1.2.4-1.fc9.x86_64


Note, the clean installation is only a test system, I've not used it enough to say whether or not this problem will occur or not.

I can provide the complete 'rpm -qa' for either of these systems if required.

--- Additional comment from dennis.wagelaar.be on 2008-10-16 04:15:35 EDT ---

For me, this happened on two x86_64 boxes. Both of them were upgraded via yum from F8 to F9 conform http://fedoraproject.org/wiki/YumUpgradeFaq

current package versions:
- PolicyKit-0.8-3.fc9.x86_64
- PolicyKit-gnome-0.8-4.fc9.x86_64

N.B. I no longer have the i386 version of PolicyKit installed. I also no longer manage to reproduce the problem. I ran the following command to clean up my installed packages:

  # package-cleanup --cleandupes

Can you guys check if this works for you too?

--- Additional comment from john.uk on 2008-10-17 04:37:50 EDT ---

(In reply to comment #12)
[snip] 
> N.B. I no longer have the i386 version of PolicyKit installed. I also no longer
> manage to reproduce the problem. I ran the following command to clean up my
> installed packages:
> 
>   # package-cleanup --cleandupes
> 
> Can you guys check if this works for you too?

# package-cleanup --cleandupes
Setting up yum
No dupes to clean

--- Additional comment from rhughes on 2008-10-27 09:12:23 EDT ---

*** Bug 452080 has been marked as a duplicate of this bug. ***

--- Additional comment from scott on 2008-11-08 22:30:58 EDT ---

Created an attachment (id=322987)
typescript of gdb session, attempting to run gpk-update-viewer

I have this same problem with rawhide (f10) -- attached is typescript from script(1) showing multiple segfaults w/their backtraces...

I first noticed this problem when I could not spawn the update viewer from the update applet.

If it matters, this is a gnome desktop, using compiz (spawned with compiz-manager --replace) and emerald (emerald --replace).

Comment 1 Matthias Clasen 2008-11-10 00:45:21 UTC
the strace several comments up looks like something might be eating up all available inotify watches. Are you running some inotify hog, like trackerd ?

Comment 2 Scott Doty 2008-11-10 02:19:51 UTC
Created attachment 323037 [details]
ps -axww output

re: me running something hogging up inotify watches

None that I know of -- I've attached a ps output.

I was also running ccsm, and had the compiz filesystem enabled, but shutting it off didn't clear up the problem.

 -Scott

Comment 3 Matthias Clasen 2008-11-10 18:00:38 UTC
inotify hog could be beagle, possibly.

Comment 4 aravind.parchuri 2008-11-10 19:49:55 UTC
Good call! It was beagle on my system. As soon as I stopped beagle, the previously crashing applications started working properly again. I have around 43k directories/sub-directories in my home directory, and I guess beagle was using up all 8192 of the available inotify watches.

I also verified that it was the inotify watch limit by doing an "inotifywatch -r" to set up around 8k watches and managed to reproduce the crash. 

Any plans to increase the default fs.inotify.max_user_watches ?

Comment 5 John Beranek 2008-11-10 20:57:00 UTC
Aha, killing beagled fixes the problems for me too!

Comment 6 Scott Doty 2008-11-10 21:28:07 UTC
I haven't tried this yet, as I'm at the office, but I shall this afternoon.

This is very cool!  And it makes sense:  iirc, i remember seeing that
beagled has a delay that it waits at startup before it starts doing its     
stuff.

I wonder how SuSE handles this situation?

 -Scott

Comment 7 Bug Zapper 2008-11-26 05:04:58 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 10 development cycle.
Changing version to '10'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 8 sashan 2009-02-13 06:35:28 UTC
Just to confirm this happens with FC10 as well. Right clicking on the desktop and then Change Desktop Background would crash. Killing beagled resolved this.

Comment 9 Nathan G. Grennan 2009-03-26 18:47:26 UTC
When is this going to get fixed? I just ran into this on my FC10 x86_64 system. I only figured it out after using strace and google.

Comment 10 Bug Zapper 2009-11-18 08:48:37 UTC
This message is a reminder that Fedora 10 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 10.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '10'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 10's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 10 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 11 Bug Zapper 2009-12-18 06:48:02 UTC
Fedora 10 changed to end-of-life (EOL) status on 2009-12-17. Fedora 10 is 
no longer maintained, which means that it will not receive any further 
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.