Bug 198935 - evolution won't start and prints "too many open files error"
Summary: evolution won't start and prints "too many open files error"
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: evolution-data-server
Version: rawhide
Hardware: All
OS: Linux
high
medium
Target Milestone: ---
Assignee: Matthew Barnes
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2006-07-14 20:11 UTC by Ray Strode [halfline]
Modified: 2007-11-30 22:11 UTC (History)
2 users (show)

Fixed In Version: evolution-data-server-1.8.0-2.fc6
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2006-09-23 00:25:32 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
strace of the crash (2.41 MB, application/octet-stream)
2006-07-14 20:20 UTC, Ray Strode [halfline]
no flags Details
Result of running "valgrind --track-fds=yes evolution" (359.79 KB, text/plain)
2006-08-21 21:38 UTC, Dave Malcolm
no flags Details
Result of running "valgrind --track-fds=yes evolution" (1.45 MB, text/plain)
2006-08-28 17:44 UTC, Dave Malcolm
no flags Details


Links
System ID Private Priority Status Summary Last Updated
GNOME Bugzilla 348888 0 None None None Never

Description Ray Strode [halfline] 2006-07-14 20:11:04 UTC
Earlier today I did a yum upgrade to the latest rawhide.  evolution was running
at the time.  Some hours later (after the upgrade finished) I closed evolution
and it crashed on exit.

Now sometimes when I start it I get this error:

[rstrode@halflap] (~) <01:05 PM>
$ evolution
CalDAV Eplugin starting up ...
evolution-shell-Message: Killing old version of evolution-data-server...

(evolution-2.8:4577): camel-WARNING **: camel_exception_get_id called with NULL
parameter.

(evolution-2.8:4577): Pango-WARNING **: shape engine failure, expect ugly
output. the offending font is 'Bitstream Vera Sans Bold 9'

(evolution-2.8:4577): Pango-WARNING **: pango_font_get_glyph_extents called with
bad font, expect ugly output
I/O error : Too many open files
I/O warning : failed to load external entity
"/usr/share/evolution/2.8/ui/evolution-mail-global.xml"

(evolution-2.8:4577): Bonobo-CRITICAL **: bonobo_ui_node_to_string: assertion
`node != NULL' failed
I/O error : Too many open files
I/O warning : failed to load external entity
"/usr/share/evolution/2.8/ui/evolution-mail-list.xml"

(evolution-2.8:4577): Bonobo-CRITICAL **: bonobo_ui_node_to_string: assertion
`node != NULL' failed
I/O error : Too many open files
I/O warning : failed to load external entity
"/usr/share/evolution/2.8/ui/evolution-mail-message.xml"

(evolution-2.8:4577): Bonobo-CRITICAL **: bonobo_ui_node_to_string: assertion
`node != NULL' failed
I/O error : Too many open files
I/O warning : failed to load external entity
"/usr/lib/evolution/2.8/plugins/org-gnome-mailing-list-actions.xml"

(evolution-2.8:4577): Bonobo-CRITICAL **: bonobo_ui_node_to_string: assertion
`node != NULL' failed
I/O error : Too many open files
I/O warning : failed to load external entity
"/usr/lib/evolution/2.8/plugins/org-gnome-mail-to-task.xml"

(evolution-2.8:4577): Bonobo-CRITICAL **: bonobo_ui_node_to_string: assertion
`node != NULL' failed

(evolution-2.8:4577): GLib-WARNING **: GError set over the top of a previous
GError or uninitialized memory.
This indicates a bug in someone's code. You must ensure an error is NULL before
it's set.
The overwriting error message was: Error opening directory
'/local-users/rstrode/.evolution/mail/imap/rstrode.redhat.com/folders/INBOX':
Too many open files

Comment 1 Ray Strode [halfline] 2006-07-14 20:20:26 UTC
Created attachment 132460 [details]
strace of the crash

Note if i try enough times evolution eventually starts up, although sometimes
it looks rather broken, like no icons showing up (just big red X's)

Comment 2 Ray Strode [halfline] 2006-07-26 20:20:25 UTC
So Alex and I had a look at this today.  We think the problem is I just have too
many vfolders. Each vfolder spawns several asynchronomous operations, each
creating a pipe (and 2 file descriptors per pipe).  I have over a hundred
vfolders, so i hit the default hard limit (of 10240) pretty fast.

I think i'm a bit of a minority having that many vfolders, so I don't think this
is a widespread problem.

We should probably get this filed upstream and work on it there.

Removing from FC6 Desktop tracker.

Here's an example stack trace showing one of the hundreds of pipes it creates:

#0  0x05f3d060 in pipe () from /lib/libc.so.6
#1  0x0410b6ab in PR_CreatePipe () from /usr/lib/libnspr4.so
#2  0x0027f9c3 in e_msgport_new () at e-msgport.c:521
#3  0x009b8e24 in camel_operation_new (status=0x36ead30 <mail_enable_stop+800>,
status_data=0x23) at camel-operation.c:136
#4  0x036ea5fb in mail_msg_new () from
/usr/lib/evolution/2.8/components/libevolution-mail.so
#5  0x036eb760 in mail_async_event_emit () from
/usr/lib/evolution/2.8/components/libevolution-mail.so
#6  0x036b158a in em_folder_tree_model_get_expanded () from
/usr/lib/evolution/2.8/components/libevolution-mail.so
#7  0x009b6aeb in camel_object_trigger_event (vo=0x8331b40, name=0x20eb74
"folder_created", event_data=0x83a07c8) at camel-object.c:1504
#8  0x002072c7 in change_folder (store=0x8331b40, name=0x837add0 "Fedora/Desktop
List", flags=<value optimized out>, count=0) at camel-vee-store.c:169
#9  0x002077c2 in vee_get_folder (store=0x8331b40, folder_name=0x8354908
"Fedora/Desktop List", flags=0, ex=0x0) at camel-vee-store.c:202
#10 0x001fe09f in camel_store_get_folder (store=0x8331b40, folder_name=0x8354908
"Fedora/Desktop List", flags=<value optimized out>, ex=0x0) at camel-store.c:260
#11 0x036f68a4 in vfolder_revert () from
/usr/lib/evolution/2.8/components/libevolution-mail.so
#12 0x036f6b99 in vfolder_load_storage () from
/usr/lib/evolution/2.8/components/libevolution-mail.so
#13 0x036e52df in mail_component_load_store_by_uri () from
/usr/lib/evolution/2.8/components/libevolution-mail.so
#14 0x005c3121 in _ORBIT_skel_small_GNOME_Evolution_Component_createView () from
/usr/lib/evolution/2.8/libeshell.so.0
#15 0x03c5f8b0 in ORBit_c_stub_invoke () from /usr/lib/libORBit-2.so.0
#16 0x005c4a73 in GNOME_Evolution_Component_createView () from
/usr/lib/evolution/2.8/libeshell.so.0
#17 0x080592c2 in POA_GNOME_Evolution_DataServer_InterfaceCheck__fini ()
#18 0x00947e89 in g_cclosure_marshal_VOID () from /lib/libgobject-2.0.so.0
#19 0x0093aedb in g_closure_invoke () from /lib/libgobject-2.0.so.0
#20 0x0094bda3 in g_signal_override_class_closure () from /lib/libgobject-2.0.so.0
#21 0x0094d29e in g_signal_emit_valist () from /lib/libgobject-2.0.so.0
#22 0x0094d459 in g_signal_emit () from /lib/libgobject-2.0.so.0
#23 0x0805bf5f in POA_GNOME_Evolution_DataServer_InterfaceCheck__fini ()
#24 0x08059c3e in POA_GNOME_Evolution_DataServer_InterfaceCheck__fini ()
#25 0x0805acfc in POA_GNOME_Evolution_DataServer_InterfaceCheck__fini ()
#26 0x08052b9e in POA_GNOME_Evolution_DataServer_InterfaceCheck__fini ()
#27 0x0805e469 in POA_GNOME_Evolution_DataServer_InterfaceCheck__fini ()
#28 0x007a4591 in g_source_is_destroyed () from /lib/libglib-2.0.so.0
#29 0x007a62f2 in g_main_context_dispatch () from /lib/libglib-2.0.so.0
#30 0x007a92cf in g_main_context_check () from /lib/libglib-2.0.so.0
#31 0x007a9679 in g_main_loop_run () from /lib/libglib-2.0.so.0
#32 0x054cea03 in bonobo_main () from /usr/lib/libbonobo-2.so.0
#33 0x0805dab6 in POA_GNOME_Evolution_DataServer_InterfaceCheck__fini ()
#34 0x05e98214 in __libc_start_main () from /lib/libc.so.6
#35 0x0804f3f1 in ?? ()


Comment 3 Ray Strode [halfline] 2006-07-26 20:21:44 UTC
Note, the e-d-s client lib should probably set a cap on how many async
operations it can do at once.

Comment 4 Matthias Clasen 2006-07-26 23:42:56 UTC
out of interest, how many vfolders do you have ?


Comment 5 Matthias Clasen 2006-07-27 02:06:46 UTC
Ray, we should also remove it from the fc6 blocker, and file an upstream bug,
I think.

Comment 6 Ray Strode [halfline] 2006-07-27 02:52:05 UTC
49, but find ~/.evolution/mail/vfolder | wc -l returns 133

Comment 7 Dave Malcolm 2006-08-21 19:11:05 UTC
I only have a few vfolders, but I'm still getting "Too many open files" errors.
 Do you want me to open a separate bug?

Comment 8 Dave Malcolm 2006-08-21 21:38:59 UTC
Created attachment 134601 [details]
Result of running "valgrind --track-fds=yes evolution"

This attachment shows large numbers of file descriptors open, with the call
stack of each one, using "valgrind --track-fds=yes evolution"

It looks to me that we're leaking file descriptors it at least one place; the
repeated traces of this form look suspicious to me:
==4580== Open file descriptor 597:

==4580==    at 0x4579382: pipe (in /lib/libc-2.4.90.so)

==4580==    by 0x41D9043: camel_operation_new (camel-operation.c:136)

==4580==    by 0x7FB396A: mail_msg_new (mail-mt.c:129)

==4580==    by 0x7FBC733: ??? (mail-session.c:383)

==4580==    by 0x422D612: camel_session_alert_user (camel-session.c:421)

==4580==    by 0x8011F0F: camel_imap_command_response
(camel-imap-command.c:332)

==4580==    by 0x8012C08: ??? (camel-imap-command.c:374)

==4580==    by 0x80132D9: camel_imap_command (camel-imap-command.c:116)

==4580==    by 0x8018234: ??? (camel-imap-folder.c:517)

==4580==    by 0x420B8C5: ??? (camel-disco-folder.c:268)

==4580==    by 0x421C8ED: camel_folder_refresh_info (camel-folder.c:298)

==4580==    by 0x7FBA469: ??? (mail-send-recv.c:747)

Comment 9 Ray Strode [halfline] 2006-08-28 15:37:50 UTC
Quite right Dave.

This got fixed upstream, should be in tomorrow's rawhide.

Comment 10 Dave Malcolm 2006-08-28 17:44:48 UTC
Created attachment 135067 [details]
Result of running "valgrind --track-fds=yes evolution"

Still giving me "too mnay open files" errors, this is with:
evolution-2.7.92-4.fc6
evolution-data-server-1.7.92-3.fc6

Comment 11 Matt Davey 2006-08-29 10:47:44 UTC
Note the upstream bug:
http://bugzilla.gnome.org/show_bug.cgi?id=350907

in which I report an fd leak with the e-d-s code that pipes messages through
external programs.   My fd-leak reports look like:
==10665== Open file descriptor 64:
==10665==    at 0x51528D2: pipe (in /lib/libc-2.4.so)
==10665==    by 0x502AA84: g_child_watch_source_new (gmain.c:3620)
==10665==    by 0x42129E5: ??? (camel-filter-search.c:595)
==10665==    by 0x43DC130: e_sexp_term_eval (e-sexp.c:710)
==10665==    by 0x43DC3D7: ??? (e-sexp.c:440)
==10665==    by 0x43DC178: e_sexp_term_eval (e-sexp.c:700)
==10665==    by 0x421384C: ??? (camel-filter-search.c:327)
==10665==    by 0x43DC178: e_sexp_term_eval (e-sexp.c:700)
==10665==    by 0x43DCEC6: ??? (e-sexp.c:255)
==10665==    by 0x43DC178: e_sexp_term_eval (e-sexp.c:700)
==10665==    by 0x43DC1FF: e_sexp_eval (e-sexp.c:1304)
==10665==    by 0x4212671: camel_filter_search_match
(camel-filter-search.c:705)

Comment 12 Matthew Barnes 2006-09-11 15:56:19 UTC
I think I have a handle on this now.
See the upstream bug report (#348888) for details.

Comment 13 Matthew Barnes 2006-09-12 14:18:23 UTC
Fixed in evolution-data-server-1.8.0-2.fc6.

Please re-open this bug if you find that you're still experiencing the problem.

Comment 14 Matthew Barnes 2006-09-15 12:25:10 UTC
It turns out that the patch I proposed for this bug actually reverts a previous
patch for a race condition in EMsgPort.  That previous patch (committed last
May) is what introduced this "too many open files" problem.

Reopening this bug until I can get both problems sorted out.

Comment 15 Matthew Barnes 2006-09-15 12:45:07 UTC
To clarify, the latest package (1.8.0-2.fc6) fixes the mass file descriptor
usage at start-up but does have a potential race condition, though it seldom
manifests.  I'd say the problem now is much more benign than Evolution failing
to start half the time.

My solution to the race condition is a bit radical (i.e. a total rewrite of the
EMsgPort implementation) and I'm not comfortable with it slipping into FC6 until
it gets more widespread testing.

So I'm okay with shipping evolution-data-server-1.8.0-2.fc6.

Comment 16 Matthias Clasen 2006-09-22 23:37:57 UTC
Matt, whats the status of this for FC6 ?

Comment 17 Matthew Barnes 2006-09-23 00:25:32 UTC
The patch has been in Rawhide since Monday and seems to be holding up.  It fixes
the problem that Ray reported as well as the race condition.  I'm more confident
now that we can ship the patch in FC6, so closing this bug.

Comment 18 Nicole Dai 2006-10-13 04:41:20 UTC
Confirmed the bug was not found in 
evolution-2.8.0-6.fc6 
evolution-data-server-1.8.0-11.fc6


Note You need to log in before you can comment on or make changes to this bug.