Bug 916890 - Evolution consumes huge amounts of memory, is slow to open new mail window and blocks on send (IMAP+, EWS)
Summary: Evolution consumes huge amounts of memory, is slow to open new mail window an...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: evolution
Version: 19
Hardware: All
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Matthew Barnes
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-03-01 08:36 UTC by Bojan Smojver
Modified: 2013-08-09 07:07 UTC (History)
3 users (show)

Fixed In Version: evolution-data-server-3.6.4
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-06-28 09:58:45 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
Valgrind log (72.80 KB, application/x-bzip2)
2013-03-10 23:10 UTC, Bojan Smojver
no flags Details
Valgrind log (514.56 KB, application/x-bzip2)
2013-03-12 04:44 UTC, Bojan Smojver
no flags Details
Valgrind log of a killed (with ^C) valgrind (1.34 MB, application/x-bzip2)
2013-03-12 04:45 UTC, Bojan Smojver
no flags Details
Valgrind log, with malloc (183.20 KB, application/x-bzip2)
2013-03-12 10:12 UTC, Bojan Smojver
no flags Details
Valgrind log, compose, killed with ^C (1.32 MB, application/x-bzip2)
2013-03-19 01:37 UTC, Bojan Smojver
no flags Details


Links
System ID Private Priority Status Summary Last Updated
GNOME Bugzilla 689476 0 Normal RESOLVED Slow composer open (emoticon and color widget leaks) 2020-04-23 16:09:45 UTC

Description Bojan Smojver 2013-03-01 08:36:44 UTC
Description of problem:
The first problem is IMAP+ specific. Right now, my Evo (on x86_64) is consuming 438 MB of RES memory. Even FF + plugin container consumes less than this. I do have a lot of mail across many folders, but do they all need to be cached or something? Same version of Evo (on i686) with EWS back end consumes 138 MB of RAM. Probably similar amount of mail. No idea why IMAP+ would consume that much...

Especially with EWS, Evo is exceptionally slow to open new mail window. Sometimes it takes 5 - 10 seconds for this. Regression for sure, when compared to F-17.

The third problem is that every time an e-mail is sent, Evo blocks the rest of the user interface for many seconds. This is especially visible with IMAP+, but occurs with EWS as well. The IMAP+ box is an i5, so plenty of grunt. So, something must be blocking somewhere.

Version-Release number of selected component (if applicable):
evolution-3.6.3-2.fc18

How reproducible:
Always.

Steps to Reproduce:
1. See above.
  
Actual results:
Huge memory consumption. Blocked user interface.

Expected results:
This was much less of a problem (if at all) in previous versions (F-17).

Additional info:

Comment 1 Milan Crha 2013-03-01 10:40:56 UTC
Thanks for a bug report. There were identified few memory leaks in new mail composer window, which should be addressed in evolution 3.6.4, to be released in the middle of the next week. The long composer close is caused by these leaks, when the composer itself is disposed from memory (the last backtrace I saw was close to it). If there are other regressions caused by changes in for example gtk3 I cannot tell - maybe there are, maybe there are not. In any case, this [1] is an upstream bug report associated for this, thus I'm closing this in favour of it. Feel free to CC there, even the bug itself is currently in a closed state.

[1] https://bugzilla.gnome.org/show_bug.cgi?id=689476

Comment 2 Bojan Smojver 2013-03-01 11:12:35 UTC
OK, do you know what could be causing IMAP+ to use half a gig of RAM?

Comment 3 Milan Crha 2013-03-04 18:22:00 UTC
Not much idea, I'm sorry. Maybe try to run evolution under valgrind overnight, and then try to quit it gracefully, then we can check leaked memory, if any. You can run evolution under valgrind like this:
   $ valgrind --num-callers=50 --leak-check=full evolution &>log.txt

it's sometimes useful to export also G_SLICE=always-malloc before running the command, or just preppend it to it, like
   $ G_SLICE=always-malloc valgrind....
but it makes it even slower.

Comment 4 Bojan Smojver 2013-03-05 03:09:18 UTC
OK, I may run the above once we get the new release. No point doing it now.

Comment 6 Bojan Smojver 2013-03-06 22:55:02 UTC
Ah, I get it, it's a gtkhtml, not Evo bug.

Comment 7 Milan Crha 2013-03-07 07:41:14 UTC
Correct, it's a mix of gtkhtml3 and evolution-data-server (for master evolution) packages being affected.

Comment 8 Fedora Update System 2013-03-07 13:09:30 UTC
evolution-ews-3.6.4-1.fc18,evolution-mapi-3.6.4-1.fc18,evolution-3.6.4-2.fc18,evolution-data-server-3.6.4-2.fc18,gtkhtml3-4.6.4-1.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/evolution-ews-3.6.4-1.fc18,evolution-mapi-3.6.4-1.fc18,evolution-3.6.4-2.fc18,evolution-data-server-3.6.4-2.fc18,gtkhtml3-4.6.4-1.fc18

Comment 9 Bojan Smojver 2013-03-10 03:48:00 UTC
Better, but still leaking, I think. I'll keep an eye on it in the next few days.

Comment 10 Bojan Smojver 2013-03-10 21:28:07 UTC
(In reply to comment #9)
> Better, but still leaking, I think. I'll keep an eye on it in the next few
> days.

Before the last reboot, Evo started at about 130 MB of RES. This went up to about 200 MB over a few days of suspend/resume cycles. That is the one connected to IMAP.

The one connected to EWS is doing a lot better. Started at about 45 MB and went up to about 80 MB.

New kernel now, so I'll keep watch again.

Comment 11 Bojan Smojver 2013-03-10 23:10:21 UTC
Created attachment 708101 [details]
Valgrind log

Note: I tried composing an e-mail, but the window hung for probably half an hour (unresponsive), so I had to kill the whole thing. CPU was pegged at 100% and valgrind consumed 1.4 GB of RES memory at one point.

Comment 12 Milan Crha 2013-03-11 09:35:55 UTC
Thanks for the update. A clean shutdown helps to identify memory leaks under valgrind, but I agree that running New Mail under valgrind is basically impossible, I left it stuck for an hour here (on a not-slow machine), but it didn't move any further. The problem seems to be with the list of configured accounts, while if I have many (tens of them), then it does this, while when I have two, then it works.

I checked your attachment and it, unfortunately, is missing the overall statistics about lost memory. There is one "in the middle of run", which claims:
>   definitely lost: 6,144 bytes in 5 blocks
but it's basically nothing (not that it should be there, just the opposite, but it's rather a minor leak, and not in evolution itself (these are from GLib)).

I noticed in the log that there is missing debuginfo for evolution-data-server, which may mean either you do not have it installed, or the version of debuginfo doesn't match the version of binary package. Please make sure you've installed the debugfinfo for at least gtkhtml3, evolution-data-server, evolution, and other evolution-* packages you use (like the evolution-ews).

Comment 13 Bojan Smojver 2013-03-12 04:15:50 UTC
(In reply to comment #12)
 
> I noticed in the log that there is missing debuginfo for
> evolution-data-server

OOPS! Missed --enablerepo=updates-testing. Let me try that again.

Comment 14 Bojan Smojver 2013-03-12 04:44:14 UTC
Created attachment 708765 [details]
Valgrind log

Log of a regular run: start, preview some e-mails, send/receive, switch folders. No compose.

Comment 15 Bojan Smojver 2013-03-12 04:45:40 UTC
Created attachment 708766 [details]
Valgrind log of a killed (with ^C) valgrind

I started a compose here (apart from doing other things) and then did a ^C in valgrind. Hope you can get something out of it.

Comment 16 Milan Crha 2013-03-12 09:03:59 UTC
Thanks for the update. I see in the log one memory leak, but it doesn't do anything significant (I'm committing it for 3.7.92 of eds). There are reported more sever "possibly lost" chunks, but they seem to be cause by the GSlice allocator. I guess you run the valgrind without G_SLICE=always-malloc, right? (I'm fine if you did, as I noted in comment #3, it's more accurate with respect of definitely leaked, but also significantly slower).

Comment 17 Bojan Smojver 2013-03-12 09:40:59 UTC
Yes, correct.

Comment 18 Bojan Smojver 2013-03-12 10:12:08 UTC
Created attachment 708887 [details]
Valgrind log, with malloc

For your bug busting pleasure.

Comment 19 Bojan Smojver 2013-03-12 22:49:44 UTC
One more detail, just in case it matters. I have a feeling most leaks occur after I suspend/resume the machine (I do this every day, at least once). Have no proof for this, of course...

Comment 20 Milan Crha 2013-03-13 08:30:32 UTC
Thanks for the update. I see the 'definitely' lost are minor, mostly come from other libraries, and those in evolution are already fixed for 3.8.0. Te most "possibly lost", which I understand as memory being still addressed from somewhere, are related to imapx and sqlite tables. That can be caused by a not-freed object at the application end. Evolution uses couple of internal caches, some are with CamelStores/Services, where I guess one of them is causing just this, thus the valgrind identified it properly (it's a false positive for us, which valgrind didn't mark as 'definitely' lost as expected).

If you feel the most leaking part is during suspend, then feel free to suspend with evolution being run under valgrind. There can be a difference with respect of suspend time, like if you resume too early, then the connections can still be alive, while if you'll resume later, then the connections will be lost and the evolution will do other stuff around.

Thinking of suspend, leaks and IMAP+, do you have enabled QResync for your IMAP+ accounts? I would try to disable it, in case you do. I was told by other user that it may do a difference.

Comment 21 Bojan Smojver 2013-03-13 22:57:45 UTC
(In reply to comment #20)
 
> If you feel the most leaking part is during suspend, then feel free to
> suspend with evolution being run under valgrind.

OK, I may try that.

> Thinking of suspend, leaks and IMAP+, do you have enabled QResync for your
> IMAP+ accounts? I would try to disable it, in case you do. I was told by
> other user that it may do a difference.

Yeah, I do (there is only one IMAP+ account). Makes things significantly quicker with large amounts of mail, right? OK, I may try that as well and see how things go.

Comment 22 Fedora Update System 2013-03-14 02:40:39 UTC
evolution-ews-3.6.4-1.fc18, evolution-mapi-3.6.4-1.fc18, evolution-3.6.4-2.fc18, evolution-data-server-3.6.4-2.fc18, gtkhtml3-4.6.4-1.fc18 has been pushed to the Fedora 18 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 23 Bojan Smojver 2013-03-14 02:54:21 UTC
(In reply to comment #19)
> One more detail, just in case it matters. I have a feeling most leaks occur
> after I suspend/resume the machine (I do this every day, at least once).
> Have no proof for this, of course...

Not the suspend/resume. I just had Evo grow form 198 MB to 321 MB RES during half a day. I received a few e-mails and sent a few as well. When I first started this instance of Evo a couple of days ago (kernel update reboot), it was taking around 130 MB of RES.

So, yeah, still leaking...

Comment 24 Bojan Smojver 2013-03-15 00:19:44 UTC
(In reply to comment #23)
> I just had Evo grow form 198 MB to 321 MB RES

At 410 MB today. So, still growing rather rapidly.

Comment 25 Bojan Smojver 2013-03-15 06:15:42 UTC
(In reply to comment #20)

> Thinking of suspend, leaks and IMAP+, do you have enabled QResync for your
> IMAP+ accounts? I would try to disable it, in case you do. I was told by
> other user that it may do a difference.

Pretty sure it is not related to this. I disabled this feature and I sync mail every 3 minutes for this account, but that didn't increase memory use.

However, composing mail still does. Evo was at 168 MB RES. Then I composed about 10 e-mails (including one that had a 10 MB attachment). My Evo is now consuming 285 MB RES.

Comment 26 Milan Crha 2013-03-15 15:44:47 UTC
OK, I found another leak, This one is nice, each message which is sent is also leaked. It's shown as a Definitely leaked memory in a valgrind log. It's caused by double-reffing message object. I'll create an update for it, though I currently cannot reach the koji for some reason, thus it'll be probably some time the next week, likely on Monday.

Comment 27 Fedora Update System 2013-03-15 16:48:39 UTC
evolution-3.6.4-3.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/evolution-3.6.4-3.fc18

Comment 28 Bojan Smojver 2013-03-15 21:23:28 UTC
Brilliant, thank you! I will give it a try.

Comment 29 Bojan Smojver 2013-03-16 03:58:54 UTC
(In reply to comment #27)
> evolution-3.6.4-3.fc18 has been submitted as an update for Fedora 18.
> https://admin.fedoraproject.org/updates/evolution-3.6.4-3.fc18

Unfortunately, I don't think this fixed it. Evo started at 128 MB RES. I sent some 10 e-mails, including one with a 10 MB attachment. Evo is now consuming 272 MB RES, after I left it idle for about 1/2 hour.

Comment 30 Bojan Smojver 2013-03-19 01:37:29 UTC
Created attachment 712324 [details]
Valgrind log, compose, killed with ^C

Just another log, in the hope it will help. I left the compose window to pickle for about an hour (it never actually finished painting the toolbar). The whole valgrind run was consuming 2.8 GB of RAM by then. I killed it then by pressing ^C.

Comment 31 Milan Crha 2013-03-20 08:45:29 UTC
Thanks for the update. I found couple more leaks yesterday on my own, nothing related to composer, but memory which can be leaking in Mailer use of evolution. The above log doesn't show anything significant in definitely lost memory, but the possibly lost suggests that something wrong is going on in gtk_action_group_add_radio_actions(), or our own code. Let's move to upstream bug [1] with this.

[1] https://bugzilla.gnome.org/show_bug.cgi?id=696173

==22577== 13,655,040 bytes in 6,720 blocks are possibly lost in loss record 42,019 of 42,026
==22577==    at 0x4A068EA: memalign (vg_replace_malloc.c:727)
==22577==    by 0x4A06985: posix_memalign (vg_replace_malloc.c:876)
==22577==    by 0x3129E1A777: slab_allocator_alloc_chunk (gslice.c:1381)
==22577==    by 0x3129E62CE8: g_slice_alloc (gslice.c:724)
==22577==    by 0x3129E62D45: g_slice_alloc0 (gslice.c:1029)
==22577==    by 0x312A630084: g_type_create_instance (gtype.c:1870)
==22577==    by 0x312A614A37: g_object_constructor (gobject.c:1854)
==22577==    by 0x312A616000: g_object_newv (gobject.c:1718)
==22577==    by 0x312A6167EF: g_object_new_valist (gobject.c:1835)
==22577==    by 0x312A616B23: g_object_new (gobject.c:1550)
==22577==    by 0x3D1F5D918D: gtk_radio_action_new (gtkradioaction.c:219)
==22577==    by 0x3D1F49EBCD: gtk_action_group_add_radio_actions_full (gtkactiongroup.c:1402)
==22577==    by 0x3D1F49ECF0: gtk_action_group_add_radio_actions (gtkactiongroup.c:1350)
==22577==    by 0x14917201: e_mail_shell_view_update_search_filter (e-mail-shell-view-actions.c:2129)
==22577==    by 0x312A60F90F: g_closure_invoke (gclosure.c:777)
==22577==    by 0x312A620D07: signal_emit_unlocked_R (gsignal.c:3551)
==22577==    by 0x312A628C8C: g_signal_emit_valist (gsignal.c:3300)
==22577==    by 0x312A628DE1: g_signal_emit (gsignal.c:3356)
==22577==    by 0x3D1F58B8DD: gtk_list_store_insert_with_values (gtkliststore.c:2250)
==22577==    by 0x10581B2B: labels_settings_changed_cb (e-mail-label-list-store.c:276)
==22577==    by 0x312A612736: g_cclosure_marshal_VOID__STRINGv (gmarshal.c:1004)
==22577==    by 0x312A60FBD6: _g_closure_invoke_va (gclosure.c:840)
==22577==    by 0x312A6283A7: g_signal_emit_valist (gsignal.c:3211)
==22577==    by 0x312A628DE1: g_signal_emit (gsignal.c:3356)
==22577==    by 0x3D19AAB62B: g_settings_real_change_event (gsettings.c:288)
==22577==    by 0x3A0A405ED7: ffi_call_unix64 (in /usr/lib64/libffi.so.5.0.10)
==22577==    by 0x3A0A4058DF: ffi_call (in /usr/lib64/libffi.so.5.0.10)
==22577==    by 0x312A6108FA: g_cclosure_marshal_generic_va (gclosure.c:1550)
==22577==    by 0x312A60FBD6: _g_closure_invoke_va (gclosure.c:840)
==22577==    by 0x312A6283A7: g_signal_emit_valist (gsignal.c:3211)
==22577==    by 0x312A628DE1: g_signal_emit (gsignal.c:3356)
==22577==    by 0x3D19AAC117: settings_backend_path_changed (gsettings.c:363)
==22577==    by 0x3D19AA7D19: g_settings_backend_invoke_closure (gsettingsbackend.c:271)
==22577==    by 0x3129E47A54: g_main_context_dispatch (gmain.c:2715)
==22577==    by 0x3129E47D87: g_main_context_iterate.isra.24 (gmain.c:3290)
==22577==    by 0x3129E48181: g_main_loop_run (gmain.c:3484)
==22577==    by 0x3D1F58DD4C: gtk_main (gtkmain.c:1160)
==22577==    by 0x403018: main (main.c:711)

Comment 32 Fedora Update System 2013-04-03 04:29:19 UTC
evolution-3.6.4-3.fc18 has been pushed to the Fedora 18 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 33 Bojan Smojver 2013-06-26 23:41:21 UTC
I'm going to move this to F-19. I see 311 MB of RES used there.

Comment 34 Milan Crha 2013-06-28 09:11:58 UTC
It's easier to open a new bug report, there is a chance that the 3.6.x has this fixed, while new leaks were added during the 3.8.x development, thus the previous close was actually correct.

In any case, 311 MB doesn't sound that many, it definitely depends on folder sizes, views you entered since evolution's start and so on. Or, do you think some amount is due to memory leaks (reported as definitely lost by valgrind)?

Comment 35 Bojan Smojver 2013-06-28 09:58:45 UTC
(In reply to Milan Crha from comment #34)
> It's easier to open a new bug report, there is a chance that the 3.6.x has
> this fixed, while new leaks were added during the 3.8.x development, thus
> the previous close was actually correct.

OK.

> In any case, 311 MB doesn't sound that many, it definitely depends on folder
> sizes, views you entered since evolution's start and so on. Or, do you think
> some amount is due to memory leaks (reported as definitely lost by valgrind)?

I have lots of folders and lots of mail, so if that is normal behaviour, maybe that it then. I still think that 300 MB just to read mail is a bit excessive, but I guess that how software is these days.

Is there a way to set something in Evo to use less aggressive caching? I mean, surely the program doesn't need to have everything in memory at all time, right?

Comment 36 Milan Crha 2013-07-01 07:43:54 UTC
You cannot limit caching in any way. Basically, when you enter a folder, then each message's summary (the information in message list) is read in memory. There is some flushing mechanism, which removes from memory those unused after 5-10 minutes. This behaviour is not seen if you've defined real Trash and Junk folders, but that's rather a side effect, than a feature.

Comment 37 Bojan Smojver 2013-08-09 07:07:33 UTC
I know this bug has been closed and all, but just wanted to make one final note here.

Currently evolution-3.8.4-2.fc19.x86_64 is using over 1/2 GB of RES memory on my machine. Combine that with gnome-shell (almost 300 MB) and firefox (around 250 MB - 2 tabs open, no addons) and there it is - 1 GB of RES memory for the most mundane of tasks in Gnome: mail and web. And in pretty much light use.

Surely, evo should be able to figure out that some of that stuff is not required and dump it (I haven't received, sent or read any e-mails in the last several hours), the amount of folders and pieces of e-mail notwithstanding. Just a friendly suggestion for future development...


Note You need to log in before you can comment on or make changes to this bug.