Bug 842099 - Leak memory in Evolution
Leak memory in Evolution
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: evolution (Show other bugs)
17
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Matthew Barnes
Fedora Extras Quality Assurance
:
: 844223 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-07-21 23:35 EDT by Mikhail
Modified: 2012-09-18 23:13 EDT (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-09-18 23:13:09 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
proof screenshot (245.77 KB, image/png)
2012-07-21 23:35 EDT, Mikhail
no flags Details
evolution valgrind log (47.54 KB, text/plain)
2012-07-23 22:18 EDT, Mikhail
no flags Details
evolution valgrind log (278.99 KB, text/plain)
2012-07-24 21:48 EDT, Mikhail
no flags Details
valgrind output after updating evolution and e-d-s (322.44 KB, text/plain)
2012-09-06 11:44 EDT, Matt Davey
no flags Details
Updated valgrind leak report, with num-callers=50 (15.73 MB, text/plain)
2012-09-12 07:28 EDT, Matt Davey
no flags Details

  None (edit)
Description Mikhail 2012-07-21 23:35:35 EDT
Created attachment 599563 [details]
proof screenshot

Description of problem:
Earlier I do this, simply could not see because Evolution always crash, the latest patches are allowed to work Evolution much longer, and now emerged that in Evolution leak memory.
Comment 1 Milan Crha 2012-07-23 04:29:39 EDT
Thanks for a bug report. If I rad the screenshot correctly, then it uses 172MB of resident memory. It's not that much. If you can install debuginfo packages of evolution-data-server, evolution and evolution-ews (all of the same version as your currently installed binary packages (rpm -qa | grep evolution | sort), and then run evolution under valgrind (maybe also install) like this:
   $ G_SLICE=always-malloc valgrind --num-callers=50 --leak-check=full \
     evolution &>log.txt
then use evolution for couple minutes (it'll be significantly slower due to all memory checking), then close evolution (it's rather important to close it properly, if it'll crash or something then it can provide inaccurate information) and attach here the resulting log.txt file. Of course, the longer you'll be able to run evolution the better, but it's really significantly slower and uses high CPU as well.

There were also done memory leak fixes in evolution-data-server last week,
within [1]. The bug is against IMAP, but the most leaking part was common to all mail providers. These will be available in 3.4.4+ of evolution-data-server.

[1] https://bugzilla.gnome.org/show_bug.cgi?id=680211
Comment 2 Mikhail 2012-07-23 22:18:32 EDT
Created attachment 599887 [details]
evolution valgrind log
Comment 3 Milan Crha 2012-07-24 08:10:08 EDT
Thanks for the update. The log looks good, it unfortunately doesn't contain the bottom of it - it is supposed to be much larger than 47KB. The complete log usually ends with summary of found issues which looks like this:

==22828== LEAK SUMMARY:
==22828==    definitely lost: 0 bytes in 0 blocks
==22828==    indirectly lost: 0 bytes in 0 blocks
==22828==      possibly lost: 0 bytes in 0 blocks
==22828==    still reachable: 30,193 bytes in 690 blocks
==22828==         suppressed: 0 bytes in 0 blocks

though evolution doesn't have zeros in each "lost" categories.
Comment 4 Milan Crha 2012-07-24 08:12:07 EDT
I forgot to add, feel free to edit your /usr/lib64/valgrind/default.supp (for 64bit version, but if you have 32bit, then it's 'lib' instead of 'lib64') and add to the top these lines:

{
   Skip any wcslen calls
   Memcheck:Addr8
   fun:wcslen
}

which will hide those false positives you've the attached log full of.
Comment 5 Mikhail 2012-07-24 12:51:25 EDT
So how correctly close evolution under valgrind? When I press close button appear window which offers "Force Quit" or "Wait". When I pressed "Force Quit" I get truncated log as already attached here. When I press "Wait" nothing happens. All another evolution controls are not respond.
Comment 6 Mikhail 2012-07-24 21:48:46 EDT
Created attachment 600200 [details]
evolution valgrind log
Comment 7 Milan Crha 2012-07-25 03:57:17 EDT
(In reply to comment #5)
> So how correctly close evolution under valgrind? When I press close button
> appear window which offers "Force Quit" or "Wait". When I pressed "Force
> Quit" I get truncated log as already attached here. When I press "Wait"
> nothing happens. All another evolution controls are not respond.

That's OK, valgrind is doing final counting, which can take its time (even minutes). It's fine as long as the application does something (eats your CPU).

From the last valgrind log, the issues of cmp_array_uids() and ml_sort_uids_by_tree() should not be there, it seems like the folder was updated while message list was generated. This error printing can do the unresponsiveness of evolution, especially with large folders.
Comment 8 Mikhail 2012-07-26 22:52:05 EDT
(In reply to comment #7) 
> That's OK, valgrind is doing final counting, which can take its time (even
> minutes). It's fine as long as the application does something (eats your
> CPU).

How many time I need to wait?? I waited 15 hours!!! (8:00 - 23:00 all day) But evolution still running under valgrind.
Comment 9 Milan Crha 2012-07-27 01:30:05 EDT
Ouch, that's not normal. It takes here not more than two minutes (usually less than one). Did you also edit the /usr/lib64/valgrind/default.supp, as I suggested in comment #4? It will stop reporting about too often false-positive, thus may make valgrind happier.
Comment 10 Mikhail 2012-07-27 02:37:53 EDT
(In reply to comment #9)
> Ouch, that's not normal. It takes here not more than two minutes (usually
> less than one). Did you also edit the /usr/lib64/valgrind/default.supp, as I
> suggested in comment #4? It will stop reporting about too often
> false-positive, thus may make valgrind happier.

Yes, I edit the /usr/lib64/valgrind/default.supp
seems that evolution stuck under valgrind :(
Comment 11 Milan Crha 2012-07-30 04:38:30 EDT
Could you remind me what packages you have installed currently, please? I'm also wondering whether this leaking is new, related to our ews test package, or whether the tests only made evolution with ews running long enough to show these memory leaks. I'm aware of one similar leaking issue fixed recently [1], maybe this is the same issue.

[1] https://bugzilla.gnome.org/show_bug.cgi?id=680211
Comment 12 Mikhail 2012-07-30 05:46:57 EDT
$ rpm -qa | grep evolution | sort
evolution-3.4.3-2.1.fc17.i686
evolution-data-server-3.4.3-1.2.fc17.i686
evolution-data-server-debuginfo-3.4.3-1.2.fc17.i686
evolution-debuginfo-3.4.3-2.1.fc17.i686
evolution-ews-3.4.3-1.14.fc17.i686
evolution-ews-debuginfo-3.4.3-1.14.fc17.i686
evolution-exchange-3.4.3-1.fc17.i686
evolution-exchange-debuginfo-3.4.3-1.fc17.i686
evolution-mapi-3.4.3-5.fc17.i686
evolution-mapi-debuginfo-3.4.3-5.fc17.i686
evolution-NetworkManager-3.4.3-2.1.fc17.i686
Comment 13 Milan Crha 2012-08-01 02:11:10 EDT
*** Bug 844223 has been marked as a duplicate of this bug. ***
Comment 14 Milan Crha 2012-08-01 02:16:06 EDT
Thanks for the update. The evolution-data-server-3.4.3-1.2 doesn't contain a memory leak fix for the above mentioned upstream bug, but I built it in version 3.4.3-1.3 just yesterday, at [1]. Please try with it.

[1] http://koji.fedoraproject.org/koji/taskinfo?taskID=4344998
Comment 15 Matt Davey 2012-08-24 05:42:51 EDT
I've recently upgraded to fedora 17 and have hit OOM with evolution.  I suspect this is the cause as I'm using evolution-data-server-3.4.3-1.fc17.x86_64.

I followed reference [1] above, but didn't see how to download either a binary or src rpm.  Has it expired from the build server?  I'd prefer a binary, or else a repository with a fixed rpm.  When is this likely to show up in fedora-updates?
Comment 16 Milan Crha 2012-08-24 08:01:52 EDT
The 3.4.4 release is in updates-testing currently, and should be in updates in a week or so. You can get it from here:
https://admin.fedoraproject.org/updates/FEDORA-2012-11822
Comment 17 Matt Davey 2012-08-27 07:00:47 EDT
Thanks Milan,

I grabbed the rpms but had problems trying to install them:
vbox-mcdavey 127 ~/Desktop$ sudo rpm -U *rpm
error: Failed dependencies:
	libgconf-2.so.4 is needed by evolution-data-server-3.4.4-2.fc17.i686
	libgdata.so.13 is needed by evolution-data-server-3.4.4-2.fc17.i686
	libgnome-keyring.so.0 is needed by evolution-data-server-3.4.4-2.fc17.i686
	libgoa-1.0.so.0 is needed by evolution-data-server-3.4.4-2.fc17.i686
	libgweather-3.so.0 is needed by evolution-data-server-3.4.4-2.fc17.i686
	libical.so.0 is needed by evolution-data-server-3.4.4-2.fc17.i686
	libicalss.so.0 is needed by evolution-data-server-3.4.4-2.fc17.i686
	libicalvcal.so.0 is needed by evolution-data-server-3.4.4-2.fc17.i686
	liboauth.so.0 is needed by evolution-data-server-3.4.4-2.fc17.i686
	libsoup-2.4.so.1 is needed by evolution-data-server-3.4.4-2.fc17.i686
vbox-mcdavey 128 ~/Desktop$ rpm -qf /usr/lib64/libgconf-2.so.4
GConf2-3.2.5-1.fc17.x86_64

I don't have time to delve into it, so maybe I'll wait for it to show up in updates.
Comment 18 Matt Davey 2012-08-27 07:05:06 EDT
Doh! I'd accidentally downloaded one i686 rpm which should have been x86_64.  After downloading the correct architecture the upgrade went fine.  I'll let you know if I have any issues.  The bug was pretty hard to miss!
Comment 19 Matt Davey 2012-08-30 05:40:11 EDT
This does not seem to have fixed the OOM for me.  Maybe it's a different issue, but I find that Evolution is being killed by oom_killer if I leave it open overnight.  It is retrieving mail from POP3, with an mh backend.  I'm running fc17 x86_64 inside virtualbox with ~1.7GB of RAM.

[451667.489728] Out of memory: Kill process 31947 (evolution) score 690 or sacrifice child
[451667.489734] Killed process 31947 (evolution) total-vm:5190812kB, anon-rss:970864kB, file-rss:1104kB


I'll have a go with valgrind when I get a chance.

vbox-mcdavey 124 ~/work/pm/windows$ rpm -qa | grep evo
evolution-data-server-3.4.4-2.fc17.x86_64
evolution-debuginfo-3.4.4-1.fc17.x86_64
evolution-3.4.4-1.fc17.x86_64
evolution-NetworkManager-3.4.4-1.fc17.x86_64
evolution-data-server-debuginfo-3.4.4-2.fc17.x86_64
Comment 20 Matt Davey 2012-09-06 11:42:59 EDT
Like Mikhail, I had trouble running evolution under valgrind.  My symptoms seem to be that the tasks to update folders on startup never complete.  I found that if I attached gdb to valgrind, interrupted and then quit gdb, then valgrind would exit.  Not quite sure what's going on there, needless to say.

In this fashion I've managed to get some valgrind output where it has quite a lot of 'possibly-lost' messages (see attached).  This was after a fairly short uptime.

Let me know if you need longer valgrind context, or other memcheck options.
Comment 21 Matt Davey 2012-09-06 11:44:15 EDT
Created attachment 610397 [details]
valgrind output after updating evolution and e-d-s
Comment 22 Milan Crha 2012-09-07 02:57:48 EDT
Hmm, the upstream bug from comment #11 is part of 3.4.4 release, which you have installed according to comment #19, thus I guess you found another place. I know that those from vfolder are fixed for 3.6.0, but the change was slightly complicate (the vfolders are rewritten for 3.6.x), thus it couldn't be easily backported. The upstream has not planned any update for 3.4.x too, thus I'm wondering whether it will worth it to spend time on it when 3.6.0 is out within few weeks.

Nonetheless, part of the possibly lost seems to come from gtk3 itself. If you wish, you can rerun the valgrind with --num-callers=50 , to get longer backtrace with better context from where the leak comes.
Comment 23 Matt Davey 2012-09-12 07:28:37 EDT
Created attachment 612068 [details]
Updated valgrind leak report, with num-callers=50
Comment 24 Milan Crha 2012-09-13 04:42:19 EDT
Thanks for the update. I see the MH-format camel provider is leaking the most memory for you, which I just fixed in evolution-data-server for 3.5.92, done
as commit f2613bf. The other possible lost memory from SQLite is most likely caused by not freed stores/folders on evolution's exit, but otherwise I suppose SQLite reuses the memory chunk for the whole life time of the object. The second most leaking place is involving vfolders, which are addressed for 3.6.0 already. I'll build an update for the MH-format provider for Fedora 17.
Comment 25 Fedora Update System 2012-09-13 05:11:31 EDT
evolution-data-server-3.4.4-3.fc17 has been submitted as an update for Fedora 17.
https://admin.fedoraproject.org/updates/evolution-data-server-3.4.4-3.fc17
Comment 26 Fedora Update System 2012-09-17 14:03:05 EDT
Package evolution-data-server-3.4.4-3.fc17:
* should fix your issue,
* was pushed to the Fedora 17 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing evolution-data-server-3.4.4-3.fc17'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-14192/evolution-data-server-3.4.4-3.fc17
then log in and leave karma (feedback).
Comment 27 Matt Davey 2012-09-18 10:54:29 EDT
Thanks for this.  Seems to be working for me, but I've had a spontaneous exit since upgrading.  As a result, I ran evo within gdb.  I now seem to have a lockup, so I'm going to comment on #854017 with my backtrace.

Thanks again.

Matt
Comment 28 Fedora Update System 2012-09-18 23:13:09 EDT
evolution-data-server-3.4.4-3.fc17 has been pushed to the Fedora 17 stable repository.  If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.