Bug 493000

Hmm, I am exactly the same configuration here and cannot reproduce it. Is there something more which could help us to reproduce it here. For example, which filesystem you use for your /home directory?

/home is actually on NFS, but that is the same for both the (i686) laptop and the (x86_64) workstation.

To be sure NFS is not the problem I have moved the profile from $HOME/.thunderbird/ to a local filesystem and added a symlink:

lrwxrwxrwx. 1 olen olen 42 2007-09-17 19:34 .thunderbird/gwagla2i.jobb -> /var/cache/olen/thunderbird/gwagla2i.jobb/

/var/cache is part of / which is LVM with ext3

$ df /var/cache/olen/thunderbird/gwagla2i.jobb/
Filesystem                        1K-blocks      Used Available Use% Mounted on
/dev/mapper/VolGroup00-LogVol00   149248172  35944648 105599768  26% /

When stracing the process I can't see anything special, but please let me know if there is anything else I can do to debug this.

First of all, could we get output of the command

	rpm -qa *xulrun* *thunderbird* *mozilla* *flash* *plugin*

Please also install thunderbird-debuginfo (debuginfo-install is from
yum-utils package).

	debuginfo-install thunderbird

Then run thunderbird and connect to it with gdb by command

gdb --pid=$(pidof thunderbird-bin)

Then make thunderbird work again with command

   (gdb) cont
   
and do whatever you did to make thunderbird freeze. When it happens, you should go back to the gdb and run

	(gdb) thread apply all backtrace

This produces usually many screens of the text. Copy all of them into a text editor and attach the file to the bug as an uncompressed attachment.

We will review this issue again once you've had a chance to attach this information.

Thanks in advance.

Here is the rpm -qa -output. Attaching gdb-output as soon as I have it:

#  rpm -qa *xulrun* *thunderbird* *mozilla* *flash* *plugin*
PackageKit-yum-plugin-0.4.6-0.3.20090324git.fc11.x86_64
plymouth-plugin-spinfinity-0.7.0-0.2009.03.10.2.fc11.x86_64
gstreamer-plugins-base-0.10.22-2.fc11.x86_64
mozilla-filesystem-1.9-4.fc11.x86_64
nspluginwrapper-1.3.0-5.fc11.x86_64
plymouth-plugin-label-0.7.0-0.2009.03.10.2.fc11.x86_64
alsa-plugins-pulseaudio-1.0.18-3.fc11.x86_64
setroubleshoot-plugins-2.0.14-2.fc11.noarch
yum-plugin-refresh-updatesd-1.1.21-2.fc11.noarch
gutenprint-plugin-5.2.3-5.fc11.x86_64
xulrunner-1.9.1-0.11.beta3.fc11.x86_64
xfce4-mailwatch-plugin-1.1.0-3.fc11.x86_64
gstreamer-plugins-good-0.10.14-2.fc11.x86_64
gstreamer-plugins-farsight-0.12.10-2.fc11.x86_64
plymouth-plugin-pulser-0.7.0-0.2009.03.10.2.fc11.x86_64
yum-plugin-fastestmirror-1.1.21-2.fc11.noarch
anaconda-yum-plugins-1.0-4.fc11.noarch
PackageKit-gstreamer-plugin-0.4.6-0.3.20090324git.fc11.x86_64
plymouth-plugin-solar-0.7.0-0.2009.03.10.2.fc11.x86_64
plymouth-plugin-fade-in-0.7.0-0.2009.03.10.2.fc11.x86_64

Sorry, missed the thunderbird-packages from the above list:

thunderbird-3.0-2.1.beta2.fc11.x86_64
thunderbird-debuginfo-3.0-2.1.beta2.fc11.x86_64

Created attachment 337291 [details]
backtrace of frozen thunderbird

Created attachment 337295 [details]
Another backtrace from the next freeze

Hm, seems to be in:

Thread 1 (Thread 0x7f9d20082800 (LWP 27790)):
#0  0x00007f9d096ab010 in IndexOf<unsigned int, nsDefaultComparator<unsigned int, unsigned int> > (comp=<value optimized out>, start=<value optimized out>, item=<value optimized out>, 
    this=<value optimized out>) at ../../../mozilla/dist/include/xpcom/nsTArray.h:393
#1  IndexOf<unsigned int> (comp=<value optimized out>, start=<value optimized out>, item=<value optimized out>, this=<value optimized out>) at ../../../mozilla/dist/include/xpcom/nsTArray.h:407
#2  Contains<unsigned int> (comp=<value optimized out>, start=<value optimized out>, item=<value optimized out>, this=<value optimized out>) at ../../../mozilla/dist/include/xpcom/nsTArray.h:379
#3  nsAutoSyncState::PlaceIntoDownloadQ (comp=<value optimized out>, start=<value optimized out>, item=<value optimized out>, this=<value optimized out>)
    at /usr/src/debug/thunderbird-3.0/mailnews/imap/src/nsAutoSyncState.cpp:165
#4  0x00007f9d0966885f in nsImapMailFolder::HeaderFetchCompleted (this=0x7f9d046f8c00, aProtocol=0x7f9cad8b8000) at /usr/src/debug/thunderbird-3.0/mailnews/imap/src/nsImapMailFolder.cpp:5181
#5  0x00007f9d1f34b46d in NS_InvokeByIndex_P (that=0x4efcd, methodIndex=2773706040, paramCount=31457288, params=0x7f9d02051e78)
    at /usr/src/debug/thunderbird-3.0/mozilla/xpcom/reflect/xptcall/src/md/unix/xptcinvoke_x86_64_linux.cpp:208
#6  0x00007f9d1f343de3 in nsProxyObjectCallInfo::Run (this=0x7f9caceb4640) at /usr/src/debug/thunderbird-3.0/mozilla/xpcom/proxy/src/nsProxyEvent.cpp:181
#7  0x00007f9d1f33fab1 in nsThread::ProcessNextEvent (this=0x7f9d1c54c280, mayWait=1, result=0x7fff280b858c) at /usr/src/debug/thunderbird-3.0/mozilla/xpcom/threads/nsThread.cpp:510
#8  0x00007f9d1f310984 in NS_ProcessNextEvent_P (thread=0x4efcd, mayWait=-1521261256) at nsThreadUtils.cpp:227
#9  0x00007f9d0dbc75c9 in nsBaseAppShell::Run (this=0x7f9d07d6ac40) at /usr/src/debug/thunderbird-3.0/mozilla/widget/src/xpwidgets/nsBaseAppShell.cpp:170
#10 0x00007f9d0cf308a2 in nsAppStartup::Run (this=0x7f9d07c5f900) at /usr/src/debug/thunderbird-3.0/mozilla/toolkit/components/startup/src/nsAppStartup.cpp:192
#11 0x00007f9d1f7b25ca in XRE_main (argc=<value optimized out>, argv=<value optimized out>, aAppData=<value optimized out>) at /usr/src/debug/thunderbird-3.0/mozilla/toolkit/xre/nsAppRunner.cpp:3279
#12 0x00000000004019bc in main (argc=1, argv=0x7fff280b8dc8) at /usr/src/debug/thunderbird-3.0/mail/app/nsMailApp.cpp:103

So it looks like issue with nsImapMailFolder::HeaderFetchCompleted() from nsAutoSyncState. We may find an upstream bug for that or file a new one.

One thing I forgot to mention is that i use ssl/tls on the IMAP accounts.
It might not be related at all, but might be worth mentioning if it has to do with fetching the headers.

Pretty much the same situation here and I see the same problem.

When this happens th memory use of thunderbird shoots up.  After startup it's about 650M.  During the 100% CPU time it shoots up to 1.6G, to go down after that.

Hmm, could it be related to https://bugzilla.mozilla.org/show_bug.cgi?id=472019 ?

(In reply to comment #11)
> Hmm, could it be related to https://bugzilla.mozilla.org/show_bug.cgi?id=472019
> ?  

Doesn't look like it.  This is no crash.  My guess is that it's just periodically doing unnecessarily much work.  Copying things around into freshly allocated memory etc.

Haven't there been any changes in the upstream thunderbird so that we could just have a new build from cvs?  This bug is really crippling.  Every 30 secs of writing is followed by 2 to 2.5 minutes of freezing.

I'm just installing beta2 to my 64-bit box so let's see if I can catch it there...

I am running rawhide on a x86_64 box. The version below is what I am using. This is bad enough that I think that the package should be downgraded to 2.0.x. One error I got was a this script is taking too long for folderpane.js. My e-mail client is now dependent on javascript code?!?!?

thunderbird-3.0-2.1.beta2.fc11.x86_64

What is up with the tabs? There doesn't seem to be a way to open a new one, but yet I have a tab bar.

  I tried switching to the i586 version below on my x86_64 box to see if it would help. It seems to be just as bad. I just switch folders or sometimes scroll in the current folder and the whole interface will freeze for a number of seconds while it eats 100% of a core.

thunderbird-3.0-2.1.beta2.fc11.i586

  I turned off the sync feature on all my accounts. Just turned the feature off and on causes freezes.

Edit | Account Settings | "Account" | Syncing & Disk Space | Keep messages for this account on this computer

  Probably related to this upstream bug.

https://bugzilla.mozilla.org/show_bug.cgi?id=442674

I started to see this again with sync off. I started getting a crazy amount of spam, and ended up with 300,000+ messages in one folder. I received the same folderpane.js script error. Thunderbird also managed to cause a OOM event.

Out of memory: kill process 16440 (thunderbird-bin) score 793324 or a child
Killed process 16440 (thunderbird-bin)

This bug appears to have been reported against 'rawhide' during the Fedora 11 development cycle.
Changing version to '11'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

thunderbird-3.0-2.3.beta2.fc11.x86_64 freezes sometimes here too (F11 + updates)

The problem seems to be fixed with
thunderbird-3.0-2.4.b3pre.hg.6a6386c16e98.fc11.x86_64 from koji.

Have been running that for a few weeks now without any trouble.

Rgds.

That version from koji doesn't help here (and has no locales).

Thunderbird still constantly tortures my hard disk (and I have the synchronization turned off).

Thunderbird 3 on F11 is currently just unusable (at least with imap).

It seems the hard disk activitiy with beta3pre was because the beta3pre has fetched and stored about 1 GB from my imap-server for which it has needed some time (because it seems to have lost a large part of the headers during upgrade to 3.0b2 because of bug 505884).

Actual the beta3pre behaves like it should, just that 1 GB in .thunderbird isn't what I'm using IMAP for (I have synchronization turned off).

Thunderbird 2 uses less than 300 MB in .thunderbird, which is just enough for the headers of about 4.5 GB mails which my imap-server actual has stored (uncompressed).

I am confirming high usage of CPU usage. It appears to be from several processes running. Further, if it sits for several hours, it has effectively chewed through 1GB of RAM, and begins swapping. I do have a massive IMAP account, with tens of thousands of messages taking up roughly 2GB. If you don't have enough info, I can provide any debugging data that would provide useful. Currently running TB 3 Beta 2.

I also noted the same 100% CPU problem after just upgrading to F11. However, it seems that the CPU usage is related to the number of mails in the folder and/or the number of unread mails.

After going through all folders and marking some 50000 messages as read and splitting some large folders, I now see the CPU usage being reduced significantly to a point where thunderbird is usable again. I still see spikes of CPU usage, which are probably from the inbox.

thunderbird-3.0-2.3.beta2.fc11.x86_64 spins and "freezes" with this ltrace output (too many same rolling lines, Thunderbird is in R state, eating 100% CPU, impossible to switch focus on it):

pthread_mutex_unlock(0x7fd3e5b8f040, 0xf0000000, 0x7fd3c7ac6000, 27, 0xf8000000)                 = 0
pthread_mutex_lock(0x7fd3e5b8f040, 88, 6, 0x7fff1a76a004, 88)                                    = 0
pthread_mutex_unlock(0x7fd3e5b8f040, 0xe0000000, 0x7fd3c7ac6000, 28, 0xf0000000)                 = 0
pthread_mutex_lock(0x7fd3e5b8f040, 88, 6, 4, 0)                                                  = 0
pthread_mutex_unlock(0x7fd3e5b8f040, 0xc0000000, 0x7fd3c7ac6000, 29, 0xe0000000)                 = 0
pthread_mutex_lock(0x7fd3e5b8f040, 88, 6, 0x7fff1a76a004, 88)                                    = 0
pthread_mutex_unlock(0x7fd3e5b8f040, 0x80000000, 0x7fd3c7ac6000, 30, 0xc0000000)                 = 0
pthread_mutex_lock(0x7fd3e5b8f040, 88, 6, 4, 0)                                                  = 0
pthread_mutex_unlock(0x7fd3e5b8f040, 0, 0x7fd3c7ac6000, 31, 0x80000000)                          = 0
pthread_mutex_lock(0x7fd3e5b8f040, 88, 6, 0x7fff1a76a004, 88)                                    = 0
pthread_mutex_unlock(0x7fd3e5b8f040, 1022, 0x7fd3c7ac6000, 0, 1023)                              = 0
pthread_mutex_lock(0x7fd3e5b8f040, 88, 6, 4, 0)                                                  = 0
pthread_mutex_unlock(0x7fd3e5b8f040, 1020, 0x7fd3c7ac6000, 1, 1022)                              = 0
pthread_mutex_lock(0x7fd3e5b8f040, 88, 6, 0x7fff1a76a004, 88)                                    = 0

*** Bug 513783 has been marked as a duplicate of this bug. ***

I don't know if my symptoms are related but whilst TB is running CPU usage seems OK, but as soon as I close TB then there is a process "thunderbird-bin" that sits there consuming around 56% of the CPU. This is consistent - I connect to an external imap server and have lightning installed but I doubt if this is relevant.

This is for an up to date F11 system running on an NC10 netbook.

I have thunderbird-3.0-2.6.b3.fc11.x86_64 since Aug 17 (regular F11 update), but still have troubles with freezing. I have GNOME with standard settings.

I am running TB 3.0beta4 for the i386 version from updates-testing in F11 and this problem has been resolved for me with this build.  However I am aware that there are significant problems with the x86_64 version - I understand that there is no official x86_64 version due to the developers believing that there are unresolved problems that will make that version not work. I wonder if installing the 32 bit version of TB, and running in an x86_64 OS will see the problems disappear?

With 64bit up-to-date Fedora 11 I tryed i586 package (thunderbird-3.0-2.6.b3.fc11.i586). If freezes like 64bit version.

the 64bit version of beta4 still freezes quite often.. (actually seems to be more often than in beta3).

I have to let it sit there for sometimes up to 5 minutes before I'll regain control of the application.  (this is a quad core 2.6Ghz phenom w/ 4GB of ram)

I see this with:

  thunderbird-3.0.4-1.fc12.x86_64

so I'm looking here to see that it's being tracked properly.  I'm not sure we've gotten past symptoms to cause here yet.

upstream 472019 is marked as the external bug but that's about crashes.

Nathan suggests upstream 442674 as relevant which is marked incomplete due to lack of feedback.

I did a quick search on some of the symbols in Martin's backtrace but didn't turn up likely candidates.

I'll try attaching with gdb next time I see the symptom.

So, on mine anyway, it was thrashing .msf files which are mork databases.  I filed bmo 561272 on it.

One thing that helped was rebuilding index files.  Roughly, if your thunderbird is hung for the same reason:

  strace -p `pidof thunderbird-bin`

you'll see:

  lseek(119, 235773952, SEEK_SET)         = 235773952
  read(119, "^90=c9aab)(^92=0)(^91=0)(^93=0)("..., 4096) = 4096
  read(119, ")(^92=0)(^91=0)(^93=0)(^94=0)]\n["..., 4096) = 4096

ad. nauseum, but ctrl-c it and note that first number (file descriptor) and:

  ls -l /proc/`pidof thunderbird-bin`/fd/119

which will show you which folder's .msf is being thrashed.  When the UI comes back you can right-click the folder's Properties... and click 'Rebuild Index'.  I had one .msf go from 249MB to 2K.  Mork databases are apparently all loaded into memory and parsed before they're used.  If they're big ... ouch.  Efforts to replace Mork in Thunderbird appear to have begun in the last century.  Originally talked about for a 3.0 conversion to sqllite, it appears nothing's been done since 2008 on it so we might be with this problem a while (if my problem is the same as others').

This message is a reminder that Fedora 11 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 11.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '11'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 11's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 11 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Reporter/maintainer: please bump to f12.

Fedora 11 changed to end-of-life (EOL) status on 2010-06-25. Fedora 11 is 
no longer maintained, which means that it will not receive any further 
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.

(In reply to comment #37)
> Reporter/maintainer: please bump to f12.

I can confirm that this is still very much an issue in F12. Nothing has changed. Please bump to F12.

Could you retest with upstream binary of Thunderbird 3.1.1 [1] or with thunderbird 3.1.1 from F13?

[1] ftp://ftp.mozilla.org/pub/thunderbird/releases/3.1.1/linux-i686/en-US/

The closest I can find to 3.1.1 in the Fedora distro is

http://www.mirrorservice.org/sites/fedora.redhat.com/development/rawhide/i386/os/Packages/thunderbird-3.1-1.fc14.i686.rpm

Using this would require upgrading a production machine to Rawhide.

Do you know of a build for F13?

There is a thunderbird 3.1.1 build in F13 updates.

http://download.fedora.redhat.com/pub/fedora/linux/updates/13/i386/thunderbird-3.1.1-1.fc13.i686.rpm
(and lightning)
http://download.fedora.redhat.com/pub/fedora/linux/updates/13/i386/thunderbird-lightning-1.0-0.26.b2pre.fc13.i686.rpm

I will be testing this later tonight once I reboot my Mac into F13 again.

I is still periodically "hanging" and consuming 100% of the CPU. (one cpu in my case:) ).    Right now installing the debuginfo packages and will dump a backtrace when it is "hung"

Created attachment 436298 [details]
backtrace during a freeze

Everytime I look at it it's always stuck in the libmork.

(In reply to comment #43)
> There is a thunderbird 3.1.1 build in F13 updates.
> 
> http://download.fedora.redhat.com/pub/fedora/linux/updates/13/i386/thunderbird-3.1.1-1.fc13.i686.rpm

Excellent .. got it, added debug, run, froze, ^C, bt ... and the winner is ??!

Yup, Mork again.

Backtrace to follow ..

Created attachment 436336 [details]
gdb backtrace of thunderbird freeze

I noted the filename /home/meredith/.thunderbird/r6flqs41.default/ImapMail/imap.ch.anvil.org/Junk.msf as part of the trace. This file is about 100M.

Andrew, could you try right-click... Properties... Rebuild Folder on your Junk box?  See how big the .msf is afterwards?

After rebuild it is now 110M

This message is a reminder that Fedora 12 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 12.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '12'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 12's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 12 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

This still occurs with at least Fedora 13 builds of thunderbird

Same issue on Windows XP with Thunderbird 3.1.7 and IMAP servers, whole thing hangs for half a minute, sometimes minutes just to load a single message and apparently 'randomly' as well, guess when its refreshing folders

This message is a reminder that Fedora 13 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 13.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '13'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 13's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 13 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

This still occurs in Fedora 15.

Fedora 13 changed to end-of-life (EOL) status on 2011-06-25. Fedora 13 is 
no longer maintained, which means that it will not receive any further 
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.

This is still occurring in Fedora 15 and I can not change the release # in this bug.

I sent a message to Jan and Ola asking for a version update and re-open.

I have not had the problem for a while.

What I did was to delete everything in .thunderbird and restart from scratch.
The I turned off local download of all folders (sync and storage), turned off "check for new mail" and turned off adaptive junk check.

For me, these settings are ok, as my imap-servers supports idle (so I don't need to pull mail), I have spamassasin in front, so I dont care about thunderbirds spamchecking and I don't normally need my email while offline (IE I am "never" offline).

I guess for others, turning off all this is not as easy to live with...

regarding the ssl comment in comment#9 

   (https://bugzilla.redhat.com/show_bug.cgi?id=493000#c9)

note a possibly related bug:

   https://bugzilla.redhat.com/show_bug.cgi?id=488066

talks about competition for single ssl thread

This still happens with thunderbird 6 on F15 x86_64, same setup (IMAP+SSL). Every once in a while when it says "Downloading message..." on the status bar CPU usage goes through the roof. Funny thing is that it makes X11 process to overload the CPU, and thunderbird itself appears on 2nd place on top's list.

IMHO it would be nice if someone could update this issue's title to reflect the fact that it still happens with latest version.

Can someone link bugzilla.Mozilla.org bug 11050

https://bugzilla.mozilla.org/show_bug.cgi?id=11050

As this seems to be the "answer" to resolving this very annoying issue.

Is this issue still seen lately? Mozilla has been trying to remove morkDB for recent Thunderbird and this could also fix this issue.

Upstream bug is still unresolved.  I don't see anything in the release notes for Thunderbird 11:

https://www.mozilla.org/en-US/thunderbird/11.0/releasenotes/buglist.html

This message is a notice that Fedora 15 is now at end of life. Fedora
has stopped maintaining and issuing updates for Fedora 15. It is
Fedora's policy to close all bug reports from releases that are no
longer maintained. At this time, all open bugs with a Fedora 'version'
of '15' have been closed as WONTFIX.

(Please note: Our normal process is to give advanced warning of this
occurring, but we forgot to do that. A thousand apologies.)

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, feel free to reopen
this bug and simply change the 'version' to a later Fedora version.

Bug Reporter: Thank you for reporting this issue and we are sorry that
we were unable to fix it before Fedora 15 reached end of life. If you
would still like to see this bug fixed and are able to reproduce it
against a later version of Fedora, you are encouraged to click on
"Clone This Bug" (top right of this page) and open it against that
version of Fedora.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

The process we are following is described here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

I have this issue in Fedora 17 too. Bz853638.

On big folders (in my case it was 450 000 of messages) thunderbird eat CPU 100% of 1 core and memory about 1-2Gb and freezes periodically.

I can fix it only by delete some messages from that folder.

Pavel, what version Thunderbird?

There was a mime performance issue resolved in TB15 or TB16 iirc.

I'm still on tb15, due to a calendaring bug in newer versions, but there a great way to exaggerate this problem is to make a saved search on one of the very large folders.  Perhaps it needs to do the search n times for each selector of the saved search?  I can have saved search windows take 3 minutes to open on a decent quad-core machine with mirrored SATA disks with the result set only in the hundreds of messages (perhaps 5 criteria in the search).  The folder it's searching is very large, though.

Ыщкнб forgot mention versions:
$ rpm -qa *xulrun* *thunderbird* *mozilla* *flash*
thunderbird-16.0.2-1.fc17.x86_64
xulrunner-16.0.2-1.fc17.x86_64
mozilla-filesystem-1.9-7.fc17.x86_64
thunderbird-debuginfo-16.0.2-1.fc17.x86_64
flash-plugin-11.2.202.236-1.R.x86_64
flash-plugin-kde-11.2.202.236-1.R.x86_64

This message is a reminder that Fedora 17 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 17. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '17'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 17's end of life.

Bug Reporter:  Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 17 is end of life. If you 
would still like  to see this bug fixed and are able to reproduce it 
against a later version  of Fedora, you are encouraged  change the 
'version' to a later Fedora version prior to Fedora 17's end of life.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Fedora 17 changed to end-of-life (EOL) status on 2013-07-30. Fedora 17 is 
no longer maintained, which means that it will not receive any further 
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.

I have symptoms of this bug. Fedora 20

During the 100% CPU utilization the strace shows read() and lseek() on various *.msf files.

I can confirm that having large mail folders will mean that every now and then TBird just freezes for a minute or so. During this time, one of the CPUs is at 100%. When I have previously examined this I also found that the activity was around the msf files for the big folders.

I dropped the ball on working on this, but if anybody wants to debug this, BMO had a suggested path:

https://bugzilla.mozilla.org/show_bug.cgi?id=561272

That does not work (unfortunately).

This bug is so painful to me, that I'm willing to pay money to whoever resolve this issue:
https://www.bountysource.com/issues/3558744-imap-temporary-100-cpu-and-freezes-with-thunderbird-3

Just to clarify, I looked at the BMO bug again, and Miroslav was experieincing this problem with Global search/indexing disabled. So, something else is pounding the msf files, apparently. 

I only found an article about firefox, but this works for thundebird too, to run under gdb:

$ thunderbird -g -d gdb
(gdb)  handle SIG33 noprint nostop
(gdb)  run

(with xulrunner and thunderbird debuginfo installed, obviously)

I'll try it on my desktop which exibits this and see if I can get a useful backtrace.

For the record. The size of my
  ~/.thunderbird/2nn7z7bh.default/Mail
is 7GB. And probably include milions of emails in dozens folders. So this can give you idea how big the directory have to be to have those symptomps nicely visibile.

please provide the details in https://bugzilla.mozilla.org/show_bug.cgi?id=561272

thanks

(In reply to Andrew Meredith from comment #47)
> Created attachment 436336 [details]
> gdb backtrace of thunderbird freeze
> 
> I noted the filename
> /home/meredith/.thunderbird/r6flqs41.default/ImapMail/imap.ch.anvil.org/Junk.
> msf as part of the trace. This file is about 100M.

(In reply to Andrew Meredith from comment #49)
> After rebuild it is now 110M

(In reply to Andrew Meredith from comment #72)
> I can confirm that having large mail folders will mean that every now and
> then TBird just freezes for a minute or so. During this time, one of the
> CPUs is at 100%. When I have previously examined this I also found that the
> activity was around the msf files for the big folders.

Andrew,

A 100M size .msf suggests the corresponding folder is ~1GB.

1GB would be very large for a junk folder, which will like cause performance issues.  (Dont' know why junk, and trash, are worse than other large folders - perhaps because they are high activity?)

Why are you letting the junk folder get so big?
And what happens if you empty your junk folder?  (right+click on junk, empty)

I was purely using the Junk folder as an example. It was so big because it had been a while since I last reviewed it and pushed the genuine spam through spamassassin.

Compacting made no difference to either the msf size or the occurances of regular stalls in TBird itself. After reducing it in size and doing another compact, the stalls ceased and the msf was minute.

This was all to further illustrate that TBird stalls for a disproportionately long time with bigger folders.

This message is a reminder that Fedora 20 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 20. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '20'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 20 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

This is still valid in Fedora 22.
And I increased the bounty for this bug resolving to $150.
  https://www.bountysource.com/issues/3558744-imap-temporary-100-cpu-and-freezes-with-thunderbird-3

(In reply to Miroslav Suchý from comment #82)
> This is still valid in Fedora 22.
> And I increased the bounty for this bug resolving to $150.
>  
> https://www.bountysource.com/issues/3558744-imap-temporary-100-cpu-and-
> freezes-with-thunderbird-3

Very generous Miroslav. I hope this helps get a fix. I'd like to offer some suggestions to increase your odds of success:

#1 you should mention this in an appropriate bug at bugzilla.mozilla.org where it will get many more eyes.

#2 if you don't find an appropriate bug at bugzilla.mozilla.org - meaning one that matches your issue precisely - then you should create one

the key to finding a precise matching bug, if one exists, is narrowing the list of symptoms, and determining whether the list of symptoms is just one item, or a combination.  Note - suggest you disable automatic compact first anything, in tools | options | advanced | network and disk.  To name some possibilities:
- high memory (over 500mb, if so how much)
- high cpu
- global index off or on
- autosync (syncrhonize messages) off or on
- IDLE enabled or not (account settings, server settings, advanced)
- situational:
 * only during startup
 * only when manually or automatically getting new messages
 * only when clicking a folder

The list of candidates bugs for the most part have pretty well narrowed summary titles - https://bugzilla.mozilla.org/buglist.cgi?bug_id=%20383718%20&bug_id_type=nowords&keywords=perf%2C&f1=short_desc&keywords_type=allwords&o3=anywordssubstr&list_id=12308919&bug_severity=blocker&bug_severity=critical&bug_severity=major&o1=nowordssubstr&resolution=---&classification=Client%20Software&classification=Components&query_format=advanced&f3=short_desc&f2=OP&component=Backend&component=Database&component=Folder%20and%20Message%20Lists&component=General&component=Networking&component=Networking%A%20IMAP&product=MailNews%20Core&product=Thunderbird

As stated in #78 the relevant report in Mozilla tracker is:
  https://bugzilla.mozilla.org/show_bug.cgi?id=561272

(In reply to Miroslav Suchý from comment #84)
> As stated in #78 the relevant report in Mozilla tracker is:
>   https://bugzilla.mozilla.org/show_bug.cgi?id=561272

Miroslav 

Yes, that was what *I* suggested a year ago, so I was asking again because I may be wrong. But I see your comment 71 "shows read() and lseek() on various *.msf files.", so perhaps your issue is a solid fit to Bills. If you are satisfied they are the same then let's continue in bug 561272.

NOTE - there's still more we need to know about Bill's issue, and yours. And you do not seem to be answering questions like https://bugzilla.mozilla.org/show_bug.cgi?id=561272#c13  It may be a hard and long process without your participation.  

I'm hoping the problem will turn out to be autosync related. Regardless, a recap from you and Bill on the following points would be very helpful to anyone who would be interested in your bounty.  So in a single comment in bug 561272 please give the following information:

- high memory? (over 500mb, if so how much)
- high cpu?
- fails with global index off?
- fails with autosync (syncrhonize messages) off? 
- fails with IDLE disabled? (account settings, server settings, advanced)
- fails with compact disabled?  (tools | options | advanced | network and disk)
- reliable network connection i.e. is not failing/is solid during your high CPU usage?
- what is the name and version of the imap server?
- maximum nunmber of messages in folder? (size is mentioned in bug 561272 comment 13, but not a message count)
- your list of addons?
- a profile url from https://developer.mozilla.org/en-US/docs/Mozilla/Performance/Reporting_a_Thunderbird_Performance_Problem_with_G
- if it fails because of imap or autosync activity then a log of the activity will help https://wiki.mozilla.org/MailNews:Logging
Anything else you think is relevant.

Lastly, your bounty is not going to be an incentive in the BMO bug until you mention it there.

p.s. Bill's comment 35 is a very cool diagnostic.  But PLEASE (anyone) do not do a repair on a folder without first copying both the folder and the .msf file to a safe location outside the profile

Please see:
  https://bugzilla.mozilla.org/show_bug.cgi?id=561272#c19
for solution.

I.e. set
  mail.db.max_open to 1000

From my POV this is resolved by Wayne Mery solution, and you can redeem the bounty for this bug (not sure what the process is as I never redeemed bounty. I am willing to assist if you have problem with redeem.).

(In reply to Miroslav Suchý from comment #87)
> Please see:
>   https://bugzilla.mozilla.org/show_bug.cgi?id=561272#c19
> for solution.
> 
> I.e. set
>   mail.db.max_open to 1000
> 
> From my POV this is resolved by Wayne Mery solution, and you can redeem the
> bounty for this bug (not sure what the process is as I never redeemed
> bounty. I am willing to assist if you have problem with redeem.).

It is pleasing that you have good results. But I do not think this bug should be closed until the reporter and/or a few others who have previously commented are able to confirm the problem is gone while version 38 with default settings.

Thunderbird 38 is currently in Fedora stable so I gave it try.
I tried to reset mail.db.max_open to default value. While the lags are not so big, after while I still found some. It was like 5 secons vs. 50 seconds as when I originally reported this issue.
After few hours I set mail.db.max_open back to 1000 and even those 5 secs lags are gone.

(In reply to Miroslav Suchý from comment #89)
> Thunderbird 38 is currently in Fedora stable so I gave it try.
> I tried to reset mail.db.max_open to default value. While the lags are not
> so big, after while I still found some. It was like 5 secons vs. 50 seconds
> as when I originally reported this issue.
> After few hours I set mail.db.max_open back to 1000 and even those 5 secs
> lags are gone.

Miroslav, 
What happens if you set mail.db.max_open to something more modest, like 60?

Modest value works as well. As long as it is bigger then number of my folders (something between 100 and 200 in my case).
When I use smaller value than my number of folders (e.g. 60 I see performance decreese).

(In reply to Miroslav Suchý from comment #91)
> Modest value works as well. As long as it is bigger then number of my
> folders (something between 100 and 200 in my case).
> When I use smaller value than my number of folders (e.g. 60 I see
> performance decreese).

How many accounts do you have? pop or imap?
Do you use any server side filters or folder properties set to check for new mail (other than Inbox)?

What extensions are installed? (listed at help | troubleshooting)
Do you see the problem with Thunderbird started in safe mode?

(In reply to Wayne Mery (:wsmwk) from comment #92)
> How many accounts do you have? pop or imap?

2 IMAP accounts
1 news (UUTP)
1 RSS

> Do you use any server side filters or folder properties set to check for new
> mail (other than Inbox)?

Yes, I have 14 folders on server (beside those basics) and I have 14 servers sides filters which move mails to those folders.
All remaining folders are local and are handled by Thunderbird filters.

> What extensions are installed? (listed at help | troubleshooting)

Awesome LdapInfoShow
Enigmail
Lightning
MailRedirect
QuoteCollapse

> Do you see the problem with Thunderbird started in safe mode?

Yes.

I believe that the bounty
  https://www.bountysource.com/issues/3558744-imap-temporary-100-cpu-and-freezes-with-thunderbird-3
still has not been picked up. Guys, that is $150 in limbo. Pick it up. This has been resolved and someone deserves the money.