Bug 559312

Summary: thunderbird corrupts mail indices (Old mail marked as new)
Product: [Fedora] Fedora Reporter: Ralf Corsepius <rc040203>
Component: thunderbirdAssignee: Gecko Maintainer <gecko-bugs-nobody>
Status: CLOSED UPSTREAM QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 12CC: dash, emcnabb, gecko-bugs-nobody, jeff, knoel, mcepl, mhlavink, mikeb, misek, mjc, sgallagh, sjoerd, steved, twoerner, xavier
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-01-28 14:30:00 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Mozilla BZ 540554 patch none

Description Ralf Corsepius 2010-01-27 17:53:17 UTC
Description of problem:

Since thunderbird was upgraded to 3.0.1, I am once again experiencing thunderbird to produce corrupt mail indices on IMAP (dovecot) folders.

Symptoms: It marks years old read mails as new.


This had been a severe problem with 3.pre's release, had somewhat settled with 3.0.0, but has reached a new negative climax with 3.0.1.

In short: To me, this thunderbird-3.0.1 is plain simply unusable.

None of the "used-to-work" workarounds ("rebuild index", run evolution, switch of folder synchronisation, rebooting, "compact folder") work.

Version-Release number of selected component (if applicable):
thunderbird-3.0.1-1.fc12.x86_64


How reproducible:
Semi-deterministic - Today it has happeend several dozens of times.

 
Actual results:
Old/read mail being marked as new (often 1000s of them at once).
<Right click on corrupt folder>->Properties->Rebuild index
seems to temporarily restore the a folders broken index, ... some hours later it's broken again.

Expected results:
Function.

Additional info:
I am not 100% sure if it's really thunderbird, who is to blame.
It would not want to exclude the culprit is another component being involved. Amongst my suspects are: nfs (nfs mounted $HOME), dovecot (running on the same machine as the thunderbird exposing the problems), the kernel, dbus (each dbus related update seems to kill my system), abrt (a source of problems of its own) ...

Comment 1 Steve Dickson 2010-01-28 00:32:37 UTC
This is happen on my laptop where NFS is not involved it all....

Also not only are read emails being marked not read (in my case that's
thousands of emails in a very large number of folders), it also destroys 
all the tags as well.... This is *data corruption* that is costing *everyone*
that is using the version 3.0.1 (at Red Hat is a very large number of 
people) a huge amount of time and pain...

At the of the day somebody F-ed something up in 3.0.1 so PLEASE PLEASE
fix the problem today or back out the patch that is causing the problem
and push out the new release.... because this is a truly truly painful bug


Question: How the heck does such a destructive bug get by your 
testing??? Or am I making assumption you actually do test on
minor release like this?

Comment 2 Ralf Corsepius 2010-01-28 04:41:16 UTC
(In reply to comment #1)
> This is happen on my laptop where NFS is not involved it all....
In my case it's automounted homes, with the account suffering from this issue (my personal one) being located on the same machine as my dovecot is.

I.e. on the machine exposing this issue, my account's HOME actually is "bind-mounted". I know, several piece of Gnome/Gtk are unable to handle this setup properly, may-be thunderbird also isn't?

> Also not only are read emails being marked not read (in my case that's
> thousands of emails in a very large number of folders), it also destroys 
> all the tags as well....
Yes, I am occasionally experiencing this as well. 

I am also suspecting me to be loosing mail. However due to the nature of this problem, this is hard to prove - You can't complain about having lost mail, because you've never seen it. You only occasionally experience complaints about you not responding to email, and you not being able to find it when searching for them.

FWIW: So far, I have only experienced these issues on x86_64, but not on i386.
This could be a coincidents of events (My primary machine is x86_64), but could also indicate 64bit incompatibilities.

> At the of the day somebody F-ed something up in 3.0.1 so PLEASE PLEASE
> fix the problem today or back out the patch that is causing the problem
> and push out the new release.... because this is a truly truly painful bug
Fully agreed.

To emphasize how unpleasant this bug is: It hit me 3 times within the last hour.

Comment 3 Jeffrey C. Ollie 2010-01-28 13:37:05 UTC
(In reply to comment #1)
> 
> Also not only are read emails being marked not read (in my case that's
> thousands of emails in a very large number of folders), it also destroys 
> all the tags as well.... This is *data corruption* that is costing *everyone*
> that is using the version 3.0.1 (at Red Hat is a very large number of 
> people) a huge amount of time and pain...

Um, NOT everyone... I use Thunderbird daily to read my work email and I have not seen these issues.

> At the of the day somebody F-ed something up in 3.0.1 so PLEASE PLEASE
> fix the problem today or back out the patch that is causing the problem
> and push out the new release.... because this is a truly truly painful bug
> 
> Question: How the heck does such a destructive bug get by your 
> testing??? Or am I making assumption you actually do test on
> minor release like this?    

Maybe because not everyone experiences the bug?  I'm sure that this is a very painful problem for you but please remain civil and don't over-generalize.

Comment 4 Matěj Cepl 2010-01-28 14:21:23 UTC
(In reply to comment #1)
> Question: How the heck does such a destructive bug get by your 
> testing??? Or am I making assumption you actually do test on
> minor release like this?    

Because we haven't seen it? To me this happened quite rarely (one or two emails every other day or so) and yes I have also local IMAP dovecot. Asking dovecot maintainer to join this party.

Comment 5 Matěj Cepl 2010-01-28 14:22:07 UTC
and there is no NFS at sight for me

Comment 6 Michal Hlavinka 2010-01-28 14:28:14 UTC
Hi, this was reported also on dovecot mailing list, the conclusion is
thunderbird has broken CONDSTOR support, you should be able to workaround this
by turning condstor off.

> Seems everyone is starting to notice this now. :)
> Yes, there are several flaws and bug reports open with Mozilla on broken 
> behavior with TB 3.0 and TB 3.0.1. Currently the easiest fix is to turn 
> off CONDSTOR either server side or in TB itself. There is also a TB 
> 3.0.2pre nightly available that seems to be working for most people.

upstream reports:
> https://bugzilla.mozilla.org/show_bug.cgi?id=540554
> https://bugzilla.mozilla.org/show_bug.cgi?id=524902
> https://bugzilla.mozilla.org/show_bug.cgi?id=517461

and yes, as Matej has said, it usually reproduce itself after a quite long time, so it's hard to spot this during testing.

Comment 7 Matěj Cepl 2010-01-28 14:30:00 UTC
We filed this bug in the upstream database (https://bugzilla.mozilla.org/show_bug.cgi?id=540554) and believe that it is more appropriate to let it be resolved upstream.

We will continue to track the issue in the centralized upstream bug tracker, and will review any bug fixes that become available for consideration in future updates.

Thank you for the bug report.

Comment 8 Matěj Cepl 2010-01-28 14:38:48 UTC
Before we get 3.1 in Fedora a workaround is to disable mail.server.default.use_condstore option. In Edit/Preferences/Advanced/Config editor change the value of this variable to false.

Comment 9 Steve Dickson 2010-01-28 15:15:59 UTC
> disable mail.server.default.use_condstore option
Just out of curiosity what function or feature am I turning off. 
Also, will I have to turn it back on when the problem is resloved?

Comment 10 Ralf Corsepius 2010-01-28 15:36:14 UTC
(In reply to comment #7)
> We filed this bug in the upstream database
> (https://bugzilla.mozilla.org/show_bug.cgi?id=540554) and believe that it is
> more appropriate to let it be resolved upstream.
> 
> We will continue to track the issue in the centralized upstream bug tracker,
> and will review any bug fixes that become available for consideration in future
> updates.
In comment #8 you provided what you believe to be a bug fix.

=> You should be replacing this broken package Red Hat currently is nagging the community with a package which has your "may-be fix" applied.

You refuse to do - What am I supposed to think of this?

(In reply to comment #3)

> Maybe because not everyone experiences the bug?
I already had been facing this issue with 3.pre (IIRC, it was Macej who closed a similar bug as "CLOSED UPSTREAM" then). It reappeared 5 minutes after having installed TB-3.0.1.

Makes me wonder what you guys are doing.

> I'm sure that this is a very
> painful problem for you but please remain civil and don't over-generalize.    
Actually, I consider this thread to be very civil.
 
Only so far: I feel like being abused as @RH's Thunderbird Guinea pig and @RH's Thunderbird to be poorly maintained.

Comment 11 Matěj Cepl 2010-01-28 15:47:19 UTC
(In reply to comment #10)
> You refuse to do - What am I supposed to think of this?

I don't refuse to do anything ... TB 3.1 is behind the corner, so we will upgrade to it when it is released upstream, which should be soon.

Comment 12 Michael Cronenworth 2010-01-28 16:16:51 UTC
Created attachment 387369 [details]
Mozilla BZ 540554 patch

Matej, F11 and F12 will be getting 3.1? Aren't major FF and TB releases only slated for major Fedora releases? Meaning: TB 3.1 will come only with F13?

I have attached the patch, which is slated for TB 3.0.2. Could you not rebuild a 3.0.1 with this patch and push it out? Thanks.

Comment 13 Ralf Corsepius 2010-01-28 16:25:46 UTC
(In reply to comment #10)
Apologies for having mistyped your name, Matej.

(In reply to comment #11)
> (In reply to comment #10)
> > You refuse to do - What am I supposed to think of this?
> 
> I don't refuse to do anything ...
You refuse to acknowledge and to really fix your bugs.

"FIXED/CLOSED UPSTREAM" and "FIXED RAWHIDE" mean to refuse to fix a distro's package's bugs and continue exposing them to these bugs.

> TB 3.1 is behind the corner, so we will
> upgrade to it when it is released upstream, which should be soon.    

I am sure, we can expect you guys will have better tested this version than 
TB-3pre and TB-3.0.1?

BTW: Your "may-be fix" seems to work for me. At least, unlike before, I haven't experienced a corrupted index since I activated it ca. 15 minutes ago :-)

Now I am still waiting for someone to answer
https://bugzilla.redhat.com/show_bug.cgi?id=559312#9
I would really like to know what I am doing and which consequences I am likely to be facing.

Comment 14 Xavier Bachelot 2010-02-23 20:06:05 UTC
*** Bug 558580 has been marked as a duplicate of this bug. ***

Comment 15 Xavier Bachelot 2010-02-23 20:06:14 UTC
*** Bug 566487 has been marked as a duplicate of this bug. ***

Comment 16 Xavier Bachelot 2010-02-23 20:10:54 UTC
(In reply to comment #12)
> Created an attachment (id=387369) [details]
> Mozilla BZ 540554 patch
> 
> Matej, F11 and F12 will be getting 3.1? Aren't major FF and TB releases only
> slated for major Fedora releases? Meaning: TB 3.1 will come only with F13?
> 
> I have attached the patch, which is slated for TB 3.0.2. Could you not rebuild
> a 3.0.1 with this patch and push it out? Thanks.    

I second this request. The bug is really annoying and upstream fixed it. What's the issue with pushing an update with the patch applied ?

Comment 17 Michael J. Chudobiak 2010-02-23 20:22:54 UTC
+1

Comment 18 Avi Kivity 2010-02-24 21:41:25 UTC
Please push an update package.  This is hurting.

Comment 19 Evan McNabb 2010-02-24 22:08:21 UTC
Also hitting this regularly.