In FC4, Evolution would use STATUS to check for mail in folders without actually opening them. It did this poorly, as is the subject of other bugs. But now in rawhide (evo-data-server 1.5.91) it's actually got worse -- it now actually opens every folder by using the SELECT command, instead of just using STATUS. Once it's changed the open folder by issuing SELECT, it then needs to re-open the folder which _should_ be open, and needs to re-fetch all the flags for that folder. That's amazingly inefficient (or just broken, if it doesn't do it).
OMFG it seems not only to be using SELECT but _also_ actually fetching all headers for all mails in all folders. That's just insane -- adding to FC5Blocker.
It's taken ten minutes so far, and it's only on the third folder of about 30-odd. What used to be a two-line transaction with the mail server (STATUS and the response) is now a multi-megabyte download taking many minutes. I think I'll go home and come back in the morning to see if it's finished starting up. This needs to get fixed if we're not going to just drop Evolution from the distribution. Has anyone actually been using Evolution with IMAP recently? I'd have kept a closer eye on it if I'd known it wasn't getting any testing.
It's also fetching all flags each time it selects each folder, so even though it is actually vaguely responsive after leaving it overnight, it's still a massive regression from before. It's hardly usable. The patches at http://david.woodhou.se/evolution-2.5.91-check-only-active-folders.patch and http://david.woodhou.se/evolution-data-server-1.5.91-check-only-active-folders.patch do help somewhat, by making it do this only for _active_ folders, not for _every_ folder including multiple years worth of mail archives -- but it's still very broken.
Created attachment 125398 [details] Partial attempt at a fix. This goes some way to fixing the problem, by making it use STATUS for any non-current folder instead of SELECT and completely updating the folder info. It's not sufficient though -- the IMAP code's idea of 'current_folder' isn't necessarily always correct, and the unseen counts aren't being shown even for the non-current folders, when the STATUS command observes that they've changed. It may be that the GUI code is fetching the counts from the cached folder info, not just the summary which my new function is updating. Still investigating...
Created attachment 125400 [details] This time without the most obvious bugs. This fixes the obvious bugs in the first attempt, but the behaviour is still the same.
This is probably the right approach, but there are two problems yet to be solved. Firstly, the GUI code is probably using camel_folder_get_unread_message_count() to get the unread message count, and that's ignoring the summary info which my patch updates -- it's making it up from the cached message index. We should probably fix that in imap_getv(), and then the message counts should work. Secondly, the check against imap_store->current_folder isn't the right thing to do. What we actually want to do is use SELECT if it's a folder for which the GUI actually cares about the message index, and use STATUS only when the GUI cares only about the unseen count. The GUI can have more than one folder 'open' at a time, in multiple windows, and it can use some folders as source for vFolders, etc. -- all these folders would fall into the first category where we should use 'SELECT'. We should probably add a folder flag to show that the GUI has a folder 'open', and then the code in my patch can use SELECT for any folder which is 'open', and STATUS for any folder which is not. Something for discussion with evo hackers upstream. I lack the patience for such things.
This bug is still present in what looks like it's going to be the final FC5 release. It means that evolution is fairly much unusable on remote mailservers with any sizeable amount of mail. Is there any prospect of a fix? Could we revert to Evolution 2.4? Does that predate the changes which broke it?
What is the status of this bug ? It still sits on the FC5 Blocker, which does not do it any good. Does it need to be moved to FC6Target or FC6Blocker ?
We need an erratum for FC5 ASAP, and it should also be on FC6Blocker.
I could use some technical assistance with this one, as it's pretty far over my head at this point. I'm afraid my knowledge of IMAP is only elementary until I can find time to study it further. David, are any of the above patches still applicable?
Applicable still as a proof of concept, yes, but not really ready for submission. At one point, a Novell Evolution hacker (shres) picked them up and was trying to beat them into shape. He got laid off though, and the patches are apparently lost on his old workstation. The IMAP part of it isn't the hard part -- it's making Evolution deal with the fact that it doesn't have all the flags and headers, for folders which aren't used as vFolder sources. The blocker is the flag which marks a folder as 'open' on the GUI side, meaning that we actually _do_ need all headers &c for it.
Realistically, I doubt a solution is going to emerge in time for Fedora Core 6. It sounds like this is going to require a large chunk of time to understand the problem, develop a solution, and test it well enough to release. Not to mention upstream coordination. While I think most Evolution users would acknowledge that the IMAP support needs improvement, it seems to work well enough for many -- including myself -- to use on a daily basis. So I think my time would be better spent fixing crashers and security vulnerabilities before FC6 is released. So unless someone else can devote time to this, I recommend that this be moved to FC6Target instead of FC6Blocker.
Hi David, can you review this with latest Evolution (2.12.1) in F8T3, please? As far as I know, there has been done some changes in imap provider of camel, even not many. Also, it can depend on a provider you are running, there are 3 imap providers in camel. (But I don't expect you use other than that default. Just in case.) Also, can you check your setup in Edit->Preferences, Edit imap account -> tab "IMAP Headers"? Sometimes can help in tab "Receiving Options" disable "Check for new messages in all folders", but I don't think we want to go in this way.
Folks, has any activity gone on with this? This appears to be an issue I am having with Evolution since FC6 sometime, up to F8, with Microsoft Exchange as an IMAP server and evolution-2.12.3-3.fc8. The mailbox size is > 2G (many, many individual messages) and over a hundred separate folders. The "Check for new messages in all folders" is set to off, but the problem still occurs. Running gdb appears to indicate that evolution is scanning inactive mailboxes for something, and eventually hangs for long periods of time. I can supply more details and gdb traces if is will help.
Hm, why moved to Fedora 9? This should stay on rawhide; it's been open for over two years already; there's no reason to believe it'll be fixed by F10, is there?
It was just part of a mass move. I'm fine with moving this one back to Rawhide.
Changing version to '9' as part of upcoming Fedora 9 GA. More information and reason for this action is here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
This bug appears to have been reported against 'rawhide' during the Fedora 10 development cycle. Changing version to '10'. More information and reason for this action is here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Closing this since an investigation is already well underway upstream. Please see [1] for further updates. [1] http://bugzilla.gnome.org/show_bug.cgi?id=336076