|Summary:||checking mail now seems to use SELECT instead of STATUS|
|Product:||[Fedora] Fedora||Reporter:||David Woodhouse <dwmw2>|
|Component:||evolution-data-server||Assignee:||Matthew Barnes <mbarnes>|
|Status:||CLOSED UPSTREAM||QA Contact:|
|Version:||10||CC:||fortran, frank, mattdm, mcrha, stu|
|Fixed In Version:||Doc Type:||Bug Fix|
|Doc Text:||Story Points:||---|
|Last Closed:||2009-06-29 02:23:17 UTC||Type:||---|
|oVirt Team:||---||RHEL 7.3 requirements from Atomic Host:|
|Cloudforms Team:||---||Target Upstream Version:|
Description David Woodhouse 2006-02-27 16:19:57 UTC
In FC4, Evolution would use STATUS to check for mail in folders without actually opening them. It did this poorly, as is the subject of other bugs. But now in rawhide (evo-data-server 1.5.91) it's actually got worse -- it now actually opens every folder by using the SELECT command, instead of just using STATUS. Once it's changed the open folder by issuing SELECT, it then needs to re-open the folder which _should_ be open, and needs to re-fetch all the flags for that folder. That's amazingly inefficient (or just broken, if it doesn't do it).
Comment 1 David Woodhouse 2006-02-27 19:24:55 UTC
OMFG it seems not only to be using SELECT but _also_ actually fetching all headers for all mails in all folders. That's just insane -- adding to FC5Blocker.
Comment 2 David Woodhouse 2006-02-27 19:41:19 UTC
It's taken ten minutes so far, and it's only on the third folder of about 30-odd. What used to be a two-line transaction with the mail server (STATUS and the response) is now a multi-megabyte download taking many minutes. I think I'll go home and come back in the morning to see if it's finished starting up. This needs to get fixed if we're not going to just drop Evolution from the distribution. Has anyone actually been using Evolution with IMAP recently? I'd have kept a closer eye on it if I'd known it wasn't getting any testing.
Comment 3 David Woodhouse 2006-02-28 13:32:14 UTC
It's also fetching all flags each time it selects each folder, so even though it is actually vaguely responsive after leaving it overnight, it's still a massive regression from before. It's hardly usable. The patches at http://david.woodhou.se/evolution-2.5.91-check-only-active-folders.patch and http://david.woodhou.se/evolution-data-server-1.5.91-check-only-active-folders.patch do help somewhat, by making it do this only for _active_ folders, not for _every_ folder including multiple years worth of mail archives -- but it's still very broken.
Comment 4 David Woodhouse 2006-02-28 15:04:36 UTC
Created attachment 125398 [details] Partial attempt at a fix. This goes some way to fixing the problem, by making it use STATUS for any non-current folder instead of SELECT and completely updating the folder info. It's not sufficient though -- the IMAP code's idea of 'current_folder' isn't necessarily always correct, and the unseen counts aren't being shown even for the non-current folders, when the STATUS command observes that they've changed. It may be that the GUI code is fetching the counts from the cached folder info, not just the summary which my new function is updating. Still investigating...
Comment 5 David Woodhouse 2006-02-28 15:33:53 UTC
Created attachment 125400 [details] This time without the most obvious bugs. This fixes the obvious bugs in the first attempt, but the behaviour is still the same.
Comment 6 David Woodhouse 2006-02-28 17:19:59 UTC
This is probably the right approach, but there are two problems yet to be solved. Firstly, the GUI code is probably using camel_folder_get_unread_message_count() to get the unread message count, and that's ignoring the summary info which my patch updates -- it's making it up from the cached message index. We should probably fix that in imap_getv(), and then the message counts should work. Secondly, the check against imap_store->current_folder isn't the right thing to do. What we actually want to do is use SELECT if it's a folder for which the GUI actually cares about the message index, and use STATUS only when the GUI cares only about the unseen count. The GUI can have more than one folder 'open' at a time, in multiple windows, and it can use some folders as source for vFolders, etc. -- all these folders would fall into the first category where we should use 'SELECT'. We should probably add a folder flag to show that the GUI has a folder 'open', and then the code in my patch can use SELECT for any folder which is 'open', and STATUS for any folder which is not. Something for discussion with evo hackers upstream. I lack the patience for such things.
Comment 7 David Woodhouse 2006-03-15 22:06:20 UTC
This bug is still present in what looks like it's going to be the final FC5 release. It means that evolution is fairly much unusable on remote mailservers with any sizeable amount of mail. Is there any prospect of a fix? Could we revert to Evolution 2.4? Does that predate the changes which broke it?
Comment 8 Matthias Clasen 2006-07-26 01:14:57 UTC
What is the status of this bug ? It still sits on the FC5 Blocker, which does not do it any good. Does it need to be moved to FC6Target or FC6Blocker ?
Comment 9 David Woodhouse 2006-07-26 01:34:50 UTC
We need an erratum for FC5 ASAP, and it should also be on FC6Blocker.
Comment 10 Matthew Barnes 2006-07-26 01:49:03 UTC
I could use some technical assistance with this one, as it's pretty far over my head at this point. I'm afraid my knowledge of IMAP is only elementary until I can find time to study it further. David, are any of the above patches still applicable?
Comment 11 David Woodhouse 2006-07-26 01:57:23 UTC
Applicable still as a proof of concept, yes, but not really ready for submission. At one point, a Novell Evolution hacker (shres) picked them up and was trying to beat them into shape. He got laid off though, and the patches are apparently lost on his old workstation. The IMAP part of it isn't the hard part -- it's making Evolution deal with the fact that it doesn't have all the flags and headers, for folders which aren't used as vFolder sources. The blocker is the flag which marks a folder as 'open' on the GUI side, meaning that we actually _do_ need all headers &c for it.
Comment 12 Matthew Barnes 2006-07-31 01:48:03 UTC
Realistically, I doubt a solution is going to emerge in time for Fedora Core 6. It sounds like this is going to require a large chunk of time to understand the problem, develop a solution, and test it well enough to release. Not to mention upstream coordination. While I think most Evolution users would acknowledge that the IMAP support needs improvement, it seems to work well enough for many -- including myself -- to use on a daily basis. So I think my time would be better spent fixing crashers and security vulnerabilities before FC6 is released. So unless someone else can devote time to this, I recommend that this be moved to FC6Target instead of FC6Blocker.
Comment 13 Milan Crha 2007-10-22 11:24:23 UTC
Hi David, can you review this with latest Evolution (2.12.1) in F8T3, please? As far as I know, there has been done some changes in imap provider of camel, even not many. Also, it can depend on a provider you are running, there are 3 imap providers in camel. (But I don't expect you use other than that default. Just in case.) Also, can you check your setup in Edit->Preferences, Edit imap account -> tab "IMAP Headers"? Sometimes can help in tab "Receiving Options" disable "Check for new messages in all folders", but I don't think we want to go in this way.
Comment 14 Frank Crawford 2008-03-22 12:28:35 UTC
Folks, has any activity gone on with this? This appears to be an issue I am having with Evolution since FC6 sometime, up to F8, with Microsoft Exchange as an IMAP server and evolution-2.12.3-3.fc8. The mailbox size is > 2G (many, many individual messages) and over a hundred separate folders. The "Check for new messages in all folders" is set to off, but the problem still occurs. Running gdb appears to indicate that evolution is scanning inactive mailboxes for something, and eventually hangs for long periods of time. I can supply more details and gdb traces if is will help.
Comment 15 David Woodhouse 2008-05-13 15:58:24 UTC
Hm, why moved to Fedora 9? This should stay on rawhide; it's been open for over two years already; there's no reason to believe it'll be fixed by F10, is there?
Comment 16 Matthew Barnes 2008-05-13 18:14:16 UTC
It was just part of a mass move. I'm fine with moving this one back to Rawhide.
Comment 17 Bug Zapper 2008-05-14 02:06:06 UTC
Changing version to '9' as part of upcoming Fedora 9 GA. More information and reason for this action is here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 19 Bug Zapper 2008-11-26 01:48:35 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 10 development cycle. Changing version to '10'. More information and reason for this action is here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping