Bug 970013 - Workaround QResync Zimbra bug in IMAP+
Workaround QResync Zimbra bug in IMAP+
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: evolution-data-server (Show other bugs)
6.4
Unspecified Unspecified
unspecified Severity medium
: beta
: ---
Assigned To: Matthew Barnes
Desktop QE
:
Depends On:
Blocks: 978525
  Show dependency treegraph
 
Reported: 2013-06-03 06:20 EDT by David Jaša
Modified: 2014-01-02 05:50 EST (History)
3 users (show)

See Also:
Fixed In Version: evolution-data-server-2.32.3-9.el6
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 978525 (view as bug list)
Environment:
Last Closed: 2013-11-21 00:13:25 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
backported eds patch (6.42 KB, patch)
2013-06-26 15:23 EDT, Milan Crha
no flags Details | Diff
backported eds patch (updated) (7.14 KB, patch)
2013-07-08 12:20 EDT, Milan Crha
no flags Details | Diff

  None (edit)
Description David Jaša 2013-06-03 06:20:07 EDT
Description of problem:

Version-Release number of selected component (if applicable):
evolution-2.32.3-1.el6.x86_64

How reproducible:
always

Steps to Reproduce:
1. set up existing IMAP+ account in evolution
2. set up the account to "automatically synchronize remote mail with local"
3. in File menu, hit "download messages for offline use"
4. put evo in offline mode
5. select some message you didn't select before

Actual results:
the message can not be viewed as it wasn't downloaded

Expected results:
the message is downloaded during mail synchronization after step 2. or explicit request to download it after step 3.

Additional info:
This also breaks full-body search in messages...

This bug would be regression from 2.28 behaviour (with IMAP account)
Comment 1 Milan Crha 2013-06-03 11:38:22 EDT
I cannot speak for imap+, it's too new in 2.32.3, from my point of view, but the old imap provider caches only newly recognized messages, if the option is set. That means that old messages are not affected by the option. Also, it might not do much with File->Download for offline, because the messages are scheduled for download when they are recognized as new.
Comment 2 David Jaša 2013-06-03 12:01:43 EDT
The old imap provider at least downloaded the messages when full-body search was invoked on top of the folder, that doesn't work either in imap+/2.32. Maybe worth of a separate bug.

Side question related to searches - when caching is not enabled, is the message indexed locally when downloaded for reading so that it can be searched in the future quickly even when not in cache?
Comment 3 Milan Crha 2013-06-03 15:11:32 EDT
(In reply to David Jaša from comment #2)
> The old imap provider at least downloaded the messages when full-body search
> was invoked on top of the folder, that doesn't work either in imap+/2.32.
> Maybe worth of a separate bug.

The old IMAP does server-side searching, downloading whole folder is impractical for various reasons. Thinking of it, maybe the "File->Download for offline" is there to get copy of the server content locally (I should check the code, of course). I would not use it myself, because I have really many messages (one folder has more than 110K messages), but I agree that it has its use-cases as well.

> Side question related to searches - when caching is not enabled, is the
> message indexed locally when downloaded for reading so that it can be
> searched in the future quickly even when not in cache?

Yes, once you view the message it's stored in a cache and the next time anything asks for the message (can be also a search), the message is taken from cache.
Comment 4 Milan Crha 2013-06-04 14:16:35 EDT
A quick look in the code, I see both imapx and imap provides use the same mechanism to fetch messages for offline, which depends on notifications about newly recognized messages. The quick code review didn't show anything obviously broken, though the imapx code isn't one of the simplest.

With respect of File->Download message for offline, this checks all folders which were opened so far (includign initial message check), and makes sure each message is downloaded in these folders). The tricky part is that the folder should be in the store's 'folders' bag, to be considered for offline, and that either the store or the folder has "sync for offline" set. You indicated above that you have one of the options set, though.
Comment 5 Milan Crha 2013-06-04 14:30:40 EDT
OK, I tested this on my local machine, and the messages are downloaded for offline for me, when keeping in mind restrictions in comment #4, basically with an individual folder on which I set to copy content for offline, and using File->Download for offline downloads all missing messages from that folder. There is one restriction, when changing setup of the account (including "Automatically synchronize remote mail locally") requires restart of evolution to take into effect properly. (Restart of evolution is a known limitation.)

I'm wondering whether either QResyn, or IDLE support of imapx is not doing trouble here. Could you try to disable "Use Quick Resync if the server supports it" and "Use I_dle if the server supports it" on your imapx account, restart evolution and wait whether it'll work, please? I have these options disabled here.

In any case, I would prefer to disable imapx in the rebase, it's not in a good shape in 2.32.3.
Comment 6 David Jaša 2013-06-04 14:45:11 EDT
(In reply to Milan Crha from comment #5)
> ...
> I'm wondering whether either QResyn, or IDLE support of imapx is not doing
> trouble here.

I disabled just Quick Resync and the sync seems to go just fine (let's see in few hours :)).

> In any case, I would prefer to disable imapx in the rebase, it's not in a good
> shape in 2.32.3.

Well, support for IDLE is one of the biggest positives of 2.32 over 2.28...
Comment 7 Milan Crha 2013-06-04 15:14:43 EDT
If it's really about QResync, then Matthew did some changes for this, in series of ~3 commits, beginning at [1], but the code is too different to just backport the changes. I'd rather think of disable and hide QResync (if you want the IDLE, even in not-so-polished imapx provider).

[1] https://git.gnome.org/browse/evolution-data-server/commit/?h=gnome-3-8&id=fe53fde76b9521638b57e28250de8af16e58d206
Comment 8 Matthew Barnes 2013-06-04 16:24:15 EDT
The quick resync issue turned out to be a legitimate Zimbra bug.

http://bugzilla.zimbra.com/show_bug.cgi?id=82493

I only worked around it as best I could.
Comment 9 David Jaša 2013-06-06 12:38:49 EDT
Even with quick resync disabled, the issue is not fully resolved:

First and foremost: when the folder downloading fails during Evo run time, hitting "Download messages for offline use" has no effect anymore (in spite of messages still lacking their bodies)

Second, the messages are downloaded from oldest to newest, the order should be reverse (usually the most recent messages are more relevant and by extension, more worthy to bother with caching them)

Third, the messages are downloaded in from one-folder at time, with no particular folder order. Download order based purely on message age across the whole account would be way better
(it would be perfect if user could ask evo to download and keep messages cached per folder or to select levels of caching - headers/bodies/attachments - but I guess that that would be more of RFE, while the points decribed here look like clear bugs to me).
Comment 10 Milan Crha 2013-06-24 09:01:01 EDT
I'm moving this to evolution-data-server, because the IMAPx code resides there.
Comment 11 Milan Crha 2013-06-26 14:19:37 EDT
Hrm, the change for the offline caching touches both evolution and evolution-data-server, the evolution's fault is that it only sets CamelSession's network-available to FALSE, but never to TRUE (still in upstream too, on 3.9.3). I left this bug for backport of Matthew's QResyc workarounds for a Zimbra bug. The second and the third from the comment #9 are more feature requests, which should be addressed upstream first.
Comment 12 Milan Crha 2013-06-26 15:23:22 EDT
Created attachment 765751 [details]
backported eds patch

for evolution-data-server;

This is backported Matthews change, with minimal impact on the code, but it doesn't seem to work correctly.

My steps:
a) run evolution with an IMAP+ account configured
b) have an empty Inbox, expunged
c) copy 3 messages to Inbox
d) verify thy are there in evolution
e) close evolution with the Inbox selected
f) open a webUI and delete/move away two of the them (maybe which 2 matters?)
g) open evolution

One should see synchronized folder, which will look the same in webui and in evolution, but I do not see them, with the patch applied. I know it's applied, because I added there a test print:
> printf ("%s: picked %d between last_known:%d total:%d\n", __FUNCTION__,
> sequence_limit, last_known_message_cnt, summary_total);
and I see it called, with output like:
> camel_imapx_command_add_qresync_parameter: picked 1 between last_known:1
> total:3
Comment 16 Milan Crha 2013-07-08 12:20:49 EDT
Created attachment 770560 [details]
backported eds patch (updated)

The same as the initial backported patch, only the QResync is disabled by default for newly created IMAP+ accounts. Please be aware of this when testing.
Comment 20 errata-xmlrpc 2013-11-21 00:13:25 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2013-1540.html

Note You need to log in before you can comment on or make changes to this bug.