Bug 708898
Summary: | offlineimap 6.3.3 deletes local emails | ||||||||
---|---|---|---|---|---|---|---|---|---|
Product: | [Fedora] Fedora | Reporter: | Peter Hutterer <peter.hutterer> | ||||||
Component: | offlineimap | Assignee: | Christoph Höger <choeger> | ||||||
Status: | CLOSED ERRATA | QA Contact: | Fedora Extras Quality Assurance <extras-qa> | ||||||
Severity: | unspecified | Docs Contact: | |||||||
Priority: | unspecified | ||||||||
Version: | 15 | CC: | amit.shah, cfergeau, choeger, lepennec, metherid, sebastian | ||||||
Target Milestone: | --- | ||||||||
Target Release: | --- | ||||||||
Hardware: | Unspecified | ||||||||
OS: | Unspecified | ||||||||
Whiteboard: | |||||||||
Fixed In Version: | offlineimap-6.3.4-1.fc16 | Doc Type: | Bug Fix | ||||||
Doc Text: | Story Points: | --- | |||||||
Clone Of: | Environment: | ||||||||
Last Closed: | 2011-11-03 00:22:18 UTC | Type: | --- | ||||||
Regression: | --- | Mount Type: | --- | ||||||
Documentation: | --- | CRM: | |||||||
Verified Versions: | Category: | --- | |||||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
Cloudforms Team: | --- | Target Upstream Version: | |||||||
Embargoed: | |||||||||
Attachments: |
|
Description
Peter Hutterer
2011-05-30 04:49:18 UTC
Hi, there are many possible sources of that error. In particular: Do you know the mailserver in place? Are the emails still there with another uid? (Some servers do not really give unique ids)? What happens if you sync with offlineimap to an empty directory, are those mails actually downloaded? (In reply to comment #1) > Do you know the mailserver in place? The mail server is dovecot 1.2.8 fwiw, this setup hasn't changed in ages, neither the server nor the local offlineimaprc. > Are the emails still there with another uid? (Some servers do not really give unique ids)? I'm not quite sure how to check if the emails are still there with another UID, any instructions would be appreciated. > What happens if you sync with offlineimap to an empty directory, are those mails actually downloaded? Yes, it syncs. Seems to have downloaded all emails as required. But when I ran the same command again (offlineimap -f lists-xorg) on the just-downloaded folder it deleted a number of them again: Deleting 5268 messages (32891, ... I have the logs for the second run (was silly enough to overwrite the log from the empty sync) but I don't see any real difference to the original one, seems to be just more of the same. Ok dovecot is known to be very reliable and works well with offlineimap. > I'm not quite sure how to check if the emails are still there with another UID, any instructions would be appreciated. You could access the server and simply grep your mailbox's uidlist for the deleted messages and also search for duplicates. (see http://wiki1.dovecot.org/MailboxFormat/Maildir#IMAP_UID_mapping for details) The question here is, why offlineimap downloads the messages first and _then_ deletes them. Somehow we need to nail the actual problem (aka, what makes those 5268 messages different from the rest), so we can inform upstream. Created attachment 502395 [details]
dovecot-uidlist
I'll just attach the whole list here. UID list and the filenames are unique across the file and I'm having a hard time figuring out what makes 32891 special (or 32768 in the previous run).
Thanks for the detailed IMAP log. This is much appreciated, and it helps for sure. OfflineImap has recently switched to IMAPLIB2, in order to be able to support IMAP IDLE and other cool stuff. But your log seems indicate that it eats data. For example the email server sends untagged_responses[FETCH] 5399 += ["16222 (FLAGS (\Seen) UID 32749)"] But the subsequent _get_untagged_response(FETCH) => ['1 (FLAGS (\\See...... does not contain the very UID, and we subsequently try to delete it locally as we think it's gone from the server. I am in contact with the imaplib2 author and will try to get this debugged ASAP. Hi Peter, I am in contact with the imaplib2 author and he would like to investigate this. Despite the great IMAP log, there is some stuff snipped out in the [...] part. Do you still have the full log and would you be willing to share it with me (or directly the imaplib2 author)? You ca reach me at sebastian. Thanks Just to let you know that we are on it: We fetch a list of UIDs from the server, but the list must currently not be interrupted with unrelated responses. THIS is send right after one of "missing UIDs" DEBUG[imap]: 26:17.31 who-t.net reader < * OK Searched 43% of the mailbox, ETA 0:12\r\n so this series (handler log): [FETCH] 10821 += ["10822 (FLAGS (\Answered \Seen) UID 27260)"] [OK] 0 += ["Searched 43% of the mailbox, ETA 0:12"] [FETCH] 0 += ["10823 (FLAGS (\Answered \Seen) UID 27261)"] is broken into several "Responses" (note the FETCH restarts counting at 0). It breaks the series of untagged_responses just as the imaplib2 author had suspected. Fixing this will require to fix imaplib2 and bundle a new version in a new version of offlineimap. Just FYI, a fix is in the current "master" tree and *I* would recommend to upgrade any fedora package to use the latest master HEAD as soon as possible to avoid further potential data loss by others. Thanks for raising this and providing us with detailed logs. It was a pleasure to fix this bug :-). Christoph, are you going to submit new packages or do I have to try from git? You can try it from git for now. I am going to update to 6.3.4 ASAP once it is out. Since offlineimap is a python package, I do not see much sense in building -rc*.rpms btw, I can confirm that v6.3.4-rc3 from git has fixed this bug. Good to hear that. Can you please update to the latest git snapshot if the release hasn't been made yet? 6.3.4 has been released on August 19 according to the git tag upstream. Please update on F15 and 16. Christoph Höger, do you need help with the update? I can push an update myself if needed. offlineimap-6.3.4-1.fc16 has been submitted as an update for Fedora 16. https://admin.fedoraproject.org/updates/offlineimap-6.3.4-1.fc16 offlineimap-6.3.4-1.fc15 has been submitted as an update for Fedora 15. https://admin.fedoraproject.org/updates/offlineimap-6.3.4-1.fc15 Package offlineimap-6.3.4-1.fc16: * should fix your issue, * was pushed to the Fedora 16 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing offlineimap-6.3.4-1.fc16' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2011-14630 then log in and leave karma (feedback). offlineimap-6.3.4-1.fc15 has been pushed to the Fedora 15 stable repository. If problems still persist, please make note of it in this bug report. offlineimap-6.3.4-1.fc16 has been pushed to the Fedora 16 stable repository. If problems still persist, please make note of it in this bug report. |