Bug 113492
Summary: | after expunge, dovecot hangs fetchmail if new e-mail came in | ||
---|---|---|---|
Product: | [Fedora] Fedora | Reporter: | Alexandre Oliva <oliva> |
Component: | fetchmail | Assignee: | John Dennis <jdennis> |
Status: | CLOSED RAWHIDE | QA Contact: | |
Severity: | medium | Docs Contact: | |
Priority: | medium | ||
Version: | 3 | CC: | fabrice, riel, tss, wtogami |
Target Milestone: | --- | ||
Target Release: | --- | ||
Hardware: | i386 | ||
OS: | Linux | ||
Whiteboard: | |||
Fixed In Version: | Doc Type: | Bug Fix | |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2004-09-30 21:16:25 UTC | Type: | --- |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: | |||
Bug Depends On: | |||
Bug Blocks: | 133398 | ||
Attachments: |
Description
Alexandre Oliva
2004-01-14 17:06:48 UTC
This is one of the other error messages I mentioned in the report above, that I couldn't quote at that time: [...] reading message aoliva.ic.unicamp.br:45 of 45 (2206 header octets) .. (2070 body octets) .. flushed imap(oliva): Error: Error indexing mbox file /var/mail/oliva: LF not found where expected Timo, have you seen anything like this before? That looks a bit as if locking isn't right. MTA using only dotlocking and appending mail while Dovecot has locked it for reading with fcntl() and later gets confused? Then again, mbox syncing does have some bugs. I didn't think there were anything too simple though.. MTA (sendmail) is using both fctnl and .lock, according to strace. Created attachment 100547 [details]
fix when imap EXPUNGE cmd doesn't update the message count
The fetchmail timeout with dovecot after the EXPUNGE command occurs because
dovecot doesn't report the updated number of RECENT and EXISTS messages, like
wu-imap did. A possible workaround is to issue a SELECT or EXAMINE command
after the EXPUNGE, to force an update of these values. It seems to work for me
with this attached patch
Created attachment 100548 [details]
possible fix for the "LF not found" problem ?
I'm not sure this is the right fix, but the input->v_offset value is modified
in mbox_index_rewrite(), and sometimes, an inconsistant value coming from this
function is propagated back to the parent, causing the error described in the
bug report. Typically, the offset is no longer at the boundary between two
messages, as it should be, but in the middle of the first message for example.
This case occurs during the EXPUNGE command when the mailbox is modifed by
another process (eg sendmail). Saving and restoring the current v_offset value
seems to solve this issue.
Created attachment 100550 [details]
LF not found patch 2
Thanks for figuring out the problem. I looked it a bit and figured out that
stream->v_size will also be too small if mbox_index_rewrite() grows the file.
This patch should fix it.
Hopefully this was the cause of the quite-rare mbox corruption that I had been
hearing. I'll release 0.99.10.5 today with this and several other important
fixes.
Created attachment 100552 [details]
LF not found patch 3
There was a bug in the patch :) This one seems to work.
And about the fetchmail/EXISTS problem - sending EXISTS after EXPUNGEs is pretty pointless and so is re-selecting the mailbox after it. It should rather just internally keep state of the message count (save EXISTS value, decrease it for each EXPUNGE). Recent counters are different though, that should be re-sent if any expunged messages was a \Recent message. I'm not fixing this for 0.99.10 though, it has other problems with recent counters as well.. Timo, Please inform us when the new release is ready, and I will prepare the official FC1 & FC2 update. After a bit of testing we will push this. Thanks, Warren Created attachment 100585 [details]
fix for recentcount and count after an imap EXPUNGE
About the timeout bug in fetchmail, as suggested, I reworked on a possible
patch. This one updates internally the value of count (EXISTS), recentcount
(RECENT) and expungedcount (EXPUNGE), when they are not updated by the imap
server itself after the expunge command. To be safe about recentcount, I reset
this value to zero, when it's not updated by the imap server. The visible
effect of this, is that messages incoming while the mailqueue is processed are
kept in queue.
dovecot-0.99.10.5-1 is currently in rawhide and I am currently testing it before releasing an update for FC1 and FC2. Timo please let me know if Fabrice's comment #11 should be applied or not. #11 is a patch for fetchmail, I can't really comment on that .. except I don't think expungedcount is a good idea, rather just decrease "count" directly. Otherwise it'd break if server sent eg.: * 1 EXPUNGE * 2 EXISTS * 1 EXPUNGE count -= expungedcount -> 0, not 1 as it should oops, sorry I should have actually read it. =) Fabrice can you please talk to fetchmail upstream about this issue? Time for me to prepare the update announcement... Created attachment 100788 [details]
new fix for recentcount and count after an imap EXPUNGE
Ah yes. I forgot this case. This new patch updates count directly. I'll report
this case on the fetchmail mailing-list.
dovecot update for FC1 and FC2 went out a few days ago. Comment #15 contains a patch for fetchmail that still hasn't been integrated. Reopening the bug and moving it to fetchmail. Ping? This is still present in rawhide (FC3 approaching test2). Huh? What's still in rawhide? The dovecot-0.99.10.5-1 rpm or the bug? Niether of which to the best of my knowledge is in rawhide. The current package in FC3 is dovecot-0.99.10.9-1.FC3.2 which has the fix for this bug in it (thats my understanding) and was up until 3 days ago the current upstream release. dovecot-0.99-11 was released on Saturday, which will go in FC3, probably today. The fetchmail bug. See, this bug has been in the fetchmail component since the dovecot problems were all fixed. But the patch in Comment #15, for fetchmail, is still needed to fix the reported problem. yep, I confirm that there's still a problem in fetchmail imap, when the RECENT and EXISTS values are not updated and reported after an EXPUNGE by the imap server. This bug is fixed by the patch of Comment #15. I mentionned it on the fetchmail-friends mailing-list in June, without much comments. It seems that the development of fetchmail now shifted to http://fetchmail.berlios.de so I'll resubmit this patch to the newly created fetchmail-devel list asap. Ok, it will be fixed upstream: http://lists.berlios.de/pipermail/fetchmail-devel/2004-September/000141.html Any chance of having this patch integrated before FC3test3? final patch applied in fetchmail-6.2.5-6 |