Bug 543687 - Compression failures in IMAP/S with cyrus-imap
Summary: Compression failures in IMAP/S with cyrus-imap
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: cyrus-imapd
Version: 11
Hardware: All
OS: Linux
low
medium
Target Milestone: ---
Assignee: Michal Hlavinka
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2009-12-02 21:04 UTC by Philip Prindeville
Modified: 2010-01-05 22:53 UTC (History)
3 users (show)

Fixed In Version: 2.3.16-1.fc11
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2010-01-05 22:53:37 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)

Description Philip Prindeville 2009-12-02 21:04:04 UTC
Description of problem:

Seeing a lot of indications of:

Dec  2 12:44:15 mail imaps[3817]: executed
Dec  2 12:50:43 mail imaps[3817]: accepted connection
Dec  2 12:50:43 mail imaps[3817]: imapd:Loading hard-coded DH parameters
Dec  2 12:50:43 mail imaps[3817]: SSL_accept() incomplete -> wait
Dec  2 12:50:43 mail imaps[3817]: SSL_accept() succeeded -> done
Dec  2 12:50:43 mail imaps[3817]: starttls: TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits new) no authentication
Dec  2 12:50:43 mail imaps[3817]: login: builder.redfish-solutions.com [192.168.10.7] philipp plain+TLS User logged in
Dec  2 12:50:43 mail imaps[3817]: seen_db: user philipp opened /var/lib/imap/user/p/philipp.seen
Dec  2 12:50:43 mail imaps[3817]: open: user philipp opened INBOX.Drafts
Dec  2 13:12:43 mail imaps[3817]: IOERROR: reading message: unexpected end of file
Dec  2 13:12:43 mail imaps[3817]: Error decompressing data, closing connection
Dec  2 13:14:15 mail master[2178]: process 3817 exited, status 0
Dec  2 13:11:15 mail imaps[3986]: executed
Dec  2 13:14:43 mail imaps[3986]: accepted connection
Dec  2 13:14:43 mail imaps[3986]: imapd:Loading hard-coded DH parameters
Dec  2 13:14:43 mail imaps[3986]: SSL_accept() incomplete -> wait
Dec  2 13:14:43 mail imaps[3986]: SSL_accept() succeeded -> done
Dec  2 13:14:43 mail imaps[3986]: starttls: TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits new) no authentication
Dec  2 13:14:44 mail imaps[3986]: login: builder.redfish-solutions.com [192.168.10.7] philipp plain+TLS User logged in
Dec  2 13:14:44 mail imaps[3986]: seen_db: user philipp opened /var/lib/imap/user/p/philipp.seen
Dec  2 13:14:44 mail imaps[3986]: open: user philipp opened INBOX.Drafts
Dec  2 13:14:44 mail imaps[3986]: Expunged 21 messages from user.philipp.Drafts
Dec  2 13:16:43 mail imaps[3986]: IOERROR: reading message: unexpected end of file
Dec  2 13:16:43 mail imaps[3986]: Error decompressing data, closing connection
Dec  2 13:17:57 mail master[2178]: process 3986 exited, status 0


or else:

Dec  2 13:19:09 mail imaps[4000]: accepted connection
Dec  2 13:19:09 mail imaps[4000]: imapd:Loading hard-coded DH parameters
Dec  2 13:19:09 mail imaps[4000]: SSL_accept() incomplete -> wait
Dec  2 13:19:09 mail imaps[4000]: sslv3 alert certificate unknown in SSL_accept() -> fail
Dec  2 13:19:09 mail imaps[4000]: imaps TLS negotiation failed: builder.redfish-solutions.com [192.168.10.7]
Dec  2 13:19:09 mail imaps[4000]: Fatal error: tls_start_servertls() failed
Dec  2 13:19:09 mail master[2178]: process 4000 exited, status 75
Dec  2 13:19:09 mail master[2178]: service imaps pid 4000 in BUSY state: terminated abnormally


indicating issues with SSL compression with IMAP.  Note that this only affects auto-saving of messages into "Drafts" or copying sent messages into the "Sent" folder.

If SSL compression is affecting submission to the local SMTP(/S) server, I'm not seeing it.

I'm running TB 3.0b4 on an FC11 x86_64 client.  The server is also FC11 x86_64, both are updated.

I'm not aware of this problem existing prior to the upgrade to TB 3.0beta.

This happens most often when sending emails with large binary attachments (e.g. .docx files).

The odd part is that the process that was previously connected to will exit, and my UA (Thunderbird) will tell me that there was an error ("I/O Error") and ask me if I want to retry.

Clicking OK will cause it to connect to a freshly spawned process and succeed.

If I had to guess, I would conclude that over time, the client and server ends get out of sync (and therefore, agreement about connection state) and the server process exits.  When a new process is created and its state initialized, the client and server processes once again are in agreement.

Therefore, debugging should look at how the two come to be out of agreement.


Version-Release number of selected component (if applicable):

Server:

cyrus-imapd-2.3.15-4.fc11.x86_64
cyrus-imapd-perl-2.3.15-4.fc11.x86_64
cyrus-imapd-utils-2.3.15-4.fc11.x86_64
cyrus-sasl-2.1.22-22.fc11.x86_64
cyrus-sasl-lib-2.1.22-22.fc11.i586
cyrus-sasl-lib-2.1.22-22.fc11.x86_64
cyrus-sasl-md5-2.1.22-22.fc11.x86_64
cyrus-sasl-plain-2.1.22-22.fc11.x86_64
openssl-0.9.8k-5.fc11.x86_64
openssl-0.9.8k-5.fc11.i686
zlib-1.2.3-22.fc11.i586
zlib-1.2.3-22.fc11.x86_64


Client:

zlib-1.2.3-22.fc11.x86_64
zlib-1.2.3-22.fc11.i586
thunderbird-3.0-2.8.b4.fc11.x86_64


How reproducible:

Configure TB 3.0b4 to have an IMAP/S connection to a server running Cyrus 2.3.15.  Enable auto-saving of messages.  Configure to save sent messages in the "Sent" folder on server.

Create a message, and attach a Word document to it.  Send it.


Steps to Reproduce:
1. above
2.
3.
  
Actual results:

The send will succeed, but the first attempt to copy of the sent message to "Sent" will fail with "I/O Error".  The UA will ask you if you want to retry the copy.  Click "Ok".  The second attempt will succeed.

Note: this doesn't happen 100% of the time.


Expected results:

The first attempt to copy the message to the "Sent" folder should always succeed.


Additional info:

This may have occurred as early as cyrus-imapd-2.3.14-1.fc10.x86_64 and the FC9 to FC10 migration of the server in question, i.e. the change in Cyrus (and not the change in TB at around the same time) might have been the true indicator.

Comment 1 llevet 2009-12-11 17:22:27 UTC
Same problem with cyrus-imapd-2.3.15-5.fc11.x86_64 and Thunderbird 3.0 Final.

No problem for me on FC9 (cyrus-imapd-2.3.11-1.fc9.i386).

Warning, compress was add Thunderbird in V3 too ! 

Ref : 

https://bugzilla.mozilla.org/show_bug.cgi?id=533789

Comment 2 Michal Hlavinka 2009-12-16 17:22:04 UTC
thanks for reporting this

compression is new feature for both server and client, so it's hard to tell who is broken. I've asked upstream of cyrus-imapd for their opinion

Comment 3 Michal Hlavinka 2009-12-18 19:46:35 UTC
does anyone of you tested both imaps and imap+tls ?

also please try to reproduce this with IDLE disabled in TB3 if it makes any difference

thanks

Comment 4 llevet 2009-12-18 22:52:41 UTC
I'm just try with imaps (ssl) and imap+tls (starttls) and both give me the same error where compression is enable.

Disable or enable IDLE with compress active give the error too.

Only complete disable of compress in TB3 resolve the issue.

Ludo.

Comment 5 Philip Prindeville 2009-12-20 21:48:59 UTC
(In reply to comment #3)
> does anyone of you tested both imaps and imap+tls ?
> 
> also please try to reproduce this with IDLE disabled in TB3 if it makes any
> difference
> 
> thanks  

So that others can try these steps and confirm or deny that they work, perhaps you can list the steps necessary to do this?

Just in case they don't know what all of the TB settings are, or how to enable/disable certain Cyrus behavior, why not list those steps...

Comment 6 llevet 2009-12-21 10:21:08 UTC
To custom TB3, in config editor the options are :

mail.server.default.use_idle : true or false

mail.server.default.use_compress_deflate : true or false

Close and restart TB3 after each modification.


Ludo.

Comment 7 Michal Hlavinka 2009-12-22 21:08:41 UTC
thanks for answering that question Ludo

anyway, I've got info that new cyrus-imapd 2.3.16 that was just released fixed this issue for at least one user. Could you please try updated version if it fixes problem for you? thanks

updated packages can be found here 

http://koji.fedoraproject.org/koji/buildinfo?buildID=148239

or you can wait when they are pushed to updates-testing (usually within two days) and then update using

yum update --enablerepo=updates-testing 'cyrus-imapd*'

Comment 8 llevet 2009-12-23 11:56:30 UTC
Hi,

I have update to cyrus-imapd-2.3.16-1.fc11.x86_64 and now, it is fixed for me.

Thank's a lot !

Ludo.

Comment 9 Michal Hlavinka 2010-01-04 13:42:12 UTC
ok, thanks for testing

Comment 10 Philip Prindeville 2010-01-04 20:17:41 UTC
(In reply to comment #8)
> Hi,
> 
> I have update to cyrus-imapd-2.3.16-1.fc11.x86_64 and now, it is fixed for me.
> 
> Thank's a lot !
> 
> Ludo.  

And for me, with TB 3.0b4.

Comment 11 Fedora Update System 2010-01-05 22:53:32 UTC
cyrus-imapd-2.3.16-1.fc11 has been pushed to the Fedora 11 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.