Bug 481408 - Evolution sometimes fails to remove trailing blanks before sending to gpg
Summary: Evolution sometimes fails to remove trailing blanks before sending to gpg
Keywords:
Status: CLOSED UPSTREAM
Alias: None
Product: Fedora
Classification: Fedora
Component: evolution
Version: 9
Hardware: i386
OS: Linux
low
low
Target Milestone: ---
Assignee: Matthew Barnes
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2009-01-24 06:29 UTC by archimerged Ark submedes
Modified: 2009-02-09 14:06 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of: 121689
Environment:
Last Closed: 2009-02-09 14:06:10 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
GNOME Bugzilla 571046 0 None None None Never

Description archimerged Ark submedes 2009-01-24 06:29:53 UTC
+++ This bug was initially created as a clone of Bug #121689 +++

Description of problem:

In some cases such as when receiving signed attachments, evolution
does not ensure that all signed messages lines end with CRLF rather
than just LF, when sending them to gpg for signature check.

Version-Release number of selected component (if applicable):
evolution-2.22.3.1-1.fc9.i386

How reproducible:
Always

Steps to Reproduce:
1. Setup a gpg private key.
2. Setup evolution to gpg-sign messages.
3. Send a test message to yourself.  The signature comes up good.
4. Forward the received message to yourself.  The inner signature
   comes up good but the outer signature is bad.
5. Then look at the forwarded message in the sent folder.
   Both signatures are good.
6. Save the sent message and the received message and compare.
   The difference is that message from the sent folder has CRLF 
   line terminators but the received message does not.
7. It might be that evolution doesn't remove the CR's
   but rather fails to add them, so that if they are not removed
   somewhere in transit, or if they are added and not removed
   before receipt, the signature might come up good.  This doesn't
   seem likely since I don't think messages as stored in /var/mail
   on a pop3 server have CRLF line terminators.

Actual results:
Received messages with multipart/signed are stored without putting 
CRLF terminators on the signed text.  The signatures fail.

Expected results:
Either stored multipart/signed messages should have CRLF terminators
on the signed lines, or these lines should be modified as they are
sent into gpg for signature check.
 
Additional info:

See also the relevant RFC's, which state that all signed lines are supposed
to be normalized to CRLF line terminators before calculating the hash.

I haven't looked into whether or not the problem in bug 121689 was the same.

[121689] --- Additional comment from paul.uk on 2004-08-31 19:54:58 EDT ---
[121689] This is fixed in the current version in rawhide (1.5.93).

Comment 1 Milan Crha 2009-01-27 14:16:54 UTC
I tried to reproduce this issue and I found that two of my pop3 accounts are working fine, one not. When comparing differences, I found out that my third provider changed content of the inner message and remove one empty line marker in the main message. (I was forwarding signed message as an attachment).

The changes they did in the original (attached) message are these:
-Received-SPF: ...
+Received-Spf: ...

...

-Content-ID: ...
+Content-Id: ...

I'm afraid we cannot do anything with this, except of using some encoding for attachments, like base64, but that's not the best thing either.

With respect to CRLF, I saved message also from the Outbox and I see there correct CRLF, same as you expected. It either involves your "sometimes" in this bug description, or it works as expected.

Can you try to go offline, and save message from the outbox, then go online, send that message and if received mail will be wrong, check the differences? (Note the message saved from other than Outbox will not have the CRLF in its file representation. It didn't for me, but the signature check works fine).

Comment 2 archimerged Ark submedes 2009-01-28 17:02:39 UTC
The problem is the X-Keywords: header, which gains a trailing blank somewhere in transit, and evolution fails to remove it when normalizing the message.

I forwarded the earlier message to myself, but left it on my pop3 server, and copied the mbox from /var/spool/mail.  There were absolutely no CR's in the file.  It was strictly Unix line terminators.  Then I told evolution to fetch the mail and looked at the message.  (Of course, the copy in outbox had three nested messages, all with valid signatures).  The received copy had a valid signature on the innermost message, but bad signatures on the outer two messages.

Next, I put a script named gpg into my ~/bin directory, set PATH to put ~/bin first, and executed evolution with that PATH in its environment.  The script is

#!/bin/bash
f="evolution-to-gpg/$(date --rfc-3339=ns | tr ' ' _)"
cat | tee -a "$f" | /usr/bin/gpg "$@"


I ran evolution initially looking at the message with two bad signatures.
Then I selected the message with good signatures.  gpg ran 6 times.
The third and last times received identical data.  The other two differed
in that the X-keywords: header had a trailing blank in the message with
bad signature.

[gamma@gamma ~]$ md5sum evolution-to-gpg/2009-01-28_11:33*
a14f5f0c6db47534e7f64ac3dc5c1d82  evolution-to-gpg/2009-01-28_11:33:24.555857930-05:00
55bb0d2be157a42e767179f0096d9e49  evolution-to-gpg/2009-01-28_11:33:26.264805116-05:00
85b394016aea70a588ff40d6e425c235  evolution-to-gpg/2009-01-28_11:33:28.130681433-05:00
26e342c6dd7870268737a853a9a50369  evolution-to-gpg/2009-01-28_11:33:42.203581978-05:00
7a5b1d11851977073e159230e15abaa2  evolution-to-gpg/2009-01-28_11:33:43.291619585-05:00
85b394016aea70a588ff40d6e425c235  evolution-to-gpg/2009-01-28_11:33:44.168275545-05:00
[gamma@gamma ~]$ diff evolution-to-gpg/2009-01-28_11:33:24.555857930-05:00 evolution-to-gpg/2009-01-28_11:33:42.203581978-05:00 
60c60
< X-Keywords: 
---
> X-Keywords:
[gamma@gamma ~]$ diff evolution-to-gpg/2009-01-28_11:33:26.264805116-05:00 evolution-to-gpg/2009-01-28_11:33:43.291619585-05:00 
33c33
< X-Keywords: 
---
> X-Keywords:
[gamma@gamma ~]$ diff evolution-to-gpg/2009-01-28_11:33:26.264805116-05:00 evolution-to-gpg/2009-01-28_11:33:43.291619585-05:00 | tr \\r @
33c33
< X-Keywords: @
---
> X-Keywords:@
[gamma@gamma ~]$

Comment 3 Milan Crha 2009-01-28 17:58:14 UTC
Thanks for your reply and investigation. As your first sentence says:
> The problem is the X-Keywords: header, which gains a trailing blank somewhere
> in transit, and evolution fails to remove it when normalizing the message.

I do not think Evolution can do a "normalization" in such a way, because, as I understand RFC 822, user defined headers can contain basically any text, including a space. Thus that can be a correct value for that, so how could evolution recognize whether it is supposed to be there or not? I also do not think it'll make any sense to try find some possibly broken things in a message, when an attempt to check signature fails. I cannot imagine such thing for my example from comment #1 at all.

Those which failed, are that inner messages or one of them is an outer/main/new message? I guess both are inners. It's incorrect on a server side to modify inner message headers, regardless which server on its journey does this. And no email client can solve such server's fault.

I cannot think of any solution on Evolution's side here, I'm sorry.

Comment 4 archimerged Ark submedes 2009-01-29 03:41:47 UTC
No, all evolution has to do is follow the RFC.  Remove all trailing spaces.  They were removed before the signature was computed when the message was sent, and they are to be removed before the signature is computed upon receipt.

Comment 5 archimerged Ark submedes 2009-01-29 03:56:54 UTC
(Any trailing spaces actually in the original X-foo: header (which is just part of the message by the time it is being forwarded) were to be escaped by the transfer encoding, so any trailing spaces found in the received message were _certainly_ added by the mail transport system).

Comment 6 Milan Crha 2009-02-04 13:23:41 UTC
I briefly looked into RFC 4880, which defines OpenPGP. Section 5.2.4 [1] says:

   For binary document signatures (type 0x00), the document data is
   hashed directly.  For text document signatures (type 0x01), the
   document is canonicalized by converting line endings to <CR><LF>,
   and the resulting data is hashed.

Can you point me to the place/RFC where is said how to preprocess signature text data, with respect to the remove of trailing spaces, please?

[1] http://tools.ietf.org/html/rfc4880#section-5.2.4

Comment 7 archimerged Ark submedes 2009-02-06 12:20:50 UTC
http://tools.ietf.org/html/rfc3156 section 3:

"For this reason all data signed according to this protocol MUST be constrained to 7 bits (8-bit data MUST be encoded using either Quoted-Printable or Base64). Note that this also includes the case where a signed object is also encrypted (see section 6).  This restriction will increase the likelihood that the signature will be valid upon receipt.

Additionally, implementations MUST make sure that no trailing whitespace is present after the MIME encoding has been applied."

Since RFC3156 requires that no line of signed data may end in whitespace, it is clearly impossible for an implementation to convert an invalid signature to a valid signature by removing trailing whitespace.  Any existing trailing whitespace was necessarily added by the mail transport system and hence should be removed.  No valid signed message has trailing whitespace.

A header which actually ends in whitespace should be encoded according to RFC1522, e.g., "X-foo-bar-header: foo bar=?US-ASCII?Q?=20?=", prior to signature hashing.  Any received header ending in unencoded whitespace should be normalized to remove the whitespace prior to signature hashing.

See also section 7 of RFC-4880, which specifies the handling of cleartext (non-mime) messages: "Also, any trailing whitespace -- spaces (0x20) and tabs (0x09) -- at the end of any line is removed when the cleartext signature is
generated."

RFC1521 p. 18 "Therefore, when decoding a Quoted-Printable body, any trailing white space on a line must be deleted, as it will necessarily have been added by intermediate transport agents."

A "content-type: message/rfc822" part cannot be quoted-printable (RFC1521, in order to prevent recursive encoding), but the headers can be quoted according to RFC1522 and the transport-encoding of the body can be changed prior to construction of the message/rfc822 part.  Once this part is enclosed in a multipart-signed, the transport-encoding cannot be changed (RFC3156), but if this were necessary, it would already have been done.

Comment 8 Milan Crha 2009-02-09 14:06:10 UTC
Thanks for all the pointers. I pushed this upstream, as a bug [1], because it would be better to have it available not only in Fedora.

[1] http://bugzilla.gnome.org/show_bug.cgi?id=571046


Note You need to log in before you can comment on or make changes to this bug.