Bug 239215

Summary: Evolution escapes 'From ' on sending mail.
Product: [Fedora] Fedora Reporter: David Woodhouse <dwmw2>
Component: evolutionAssignee: Milan Crha <mcrha>
Status: CLOSED UPSTREAM QA Contact:
Severity: medium Docs Contact:
Priority: medium    
Version: 7CC: lsof, mbarnes, mcepl
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2008-03-13 08:02:43 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
proposed patch 1/2
none
proposed patch 2/2 none

Description David Woodhouse 2007-05-06 09:19:43 UTC
When sending mail with lines starting 'From ', Evolution will escape them by
adding '>' at the beginning of the line.

It's not clear why Evolution would ever do that on sending mail -- that's _only_
required when delivering into mbox storage, and Evolution doesn't _do_ any mail
delivery to mboxes (except possibly in the case of copying a mail from non-mbox
to mbox storage).

It certainly shouldn't be doing it on _sending_.

This has confused me for a while because I think bugzilla might also do the same
when it sees a line starting with
From  -- but I've just sent test messages with both Evolution and 'mail' and
inspected them immediately on the queue of the mail server, and Evolution is
definitely doing it.

Comment 1 David Woodhouse 2007-05-06 09:22:07 UTC
This escaping also happens in the sent-mail folder (IMAP, backed by Maildir).
It's not just in the SMTP transaction.

Comment 2 David Woodhouse 2007-05-21 12:19:40 UTC
GNOME bug 245683 is closed. I'm not going to get involved with the muppet who
closed it -- please could someone else reopen it.

Note that this corruption _is_ noticeable to Evo users -- I see the '>From'
quite clearly, and lines appearing in grey as if they're quotes, rather than
black for original text.

Comment 3 Milan Crha 2007-05-22 16:45:43 UTC
We did reopen that GNOME bug. I will provide a patch, in a while, which could
help and I'll be glad if you could test it. Then, if it will work good, I'll put
that patch upstream.

Comment 4 Milan Crha 2007-05-22 17:19:08 UTC
Created attachment 155181 [details]
proposed patch 1/2

for evolution;

Comment 5 Milan Crha 2007-05-22 17:19:57 UTC
Created attachment 155182 [details]
proposed patch 2/2

for evolution-data-server;

Comment 6 David Woodhouse 2007-05-22 17:30:48 UTC
Hm, it seems I can no longer commit to a private branch in CVS, as I've been
doing for as long as I can remember to get usable evolution and e-d-s builds.

Comment 7 David Woodhouse 2007-05-22 19:26:36 UTC
With a local build, though, that does seem to work. Thanks.

Comment 8 Matthew Barnes 2007-05-22 19:52:06 UTC
Cool, I'll get these patches into Rawhide then.

Comment 9 David Woodhouse 2007-05-22 20:00:40 UTC
Note that I don't actually use mbox files -- except for the local Outbox of course.

So it's probably worth having someone who _does_ use mbox mailboxes test too.

My packages are in a yum repo at ftp://ftp.infradead.org/pub/dwmw2-fc7/ but
since I couldn't commit to CVS I haven't built in koji -- it's just a local
build so it's ppc only. The src.rpm is there too though.


Comment 10 Milan Crha 2007-05-23 07:22:48 UTC
I copied those patch files to upstream bug, maybe they will commit it to trunk too.

Comment 11 Matthew Barnes 2007-05-23 15:07:25 UTC
Based on Jeff's comments in the upstream bug [1], it seems the root problem here
is that Evolution uses mbox for its Outbox folder when it really should be using
maildir.  That's an issue for upstream to address.

But in the meantime Jeff also suggested passing a newly-written message through
CamelMimeFilterCanon to change lines beginning with "From " to "=46rom " using
Quoted-Printable encoding.  That sounds like a reasonable workaround and
something we could do.

Milan, would you like to take a shot at implementing Jeff's suggestion?

[1] http://bugzilla.gnome.org/show_bug.cgi?id=245683#c8

Comment 12 David Woodhouse 2007-05-23 15:21:45 UTC
Sounds like making it use maildir would be the better fix. Why can't we do that
and offer it to upstream?

Comment 13 Matthew Barnes 2007-05-23 16:14:27 UTC
It would be, but Jeff seems to think the transition would be painful.  I don't
really have a sense of it myself though I can imagine backward compatibility
being a major concern.

I'm okay with offering upstream a patch to use maildir, but I don't want to have
to maintain such a patch in Fedora/RHEL.  Upstream would have to accept the
patch and trickle it down to us by way of a new release.

The workaround, however, sounds simpler and I'd be okay with maintaining such a
patch (at least for awhile), assuming it works.

Comment 14 Milan Crha 2007-06-13 11:12:34 UTC
From email conversation about this with Matt, 
> Matt wrote:
>         I think we should try to implement Jeff's suggestion to use
>         quoted-printable encoding to keep mbox "From " lines from
>         getting mangled.  Once David Woodhouse is satisfied with it,
>         forward the solution upstream.  I imagine varadhan will review
>         it.
>         
>         Do you think you can try that?
> 

I think that quoted-printable isn't the right way, because you need to
change original encoding to ours and, for example, when you have a mail
in plain-text/7bit, somewhere inside it with text: "69-4=65", then after
changing from 7bit to quote-printable, you will get in evolution "69-4A"
which isn't obviously that on "input" (you know, same output like
input). The same for camel-mime-filter-canon, which is only one-way,
from "From " to "=46rom " no way to get back properly. All that same
output as input is also for signatures, because you alter message body,
then you will never restore back what you'd got on input.
Most easier way was about that qmail behavior, which Jeff rejected. So a
workaround is changing for default local stores to other type, which
didn't suffer from this kind of troubles, like was mentioned in GN-bug.
As I was talking to varadhan last week, he need to do an import because
of change from "spool" protocol to "spoolfile"/"spooldir", which he
thinks is better than changing UI, and I suggest hit that we could join
this import together with changing default local stores, and he admitted
that we could. Unfortunately there is still no progress about
varadhan's talk to Jeff about "spool".

As I think about this now, it could be some kind of work to have a
checkbox with mbox, which will say something like "Behave as qmail
mbox" (but something more sense-able for customers), maybe default
checked, and it will turning on/off this ">>From " => ">>>From "
conversions from/to mbox file.


Comment 15 Matthew Barnes 2007-09-12 19:56:36 UTC
*** Bug 288351 has been marked as a duplicate of this bug. ***

Comment 16 Need Real Name 2007-09-13 18:59:15 UTC
This bug is annoying. I could understand changing my e-mail while I write it,
but not after I have clicked send.

Comment 17 Milan Crha 2008-02-11 19:17:17 UTC
This bug is somehow stuck in upstream, unfortunately. Jeff is against any "easy"
way how to solve this, and any other solution involves changing that cannot be
easily held by migration from older version. I do not know what to do more with
this than I did in upstream bug, I'm sorry.

Comment 18 Matěj Cepl 2008-03-13 08:02:43 UTC
(In reply to comment #17)
> I do not know what to do more with this than I did in upstream bug, I'm sorry.

Well, the is one standard way how to deal with the situation like this --
CLOSED/UPSTREAM (https://bugzilla.redhat.com/page.cgi?id=fields.html#upstream).
Everybody moves upstream, where the real solution is, our bugzilla is not
plagued by the bug which is not the Red Hat problem, and it could be more useful
for us. I do it for other (especially gecko) bugs all the time and we are
finally getting into Bugzilla being almost useful for us.

There is only one problem with this solution. David absolutely hates this, and
closing bug where he is reporter as UPSTREAM leads to endless flamewar with him.
In my opinion he has the problem distinguishing his mathematical ideal with our
reality where every action has some real costs, but I haven't managed to
persuade him yet.

I learn my children to do what they believe is right. I think I should do
likewise. So, putting on flame-resistant armor, closing down my helmet, and
crawling to corner expecting flamethrower attack, I am closing this bug as
UPSTREAM (the upstream bug is of course
http://bugzilla.gnome.org/show_bug.cgi?id=245683).

Comment 19 David Woodhouse 2008-03-13 08:12:30 UTC
Closing stuff UPSTREAM is fair enough as long as upstream is sane and bugs get
_fixed_. I don't much like the fact that bugzilla calls it 'CLOSED/UPSTREAM'
rather than something like 'PENDING/UPSTREAM' which would be more like the
'NEEDINFO' state. But as long as it gets fixed somehow, it's fine.

Do we have it fixed in Fedora 9 yet, or is Evolution still corrupting mail?