Bug 209105 - Bad decoding of multiple Subjects
Bad decoding of multiple Subjects
Status: CLOSED WONTFIX
Product: Red Hat Enterprise Linux 3
Classification: Red Hat
Component: squirrelmail (Show other bugs)
3.0
All Linux
medium Severity medium
: ---
: ---
Assigned To: Martin Bacovsky
:
Depends On:
Blocks: 241860 241861
  Show dependency treegraph
 
Reported: 2006-10-03 05:33 EDT by Bastien Nocera
Modified: 2007-11-30 17:07 EST (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2007-05-14 11:04:41 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)
multiline-i18n-header-decode.patch (557 bytes, patch)
2006-10-03 05:33 EDT, Bastien Nocera
no flags Details | Diff
functions/i18n.php sm-1.4.10a patch (2.90 KB, patch)
2007-05-14 13:15 EDT, Tomas
no flags Details | Diff

  None (edit)
Description Bastien Nocera 2006-10-03 05:33:00 EDT
squirrelmail-1.4.6-7.el3

Squirrelmail doesn't decode properly multi-line subjects.
For example, the ASCII version of an e-mail sent to Squirrelmail:

Subject: =?ISO-2022-JP?B?GyRCJCIbKEI=?=a=?ISO-2022-JP?B?GyRCJCQbKEI=?=i=
   ?ISO-2022-JP?B?GyRCJCYbKEI=?=u=?ISO-2022-JP?B?GyRCJCgbKEI=?=e=
   ?ISO-2022-JP?B?GyRCJCobKEI=?=o

The decodeheader i18n option doesn't handle newlines, or tabs at the beginning
of the lines:
-            $ret = str_replace("\t", "", $ret);
+            $ret = str_replace(array("\t", "\n"," "), "", $ret);

Patch from Hideshi Fuchi <hfuchi@redhat.com>
Comment 1 Bastien Nocera 2006-10-03 05:33:01 EDT
Created attachment 137633 [details]
multiline-i18n-header-decode.patch
Comment 4 Warren Togami 2006-10-23 14:34:43 EDT
Is this broken in RHEL3 but not RHEL4?

Either way, *PLEASE* submit this patch and have upstream accept it.
Comment 8 Tomas 2007-02-04 11:36:51 EST
Could you attach encoded header instead of pasting it in bug report? I suspect
that some data is lost.

If header looks same way as in bug report, could you explain why encoded word is
splitted into two lines and why there is no separator between two atoms? With
appropriate references to rfcs.

rfc 2047 states that "'encoded-word's are designed to be recognized as 'atom's
by an RFC 822 parser". According to rfc822 atoms are separated with single
space. They don't look separated in your bug report.
Comment 10 Martin Bacovsky 2007-03-26 06:27:03 EDT
This issue should be fixed squirrelmail-1.4.8-5.el3
Comment 11 Tomas 2007-03-26 06:45:16 EDT
You can't fix it in SquirrelMail, because bug is not in SquirrelMail.
Comment 13 Jan Hutař 2007-03-27 03:58:18 EDT
Hello,
provided Subject (multi)line seems wrong. It is not decoded correctly even in 
Evolution/Sylpheed, so I used this, slightly modified:

Subject: =?ISO-2022-JP?B?GyRCJCIbKEI=?=
  =?ISO-2022-JP?B?GyRCJCQbKEI=?=
  =?ISO-2022-JP?B?GyRCJCYbKEI=?=
  =?ISO-2022-JP?B?GyRCJCgbKEI=?=
  =?ISO-2022-JP?B?GyRCJCobKEI=?=

But this one is OK everywhere, SquirrelMail works correctly with it. Please 
could you attach Subject (multi)line which causes problems and description of 
that problems?

Thanks in advance,
Jan
Comment 14 Tomas 2007-03-27 05:13:09 EDT
Jan,

header in original report contains incorrectly encoded mixed Japanese and ASCII
characters. Probably incorrect decision by some smart mime header encoding
function. No spaces between atoms and incorrect multiline wrapping.

When you modified header, you've removed ASCII chars.

I suspect that header contains encoded "あ a い i う u え e お o" or "あaいiうu
えeおo" text. Depends on how broken mime encoder is.
Comment 16 Tomas 2007-04-16 04:30:45 EDT
Header is generated with SquirrelMail Japanese translation. Create message with
"あaいiうuえeおo" subject and save it as draft. SquirrelMail has similar bug
report (#1693322). Fix proposed in bugzilla is not correct. Japanese XTRA_CODE
functions must be fixed.

https://sourceforge.net/tracker/index.php?func=detail&aid=1693322&group_id=311&atid=100311
Comment 17 Martin Bacovsky 2007-04-24 11:23:26 EDT
Problem seems to be in foldLine() function in class/deliver/Deliver.class.php.
It tries to split long headers along RFC2047, but it has troubles with more than
one Japanese char separated by ASCII chars. 

I rewrote foldLine() so that it split subject at right places, but since the
ascii chars are left as they are, we got some unwanted extra spaces in header
(like this あaいi うuえe おo). I hope that converting ascii chars to ISO-2022-JP
will help to avoid those extra spaces.

I tried to create mail with our subject in thunderbird and it produces this:
Subject: =?ISO-2022-JP?B?GyRCJCIbKEJhGyRCJCQbKEJpGyRCJCYbKEJ1GyRCJCgbKEJl?=
 =?ISO-2022-JP?B?GyRCJCobKEJv?=
which is nice also according to RFC2047.

I'm not sure wahat is the preffered way to fix this for the upstream so I have
to consult them first.
Comment 18 Tomas 2007-04-24 11:59:05 EDT
I am not sure if there is a bug in foldLine. Invalid code is in
japanese_charset_xtra() function, encodeheader switch.

Header posted by Bastien Nocera can be generated with SquirrelMail in Japanese
translation. Before you change foldLine() make sure that header is encoded
correctly.
Comment 20 Martin Bacovsky 2007-05-10 03:04:48 EDT
(In reply to comment #18)
As I understand it, header is encoded properly (but not very nicely)
(=?ISO-2022-JP?B?GyRCJCIbKEI=?=a=?ISO-2022-JP?B?GyRCJCQbKEI=?=i=?ISO-2022-JP?B?GyRCJCYbKEI=?=u=?ISO-2022-JP?B?GyRCJCgbKEI=?=e=?ISO-2022-JP?B?GyRCJCobKEI=?=o)
If it is not folded later most mail clients display the header properly.
Foldmethod as it is by now folds the header inside the multibyte atoms (it is
wrong). "Fixed" foldmethod folds the header on borders of atoms, but this adds
some extra spaces when interpreted by mail client.

I think the ideal way to fix this is to convert all chars in headers to either 
ISO-2022-JP or UTF-8 or UNICODE. Unfortunatley I'm not very familiar with
encodings conversion so it will need some more investigation.
Comment 23 Suzanne Yeghiayan 2007-05-14 11:04:41 EDT
Patch provided was insufficient.  Devel is working with the upstream community
however they do not expect to have a new patch by the 3.9 deadline.
Marco (GSS) checked with customer and they are happy with a php workaround.
The errata has been dropped from RHEL3-QU9.
Changed bugzilla status to CLOSED/WONTFIX.
If this continues to be an issue post 3.9 GA, feel free to reopen and raise for
an async errata.
Comment 24 Tomas 2007-05-14 13:15:53 EDT
Created attachment 154673 [details]
functions/i18n.php sm-1.4.10a patch

replaces 'encodeheader' code with simple non-smart encoding.
Comment 25 Tomas 2007-05-14 13:41:29 EDT
"Bad decoding of multiple Subjects" issue can't be fixed. rfc2047 - "6.3. Mail
reader handling of incorrectly formed 'encoded-word's" chapter.

---
A mail reader need not attempt to display the text associated with an
'encoded-word' that is incorrectly formed.  However, a mail reader
MUST NOT prevent the display or handling of a message because an
'encoded-word' is incorrectly formed.
---

"Bad encoding issue" should be fixed in attached patch. Please note that I am
the one who wrote encodeHeaderBase64(). SquirrelMail has three header encoding
functions and I trust only encodeHeaderBase64(). Patch requires SquirrelMail
1.4.6 and php mbstring extension with Japanese support. If patch is applied to
older SquirrelMail version, you must port related functions. If code executed on
PHP install without mbstring support, Japanese translation should fail on
set_up_language() call.

Fix is simple. I have added blocks of comments in order to explain what I am
doing. If you need explanations why original code is broken, I can tag invalid
decisions made by original code.

Note You need to log in before you can comment on or make changes to this bug.