Bug 461707 - mailman breaks CC field incorrectly
mailman breaks CC field incorrectly
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: mailman (Show other bugs)
5.4
All Linux
high Severity medium
: rc
: ---
Assigned To: Jan Kaluža
qe-baseos-daemons
:
Depends On:
Blocks: 499522 603635
  Show dependency treegraph
 
Reported: 2008-09-09 21:17 EDT by ritz
Modified: 2013-04-15 04:59 EDT (History)
6 users (show)

See Also:
Fixed In Version: mailman-2.1.9-5.el5
Doc Type: Bug Fix
Doc Text:
If the original Cc: header field contained an address that spanned across multiple lines, mailman was not able to parse it correctly, causing the resulting address to be invalid. This error has been fixed, and email addresses are now parsed as expected.
Story Points: ---
Clone Of:
: 603635 (view as bug list)
Environment:
Last Closed: 2010-09-29 11:51:46 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
this is the sample of the mail which causes the problem (655 bytes, text/plain)
2009-06-12 08:53 EDT, Daniel Novotny
no flags Details
the proposed patch (1011 bytes, patch)
2009-06-12 09:06 EDT, Daniel Novotny
no flags Details | Diff

  None (edit)
Description ritz 2008-09-09 21:17:36 EDT
* Description of problem:
When customer send an email in certain way, CC filed in
received email shows wiered output.

For example, If email is sent in the following strings,

To: test-list@example.com
Cc: DS◯◯半導体 ◯◯課長1 <test1@node1.example.com>
Cc: DS◯◯半導体 ◯◯課長2 <test1@node1.example.com>

Output of Cc field in received email becomes as follows.

To: test-list@example.com
Cc: 長2, DS◯◯半導体 ◯◯課長,DS◯◯半導体 ◯◯課@node1.example.com

This problem occurs only when a mailing list created in mailman is specified in To/Cc field as one send an email. So this seems to be mailman specific problem.   


* How reproducible:
Always.


*Steps to Reproduce:

1. On one machine(eg, node1.example.com), setup postfix so
  that it can send mail to each user within the local
  machine and setup mailman and create one
  list(eg,test-list).

2. Create at least 3 users for mail accounts(eg,user1,user2,
  user3) and include user1 into the list(test-list).

3. Send a mail from user2 or user3 account in the following
  way.

  To: test-list@node1.example.com
  Cc: DS◯◯半導体 ◯◯課長1 <user2@node1.example.com>
  Cc: DS◯◯半導体 ◯◯課長2 <user3@node1.example.com>

4. Check the mail in user1 account(user1 should have received
  the mail through the test-list) and see Cc field on the
  received mail. It should look like the following.

  From: user2(or user3)
  Sender: test-list-bounces@node1.example.com
  Date: XX:XX XM
  To: test-list@node1.example.com
  Cc: 長2,◯◯半導体 ◯◯課長1,DS◯◯半導体 ◯◯課@node1.example.com

Note: This problem is reproduceable even if you use only
     English charactors. I couldn't get what pattern of
     string it occurs. But looks like it occurs when you put
     long strings in Cc field. Here is example of step 3 in
     English charactors.

3. Send a mail from user2 or user3 account in the following
  way.
  
 To: test-list@node1.example.com
 Cc: DSDS◯◯ handotai ◯◯kachokacho2 <user2.node1.example.com>
 Cc: DSDS◯◯ handotai ◯◯kachokacho3 <user3.node1.example.com>

4. Check the mail in user1 account(user1 should have
  received the mail through the test-list) and see Cc field
  on the received mail. It should look like the following.

  achokacho3,DSDS◯◯ handotai ◯◯kachokacho2,DSDS◯◯ handotai ◯◯k@node1.example.com

*The display in Cc field might look different depending on if you sent the mail in UTF-8 or other charactor set. It does look wierd anyway though.  


*Actual results:
Output in Cc field in received mail looks wierd.


*Expected results:
Display name should be printed in Cc field correctly.


*Additional info:
mailman-2.1.9-2/ RHEL5.1

----------------------------------

The problem is that python script in mailman put COMMASPACE(", ") in between strings in Cc field where it shouldn't. By putting COMMASPACE in between strings, it'll split the strings in Cc field on the display of email clients.

*CC field in Recieved mail header

Cc: =?ISO-2022-JP?B?GyRCIXkheSF5IXkheSF5IXkheSF5IXkheSF5IXkheSF5IXkheRsoQg==?=@dhcp-1-164.bne.redhat.com,
       =?ISO-2022-JP?B?GyRCIXkheSF5GyhC?= <user5@dhcp-1-164.bne.redhat.com>

Note: COMMASPACE(", ") after @dhcp-1-164.bne.redhat.com is causing the problem.

The code adding the COMMPASPACE(", ") in the Cc field is the following.

From /usr/lib/mailman/pythonlib/email/Utils.py

115 def getaddresses(fieldvalues):
116     """Return a list of (REALNAME, EMAIL) for each fieldvalue."""
117     all = COMMASPACE.join(fieldvalues)
118     a = _AddressList(all)
119     return a.addresslist

I've changed the line:117 as follows for testing. This way, it doesn't put COMMASPACE in between the strings.

117     all = fieldvalues

After I've made the above change, I couldn't observe the problem any longer. Even if I put multiple recipients in Cc field. But I doubt this is the right method


getaddresses is used in /usr/lib/mailman/Mailman/Handlers/AvoidDuplicates.py

def process(mlist, msg, msgdata):
   ...
   # RFC 2822 specifies zero or one CC header
   del msg['cc']
   if ccaddrs:
       msg['Cc'] = COMMASPACE.join([formataddr(i) for i in ccaddrs.values()])
Comment 2 Issue Tracker 2008-10-29 20:39:58 EDT
Hi,

It's been few months since I've escalated this ticket. My customer is
continuously contacting me for the fix.

Could you give the current status of this issue? The customer especially
wonder when it will be fixed so that they can plan the deployment of the
system.

Please let me know which update you're planning to fix the issue.

Regards,

Masahiro


This event sent from IssueTracker by mokubo 
 issue 184316
Comment 3 Daniel Novotny 2008-11-06 04:44:53 EST
hello,

I am a new maintainer of mailman and I've started to look at this bug. From the analysis you showed above I think there is a different problem:

 def getaddresses(fieldvalues):
116     """Return a list of (REALNAME, EMAIL) for each fieldvalue."""
117     all = COMMASPACE.join(fieldvalues)

the problem is, that "fieldvalues" should contain a list, where each item will be *one mail address* - that way, when you put COMMASPACE between them, you will get addresses separated by commas.

in the example of the errorneous behavior above it is clear, that the *splitting* is done wrong: it splits inbetween, so one adress is teared apart and mailman thinks they are two (and puts COMMASPACE between "them")

the question is, where does this splitting occur: if you send wrong "fieldvalues" -splitted inbetween- to getaddresses(), you will get wrong cc field, obviously

so the analysis have to look, why the caller of the function "getaddresses()" puts two separated items in the list, where there should be one...
Comment 4 RHEL Product and Program Management 2009-03-26 13:27:27 EDT
This request was evaluated by Red Hat Product Management for
inclusion, but this component is not scheduled to be updated in
the current Red Hat Enterprise Linux release. If you would like
this request to be reviewed for the next minor release, ask your
support representative to set the next rhel-x.y flag to "?".
Comment 6 Daniel Novotny 2009-06-12 08:53:53 EDT
Created attachment 347555 [details]
this is the sample of the mail which causes the problem
Comment 7 Daniel Novotny 2009-06-12 08:58:15 EDT
looking at the sample mail (comment #6)
I can see, that the name is broken apart by a newline character, because it is long... this seems to be parsed by _parseaddr.py . The investigation led me to a similar Debian bug:

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=516300

they solved the problem by patching the _parseaddr.py, backporting from a newer version:

--- /usr/lib/mailman/pythonlib/email/_parseaddr.py.distrib	2006-06-13 05:43:49.000000000 +0200
+++ /usr/lib/mailman/pythonlib/email/_parseaddr.py	2009-02-20 13:19:35.000000000 +0100
@@ -170,6 +170,7 @@
         self.pos = 0
         self.LWS = ' \t'
         self.CR = '\r\n'
+        self.FWS = self.LWS + self.CR
         self.atomends = self.specials + self.LWS + self.CR
         # Note that RFC 2822 now specifies `.' as obs-phrase, meaning that it
         # is obsolete syntax.  RFC 2822 requires that we recognize obsolete
@@ -416,7 +417,7 @@
         plist = []
 
         while self.pos < len(self.field):
-            if self.field[self.pos] in self.LWS:
+            if self.field[self.pos] in self.FWS:
                 self.pos += 1
             elif self.field[self.pos] == '"':
                 plist.append(self.getquote())
Comment 8 Daniel Novotny 2009-06-12 08:59:37 EDT
can you try this, if it helps our problem?
Comment 9 Daniel Novotny 2009-06-12 09:06:48 EDT
Created attachment 347558 [details]
the proposed patch
Comment 10 Issue Tracker 2009-06-22 20:13:51 EDT
Event posted on 06-22-2009 08:13pm EDT by mokubo

Hi,

Any update on this? Our customer want the fix as early as possible and
wants to know when the fix
will be included.

Please let us know when the patch will be included in the package.

Regards,

Masa





This event sent from IssueTracker by mokubo 
 issue 184316
Comment 11 Daniel Novotny 2009-06-24 06:29:20 EDT
hello Masa,

with my patch, the cc field seems to be all in one row and not broken, but I'm not sure if I reproduced the problem correctly, because I don't know Japanese characters. I've built RPM packages of mailman with my patch, so the customer (or you) can use them and see if this helps

http://people.fedoraproject.org/~dnovotny/ccfield


 Daniel
Comment 22 Jaromir Hradilek 2010-09-21 09:53:19 EDT
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
If the original Cc: header field contained an address that spanned across multiple lines, mailman was not able to parse it correctly, causing the resulting address to be invalid. This error has been fixed, and email addresses are now parsed as expected.
Comment 24 errata-xmlrpc 2010-09-29 11:51:46 EDT
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2010-0726.html

Note You need to log in before you can comment on or make changes to this bug.