Bug 393911 - mail is not added to the archive
Summary: mail is not added to the archive
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: mailman
Version: 7
Hardware: All
OS: Linux
low
low
Target Milestone: ---
Assignee: Tomas Smetana
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2007-11-21 10:19 UTC by Zoltan Arpadffy
Modified: 2008-01-22 15:58 UTC (History)
0 users

Fixed In Version: 2.1.9-8.1.fc8
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2008-01-22 15:58:50 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
mbox file example (7.71 KB, application/x-gzip)
2008-01-08 21:11 UTC, Zoltan Arpadffy
no flags Details

Description Zoltan Arpadffy 2007-11-21 10:19:42 UTC
Description of problem:
mailman list is fully functional, but the mail is not added to the archive.
Actually - it seems that tha last mail is addes and the earlier are removed for 
some reason

Version-Release number of selected component (if applicable):
mailman-2.1.9-5
python-2.5-14.fc7

How reproducible:
Unfortunately I do not know how to reporduce... I host more than 200 lists and 
I guess it started after some upgrade.

Steps to Reproduce:
1.
2.
3.
  
Actual results:
In /var/lib/mailman/archives/private/uhu/2007q4 just the last mail is kept

-rw-rw-r-- 1 mailman mailman 4121 2007-11-20 11:49 000085.html
lrwxrwxrwx 1 mailman mailman   11 2007-11-20 11:49 index.html -> thread.html

after sending a new mail the archive looks like below:
-rw-rw-r--  1 mailman mailman 1882 2007-11-21 11:11 000085.html
lrwxrwxrwx  1 mailman mailman   11 2007-11-21 11:11 index.html -> thread.html

The page 000085.html contais the new mail - the old is overwritten.

Also the thread.html and the other html files are missing from the archive.

Expected results:
The archive directory to be populated like:

-rw-rw-r-- 1 mailman mailman 1951 2007-09-20 09:47 000084.html
-rw-rw-r-- 1 mailman mailman 3588 2007-09-21 14:06 000085.html
-rw-rw-r-- 1 mailman mailman 1659 2007-09-20 09:47 author.html
-rw-rw-r-- 1 mailman mailman 1663 2007-09-20 09:47 date.html
lrwxrwxrwx 1 mailman mailman   11 2007-09-20 09:47 index.html -> thread.html
-rw-rw-r-- 1 mailman mailman 1657 2007-09-20 09:47 subject.html
-rw-rw-r-- 1 mailman mailman 1682 2007-09-20 09:47 thread.html

... every mail should be added as a new html file and thread, date etc html 
files regenerated.

Additional info:

Please note that it can have major impact in case the archive is important.

Comment 1 Tomas Smetana 2007-12-05 13:49:37 UTC
It's probably a silly question but... Have you tried to run the check_perms
script?  Mailman is very sensitive to the permissions settings and this may be a
reason.

I'm trying to reproduce the problem, but with no success yet.

Comment 2 Zoltan Arpadffy 2007-12-05 15:17:14 UTC
Hello,

all of the lists hosted by the server are old lists and no change has been done 
except "yum -y update" that freshen the packages and problems started with Q4 
this year (because the archive is done quarterly).

regardless, for security reason I wanted to try to run check_perms ... and here 
is the result:

[root@gate bin]# ./check_perms
Traceback (most recent call last):
  File "./check_perms", line 382, in <module>
    checkarchives()
  File "./check_perms", line 216, in checkarchives
    print _("""\
NameError: global name '_' is not defined


BUT seem the this is a newer mailman.
[root@gate bin]# rpm -q mailman
mailman-2.1.9-5.1

Regards, 
Zoltan Arpadffy   

Comment 3 Tomas Smetana 2007-12-05 16:32:21 UTC
Oh no...  These are the "wonderful" LC_CTYPE patches.  I'm already building a
new package.

Comment 4 Fedora Update System 2007-12-06 20:53:15 UTC
mailman-2.1.9-5.2 has been pushed to the Fedora 7 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update mailman'

Comment 5 Zoltan Arpadffy 2007-12-11 10:45:26 UTC
Hello,

I have upgraded the mailman and the system to F8 as well.
But the problem still persists.

I have tried to recreate the archive and it behaved exactly the same:

- it failed the create the archive. 

[root@gate 2004q2]# pwd
/var/lib/mailman/archives/private/postgres-list/2004q2
[root@gate 2004q2]# ls -al
total 12
drwxrwsr-x 2 root mailman 4096 2007-12-11 11:37 .
drwxrwsr-x 3 root mailman 4096 2007-12-11 11:37 ..
-rw-rw-r-- 1 root mailman 2684 2007-12-11 11:37 000000.html
lrwxrwxrwx 1 root mailman   11 2007-12-11 11:37 index.html -> thread.html

thread.html and others files are missing.


- Running the arch by hand shows the following error:

[root@gate bin]# ./arch --wipe postgres-list
#00000 <20040413044541.52700.qmail.yahoo.com>
figuring article archives
2004q2
Pickling archive state into /var/lib/mailman/archives/private/postgres-
list/pipermail.pck
Traceback (most recent call last):
  File "./arch", line 200, in <module>
    main()
  File "./arch", line 188, in main
    archiver.processUnixMailbox(fp, start, end)
  File "/usr/lib/mailman/Mailman/Archiver/pipermail.py", line 580, in 
processUnixMailbox
    self.add_article(a)
  File "/usr/lib/mailman/Mailman/Archiver/pipermail.py", line 624, in 
add_article
    author = fixAuthor(article.decoded['author'])
  File "/usr/lib/mailman/Mailman/Archiver/pipermail.py", line 62, in fixAuthor
    while i>0 and (L[i-1][0] in lowercase or
UnicodeDecodeError: 'ascii' codec can't decode byte 0xb5 in position 26: 
ordinal not in range(128)


- Additional info:

[root@gate bin]# rpm -q python
python-2.5.1-15.fc8
[root@gate bin]# rpm -q mailman
mailman-2.1.9-8.fc8

Any help would be appreciated.
If you need mbox file, please let me know.

Thank you in advance.

Regards, 
Z

Comment 6 Tomas Smetana 2008-01-08 08:13:55 UTC
Sorry for the delay.  This really doesn't look to be caused by the LC_CTYPE
chaos in Mailman.  If you could attach the mbox (or its part) it may be helpful.
 Thank you.

Comment 7 Zoltan Arpadffy 2008-01-08 21:11:42 UTC
Created attachment 291088 [details]
mbox file example

Please find attached one of the mbox file that produces such a weird beaviour.
Thank you in advance.

Comment 8 Tomas Smetana 2008-01-09 11:43:43 UTC
I have built a testing package. If you wish to try it also yourself:

http://tsmetana.fedorapeople.org/mailman/mailman-2.1.9-8.1.fc8.i386.rpm
http://tsmetana.fedorapeople.org/mailman/mailman-2.1.9-8.1.fc8.src.rpm

Note -- it may not work at all (not very probable but I have to warn you), so
don't run it on a production system.

Comment 9 Zoltan Arpadffy 2008-01-09 22:24:38 UTC
Hello,

I have tested your new build and got the following error:

[root@gate bin]# ./arch --wipe postgres-list
Traceback (most recent call last):
  File "./arch", line 69, in <module>
    from Mailman.Archiver.HyperArch import HyperArchive
  File "/usr/lib/mailman/Mailman/Archiver/HyperArch.py", line 38, in <module>
    import HyperDatabase
  File "/usr/lib/mailman/Mailman/Archiver/HyperDatabase.py", line 28, in 
<module>
    import pipermail
  File "/usr/lib/mailman/Mailman/Archiver/pipermail.py", line 66
    i = i - 1
    ^
IndentationError: expected an indented block

Hope this helps to solve the issue.

Regards,
Z

Comment 10 Zoltan Arpadffy 2008-01-09 22:34:43 UTC
Hello,

actually... after fixing the indentation error the archive worked perfect. I 
was able to recreate all my corrupt lists' archives.

I guess, yet another testing package is needed - but case is closed from my 
side. 

Thank you very much for all your effort.

Regards,
Z 

Comment 11 Tomas Smetana 2008-01-10 06:58:49 UTC
Oh, I look silly again:  You managed to run it first...

In fact I think the problem is on the Python's side -- the lowercase variable
from the string module should be set according to locale and therefore the
comparison should work for non-ascii characters as well.  At least this is how I
understand the documentation.  The fix just avoids the error but it's clear that
the  fixAuthor function doesn't do what I'd expect in this particular case.  But
since it seems to be rare (I'd have more bugs...), I'll leave it as is.

Thank you for your help.

Comment 13 Fedora Update System 2008-01-15 22:55:18 UTC
mailman-2.1.9-8.1.fc8 has been pushed to the Fedora 8 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update mailman'

Comment 14 Fedora Update System 2008-01-22 15:58:49 UTC
mailman-2.1.9-8.1.fc8 has been pushed to the Fedora 8 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.