Bug 123022

http://dovecot.fi/patches/imap-fetch-body-section.patch
http://dovecot.fi/patches/maildir.patch
http://dovecot.fi/patches/customflags-fix.patch
dovecot-0.99.10.4-4 added these three patches.

http://www.redhat.com/archives/fedora-test-list/2004-May/msg01303.html
User claims that this caused his dovecot to crash.  User supplies
dovecot.conf file.

http://people.redhat.com/wtogami/temp/
I supplied test dovecot packages that disabled this the maildir patch
alone.

http://www.redhat.com/archives/fedora-test-list/2004-May/msg01386.html
User confirms that my test packages with maildir patch disabled
prevents the crash on his server.  User has not sent any more
debugging information.

So if the user's reports are accurate, the maildir patch causes a rare
corner case segfault in dovecot, perhaps only triggerable from kmail
with certain message contents within the maildir.

Can the user supply a compressed maildir that reproduces this issue? 
Without the ability to reproduce the issue, or other debug information
like gdb backtraces, it is highly unlikely we will be able to solve
the user's problem.  Since nobody else has complained and maildir
works for me, I will not revert the patch in -4 unless the user
responsibly proves that this is a reproducible problem, with a
distributable test-case.

I'd love to help, but I can't do that.  This mail would include 
confidential information. 
 
I am quite sure that dovecot crashed every time with this patch, that 
the previous version did not crash, and that the reverted patch 
version is still working right now. 
 
If it makes you feel better, I have 20+ years experience in unix 
programming and open source software. 
 

Can you create a NEW maildir that contains test mail that reproduces
this crash with -4 version?   Either that or copy your maildir into
another user account on a test server, and delete messages until you
isolate a single message that causes this?

I appreciate that you filed the report, but there is possibly no way
we can fix this without your debugging information.

I can reproduce it on my linux-kernel mailing list folder.
Not private, but not small either.

I had a mail outage recently and I have the distinct impression that
it was happening a lot more while the queue from the MX backup machine
was being flushed.... could it be related to a race with mail delivery?

Will attach the last 1000 lines of strace output last time it died,
and try to catch it in gdb next time. 

Created attachment 100569 [details]
Last 1000 lines of strace output

Well I've attached to it but I'm not sure I'm going to get anything
useful when it crashes.... 

(I see the same if I run 'gdb
/usr/lib/debug/usr/libexec/dovecot/imap.debug 8536' too)

Attaching to program: /usr/libexec/dovecot/imap, process 8536
Error while mapping shared library sections:
: Success.
Error while reading shared library symbols:
: No such file or directory.
Reading symbols from /lib/libdl.so.2...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /lib/tls/libc.so.6...done.
Loaded symbols for /lib/tls/libc.so.6
Reading symbols from /lib/ld-linux.so.2...done.
Loaded symbols for /lib/ld-linux.so.2
0x00464402 in ?? ()

Program received signal SIGSEGV, Segmentation fault.
maildir_hash (p=0xf57ac710) at maildir-sync.c:254
254             while (*s != ':' && *s != '\0') {
(gdb) p/x s
$1 = 0xf57ac710
(gdb) bt
#0  maildir_hash (p=0xf57ac710) at maildir-sync.c:254
#1  0x08087f68 in hash_lookup (table=0x92c21e8, key=0xf57ac710) at
hash.c:173
#2  0x08060602 in maildir_index_sync_readonly (index=0x92c21e8,
fname=0x92c21e8 "@�\t\bP\026,\t",
    found=0xfef1cbc8) at maildir-sync.c:1356
#3  0x0805e436 in maildir_open_mail (index=0x92c2298, rec=0xf589a108,
internal_date=0x92c159c,
    deleted=0xfef1cc74) at maildir-open.c:72
#4  0x0806bd6a in index_mail_next (mail=0x92c155c, rec=0x1) at
index-mail.c:761
#5  0x0806a813 in index_storage_fetch_next (ctx=0x92c1550) at
index-fetch.c:110
#6  0x08055e1c in imap_fetch (client=0x92bab40, fetch_data=53,
imap_data=IMAP_FETCH_UID,
    bodies=0x92b20e8, messageset=0x92c21e8 "@�\t\bP\026,\t",
uidset=153887208) at imap-fetch.c:285
#7  0x0805245a in cmd_fetch (client=0x92bab40) at cmd-fetch.c:351
#8  0x08054170 in cmd_uid (client=0x92bab40) at cmd-uid.c:19
#9  0x08054959 in client_handle_input (client=0x92bab40) at client.c:314
#10 0x080549e8 in _client_input (context=0x92bab40) at client.c:350
#11 0x0808adc8 in io_loop_handler_run (ioloop=0x92ba1c0) at
ioloop-poll.c:214
#12 0x0808a758 in io_loop_run (ioloop=0x92ba1c0) at ioloop.c:258
#13 0x0805aa19 in main (argc=1, argv=0x92c21e8, envp=0x92c21e8) at
main.c:179

This is happening for me with Evolution. Changing summary.

Looks like severe memory corruption...

(gdb) up 2
#2  0x08060602 in maildir_index_sync_readonly (index=0x92c21e8,
fname=0x92c21e8 "@�\t\bP\026,\t",
    found=0xfef1cbc8) at maildir-sync.c:1356
1356                    hash_rec = hash_lookup(ctx->files, fname);
(gdb) p *index
$2 = {open = 0x809ab40 <static_system_pool>, free = 0x92c1650,
set_lock = 0, try_lock = 0x6d,
  set_lock_notify_callback = 0x1, rebuild = 0, fsck = 0x6d,
sync_and_lock = 0x92c6230,
  get_header = 0xf4a67868, lookup = 0x805e5d0 <maildir_hash>, next =
0x805e630 <maildir_cmp>,
  lookup_uid_range = 0x21, lookup_field = 0x92c1448, lookup_field_raw
= 0x63b7e0 <main_arena+96>,
  cache_fields_later = 0x692e2f76, open_mail = 0x2e70616d,
get_internal_date = 0x65646e69,
  expunge = 0x6f6c2e78, update_flags = 0x20, append_begin = 0x38,
append_end = 0x6d6f682f,
  append_abort = 0x77642f65, update_begin = 0x2f32776d, update_end =
0x6c69614d,
  update_abort = 0x2f726964, update_field = 0x73696c2e,
update_field_raw = 0x6d2e7374,
  get_last_error = 0x2f6b3836, get_last_error_text = 0x616d692e, data
= 0x6e692e70,
  tree = 0x2e786564, modifylog = 0x65657274, custom_flags = 0x0,
  dir = 0x29 <Address 0x29 out of bounds>,
  filepath = 0x92c2aa0
"\200�\005\b\220�\005\b�\020\a\b\020\021\a\b@\021\a\b��\005\b@+\a\b
\006\006\b�\021\a\b0\022\a\b�\023\a\bp\024\a\b�\025\a\bP\026\a\b
\027\a\b",
  mailbox_path = 0x4 <Address 0x4 out of bounds>,
  control_dir = 0x92c2d40
"/home/dwmw2/Maildir/.lists.linabi/.imap.index.data",
  default_cache_fields = 4111347712, never_cache_fields = 10264,
indexid = 760,
  sync_id = 4111347712, excl_lock_counter = 0, mbox_fd = 0,
mbox_stream = 0x141,
  mbox_lock_type = 134600064, mbox_dotlock = {dev =
578449631533718672, ino = 578450043850658064,
    mtime = 134603952}, mbox_lock_counter = 134687552,
mbox_sync_counter = 134612512,
  mbox_size = 578451074642809312, mbox_dev = 578453548543972288,
mbox_ino = 578455610128274912,
  last_new_mtime = 134682400, last_uidlist_mtime = 134603520,
maildir_cur_dirty = 134602256,
  next_dirty_flush = 134683344, maildir_lock_fd = 134615840,
new_filename_pool = 0x8071f00,
  new_filenames = 0x8072020, fd = 134684944,
  error = 0x8074c90 "U\211�WVS\203�\034\213}\b\203�\034\001",
mmap_base = 0x80753a0,
  mmap_used_length = 134698160, mmap_full_length = 134698688, header =
0x8075700,
  lock_type = 134685136, file_sync_stamp = 134685216, first_recent_uid
= 153886568,
  lock_notify_cb = 0x92c1ae8, lock_notify_context = 0x92c16b8,
set_flags = 153885000,
  set_cache_fields = 153884304, anon_mmap = 0, opened = 0, rebuilding
= 0, mail_read_mmaped = 0,
  inconsistent = 1, nodiskspace = 0, index_lock_timeout = 0,
allow_new_custom_flags = 0,
  mailbox_readonly = 0, mailbox_lock_timeout = 1, maildir_keep_new =
0, maildir_have_new = 1}

Found it, it's all because I use pointers to mmaped .imap.index.data file. If it grows, the 
pointers get broken. 0.99.10.5 fixes this, I'll release it now :)

dovecot-0.99.10.5-1 built in rawhide. Leaving the bug open cos someone
needs to babysit me through the process of releasing an erratum for
FC1/FC2.

pushed to FC2 testing

IMHO, we do not need any further testing.  This package has more than
proven itself at this point.  Also I notice that you didn't rebuild
for FC1.  I am preparing updates for FC1 and FC2 now, with a different
NVR than rawhide.

Argh... the package pushed to FC2 updates/testing is identical in
N-V-R to rawhide, which is a bit problematic.  I am pushing the -0.FCX
packages that were in testing for the past two weeks.

dovecot-0.99.10.5-0.FC2 did it today. Same backtrace.

David, dovecot-0.99.11 is the current upstream release and has a
number of bug fixes beyond what you're running, its in rawhide and
FC3, perhaps you might want to upgrade to 0.99.11 and see if you
continue hit the same problem.

Yes, that works -- at least it does as soon as I kill the gdb and the
instance of 'imap' which was locking the folder in question.

Are we going to ship it as an erratum for FC2 then?

Closing this as 0.99.11 was shipped in FC3. Shortly I'll push a
0.99.11 version to FC2.