Bug 21487 - fetchmail chokes on messages w/single period on a line
fetchmail chokes on messages w/single period on a line
Status: CLOSED WORKSFORME
Product: Red Hat Linux
Classification: Retired
Component: fetchmail (Show other bugs)
6.2
i386 Linux
medium Severity high
: ---
: ---
Assigned To: Nalin Dahyabhai
Brock Organ
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2000-11-29 11:49 EST by Tim Jackson
Modified: 2007-04-18 12:30 EDT (History)
1 user (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2002-12-14 23:03:35 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)

  None (edit)
Description Tim Jackson 2000-11-29 11:49:41 EST
If a mail message is sent with a single period (".") on it's own on a line,
somewhere along the line (I'm not sure where) this gets converted to a
double period ("..").  However, retrieving such a message causes fetchmail
to fail - the message will be retrieved up to the line with the periods,
and then fetchmail will die.  A typical session is as follows:

1 message for username at pop3.myisp.com (5373 octets).
reading message 1 of 1 (5373 octets) .....fetchmail: SIGPIPE thrown from an
MDA or a stream socket error
fetchmail: socket error while fetching from pop3.myisp.com
fetchmail: Query status=2 (SOCKET)

No more messages will be fetched from the server, meaning that any messages
received following the "culprit" message will not be able to be retrieved
until the offending message is 
cleared manually or with a different program.

This has been tested with the original RedHat 6.2 RPM (5.3.1) and also the
updated 6.2 RPM (5.6.0).  The bug is present in both cases.   I tested the
bug whilst fetching mail using POP3 - I haven't tried IMAP or other
protocols.

This bug is a bit awkward to reproduce, but is 100% reproducible.  An
example of a mail which will cause this problem is as follows: (smaller
messages may also cause the problem, but with only a small amount of text
following the period, I had problems getting the bug to appear)

--- start of message






  soeirjg oijoejgoiegjrepg

  skrgpokopekgrekgp grgkrpg kpogkopk pookegk pog kgpoekpogkre pogekgpokr
  gporekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 


.
is this line here?
  soeirjg oijoejgoiegjrepg

  skrgpokopekgrekgp grgkrpg kpogkopk pookegk pog kgpoekpogkre pogekgpokr
  gporekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 

  soeirjg oijoejgoiegjrepg

  skrgpokopekgrekgp grgkrpg kpogkopk pookegk pog kgpoekpogkre pogekgpokr
  gporekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 
  soeirjg oijoejgoiegjrepg

  skrgpokopekgrekgp grgkrpg kpogkopk pookegk pog kgpoekpogkre pogekgpokr
  gporekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 
  soeirjg oijoejgoiegjrepg

  skrgpokopekgrekgp grgkrpg kpogkopk pookegk pog kgpoekpogkre pogekgpokr
  gporekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 
  soeirjg oijoejgoiegjrepg

  skrgpokopekgrekgp grgkrpg kpogkopk pookegk pog kgpoekpogkre pogekgpokr
  gporekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 
  soeirjg oijoejgoiegjrepg

  skrgpokopekgrekgp grgkrpg kpogkopk pookegk pog kgpoekpogkre pogekgpokr
  gporekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 
  soeirjg oijoejgoiegjrepg

  skrgpokopekgrekgp grgkrpg kpogkopk pookegk pog kgpoekpogkre pogekgpokr
  gporekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 
  soeirjg oijoejgoiegjrepg

  skrgpokopekgrekgp grgkrpg kpogkopk pookegk pog kgpoekpogkre pogekgpokr
  gporekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 
  soeirjg oijoejgoiegjrepg

  skrgpokopekgrekgp grgkrpg kpogkopk pookegk pog kgpoekpogkre pogekgpokr
  gporekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 

 porekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 
  soeirjg oijoejgoiegjrepg

  skrgpokopekgrekgp grgkrpg kpogkopk pookegk pog kgpoekpogkre pogekgpokr
  gporekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 
 porekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 
  soeirjg oijoejgoiegjrepg

  skrgpokopekgrekgp grgkrpg kpogkopk pookegk pog kgpoekpogkre pogekgpokr
  gporekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 
 porekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 
  soeirjg oijoejgoiegjrepg

  skrgpokopekgrekgp grgkrpg kpogkopk pookegk pog kgpoekpogkre pogekgpokr
  gporekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 
 porekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 
  soeirjg oijoejgoiegjrepg

  skrgpokopekgrekgp grgkrpg kpogkopk pookegk pog kgpoekpogkre pogekgpokr
  gporekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 
 porekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 
  soeirjg oijoejgoiegjrepg

  skrgpokopekgrekgp grgkrpg kpogkopk pookegk pog kgpoekpogkre pogekgpokr
  gporekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 
 porekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 
  soeirjg oijoejgoiegjrepg

  skrgpokopekgrekgp grgkrpg kpogkopk pookegk pog kgpoekpogkre pogekgpokr
  gporekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 
 porekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 
  soeirjg oijoejgoiegjrepg

  skrgpokopekgrekgp grgkrpg kpogkopk pookegk pog kgpoekpogkre pogekgpokr
  gporekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 
 porekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 
  soeirjg oijoejgoiegjrepg

  skrgpokopekgrekgp grgkrpg kpogkopk pookegk pog kgpoekpogkre pogekgpokr
  gporekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 
 porekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 
  soeirjg oijoejgoiegjrepg

  skrgpokopekgrekgp grgkrpg kpogkopk pookegk pog kgpoekpogkre pogekgpokr
  gporekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 
 porekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 
  soeirjg oijoejgoiegjrepg

  skrgpokopekgrekgp grgkrpg kpogkopk pookegk pog kgpoekpogkre pogekgpokr
  gporekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 
 porekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 
  soeirjg oijoejgoiegjrepg

  skrgpokopekgrekgp grgkrpg kpogkopk pookegk pog kgpoekpogkre pogekgpokr
  gporekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 
--- end of message

The fetchmail author has been notified around 1600 UTC today (29-Nov-2000)
Comment 1 Tim Jackson 2000-11-29 11:56:46 EST
I made a slight typo. Please read "..updated 6.2 RPM (5.6.0)" as "...updated 6.2
RPM (5.5.0)"
Comment 2 Nalin Dahyabhai 2000-11-29 12:50:34 EST
This appears to be a protocol violation on the server side.  The "." is the
end-of-message indicator, and the server should be "stuffing" it with a second
".", so that the line appears as "..".  If the output of fetchmail with the
"-vv" flag shows that this is not happening, then the problem may not be fixable
in fetchmail itself.
Comment 3 Tim Jackson 2000-11-29 13:45:36 EST
Sorry, I thought I had made this clear.  The server is acting correctly.  The
message I attached is the *outgoing message*, as sent from a mail client.  When
retrieving the message from the server, the server *does* 'stuff' the period to
make it a double period ("..").  But fetchmail still fails.
The output from a POP3 session (done by telnetting to port 110) is like this:

RETR 1
<snip>
  gporekgpeokg pr grekgrpeog repogkrepgkre roekgpergk epogkorpegk pg 


..   <- note double dots
is this line here?
  soeirjg oijoejgoiegjrepg
<snip>

As you can see, the server is correctly sorting out the periods, so it looks
like it's fetchmail at fault.
Comment 4 Nalin Dahyabhai 2000-11-29 15:17:53 EST
Ah, yes, I understand the problem now.  However, I can't reproduce it with a
lengthy message containing a line by itself.  This is on a 6.2 machine with
fetchmail-5.5.0-2.6 connecting to imap-2000-2.6 on localhost.
Does running fetchmail under 'strace' ("strace fetchmail -vv...") give any
indication as to what the socket error fetchmail appears to be seeing is?
Comment 5 Tim Jackson 2000-11-30 01:04:19 EST
Response from author:

Begin forwarded message:

Date: Wed, 29 Nov 2000 21:48:59 -0500
From: "Eric S. Raymond" <esr@thyrsus.com>
To: Tim Jackson <tim@timj.co.uk>
Subject: Re: NASTY BUG in fetchmail


Tim Jackson <tim@timj.co.uk>:
> It took quite a bit of work and telnetting into POP3 servers to solve the
> problem, but eventually I narrowed it down.  The problem was that someone
> had sent the user in question an e-mail which had a single line containing
> just two dots ("..").  This was causing fetchmail to choke, resulting in
> the delivery of that message, up to the line with the dots in, and no more
> (meaning that mail was stacking up on the remote server).

I believe you'll find this fixed in 5.5.1 and after.
Comment 6 Tim Jackson 2000-11-30 01:30:46 EST
Have just tested it and even fetchmail 5.6.0 definitively does NOT fix this
bug.  Same result.  Author notified as follows:

> I believe you'll find this fixed in 5.5.1 and after.

 I'm afraid not.  I just installed 5.6.0 (from RPM on your site), chucked
 some of my test messages at it again, and:

 [root@myhost tim]# fetchmail --version
 This is fetchmail release 5.6.0
 [root@myhost]# su fetchmailuser -c "/usr/bin/fetchmail --version -f
 /home/fetchmailuser/.fetchmailrc --nodetach"
 2 messages for myusername at pop.myisp.com (10742 octets).
 reading message 1 of 2 (5369 octets) ..... flushed
 reading message 2 of 2 (5373 octets) .....fetchmail: SIGPIPE thrown from
 an MDA or a stream socket error

 Note how the first message came through - this is what I've been seeing
 consistently - some messages get through and some don't. The only
 difference between the two is that the second message had 2 extra blank
 lines at the start of the message body.  Also, although the first message
 was apparently retrieved correctly, I am only seeing the part of the
 message before the ".." in my inbox.
Comment 7 Tim Jackson 2000-11-30 01:34:04 EST
strace dump of fetchmail -vv has been sent direct to nalin@redhat.com
Comment 8 Nalin Dahyabhai 2000-11-30 10:51:19 EST
This makes sense, given that 5.5.0 is the latest version we've included in a
release or as an errata.  5.6.0 will be in the next Raw Hide snapshot, so I'll
tag this one as fixed in Raw Hide.  In the meantime, those packages will be in
http://people.redhat.com/nalin/test/.
Comment 9 Tim Jackson 2000-11-30 11:51:00 EST
Two things:
1. Did you read my later message? Unless you have implemented a fix yourself (if
so please let us know), version 5.6.0 *DOES NOT* fix this bug, contrary to what
the author said.  I have tried the version from his site.

2. The RPM on the URL you gave does not appear to work with RH6.2:

[root@myhost]# rpm -U fetchmail-5.6.0-2.i386.rpm 
error: failed dependencies:
	libk5crypto.so.3   is needed by fetchmail-5.6.0-2
	libkrb5.so.3   is needed by fetchmail-5.6.0-2
	libc.so.6(GLIBC_2.2)   is needed by fetchmail-5.6.0-2
	libresolv.so.2(GLIBC_2.2)   is needed by fetchmail-5.6.0-2
Comment 10 Nalin Dahyabhai 2000-11-30 17:20:04 EST
Aargh.  Need to read all of my bugzilla mail before replying to bug reports.... 
The SIGPIPE would indicate that the child being delivered to died, which makes
it impossible for fetchmail to write to the pipe it's using.  Because the part
of the text of fetchmail's message that indicates what it's doing is "
about to deliver with:"..., I think the MDA is actually barfing on the message.

The Raw Hide SRPM would need to be rebuilt ("rpm --rebuild") for 6.2, because
RHL 6.2 uses Kerberos 1.1.1 and RHL 7 uses Kerberos 1.2.1, and the shared
library version numbers are different between the two versions.
Comment 11 Tim Jackson 2000-11-30 17:45:44 EST
From my .fetchmailrc:
<snip>
     mda "/usr/sbin/sendmail -oem -f%F %T"
<snip>

So......does this in fact indicate a problem with sendmail? Or is this a
configuration issue and I should be using something other than sendmail, or
perhaps using the wrong parameters.
The fact that it's worked reliably for >1 year suggests that maybe this is a
bug...?

[root@myhost /root]# rpm -qa|grep sendmail
sendmail-8.9.3-20

So, assuming my above MDA line confirms your diagnosis, should I open a
bugreport for sendmail? The MDA line came from the fetchmail documentation.  

Either way, I would suggest that it seems a *workaround* for this problem would
be to get fetchmail to forward to a local SMTP host instead, rather than using
the MDA system, but my system's not currently configured for that to work.
Comment 12 Nalin Dahyabhai 2000-11-30 17:52:37 EST
On my development machine (sendmail-8.11.1-1) the retrieval appears to go
correctly.  Try feeding the offending message to the MDA by hand ("cat
badmsg.txt | /usr/sbin/sendmail -oem -fusername username@localhost"), and see if
that triggers any error messages.
Comment 13 Tim Jackson 2000-12-15 12:08:37 EST
Annoyingly, I can't reproduce the failure using this method of piping messages
which would normally cause the problem to sendmail on the command line.  It
seems to work OK. Bit of a sticking point.  But then this problem always has
been a bit tough.  Any suggestions?
I do wonder if the sendmail source is worth a quick look over, particularly
whatever handles bits piped from the command line (as opposed to delivered via
the SMTP interface)?
Comment 14 Mike A. Harris 2001-10-17 09:17:54 EDT
I would suggest taking the mailboxes that cause the problem to
occur, gzipping them, and sending them to Eric Raymond, as well
as attaching them here.  That could possibly shed some light.

Just an idea.

Note You need to log in before you can comment on or make changes to this bug.