Bug 79310 - write to pipe doesn't block
write to pipe doesn't block
Status: CLOSED CURRENTRELEASE
Product: Fedora
Classification: Fedora
Component: cvs (Show other bugs)
2
i686 Linux
medium Severity medium
: ---
: ---
Assigned To: Martin Stransky
Ben Levenson
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2002-12-09 17:39 EST by Sjoerd Mullender
Modified: 2007-11-30 17:10 EST (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2005-04-26 12:15:45 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
tar file containing strace output of cvs and less commands from bug report (30.00 KB, application/octet-stream)
2002-12-09 17:40 EST, Sjoerd Mullender
no flags Details
strace output of bash showing problematic invocation (128.56 KB, text/plain)
2004-06-19 08:39 EDT, Sjoerd Mullender
no flags Details

  None (edit)
Description Sjoerd Mullender 2002-12-09 17:39:26 EST
Description of Problem:

In certain circumstances, a write to a pipe may return with EAGAIN instead of
blocking.


Version-Release number of selected component (if applicable):
kernel-2.4.18-18.8.0.src.rpm

How Reproducible:
80% of the time.

Steps to Reproduce:
1. from a shell: cvs annotate some-file 2>&1 | (sleep 1; less)
2. 
3. 

Actual Results:
the 3 lines that cvs annotate writes to stderr followed by the tail of the data
that cvs annotate writes to stdout (i.e. the top of the standard output is missing).


Expected Results:
All cvs output.

Additional Information:
cvs annotate first writes 3 short lines to standard error in 5 write system
calls.  It then writes the annotated file to standard output 4096 bytes at a
time.  If the process reading from the pipe isn't ready, the write should block,
but in actual fact, I see that the write system call returns with the error EAGAIN.

See the attachment for the trace output of cvs and of less.  In the cvs trace
search for "EAGAIN", in the less trace search for "read(0,".  Note that the
first read returns the data that cvs wrote the stderr, but that the second read
does not return the same data as the first write to stdout by cvs.
Comment 1 Sjoerd Mullender 2002-12-09 17:40:57 EST
Created attachment 88085 [details]
tar file containing strace output of cvs and less commands from bug report
Comment 2 Sjoerd Mullender 2003-10-31 17:57:44 EST
This bug still exists in Fedora Core 1 Test 3 with
kernel-2.4.22-1.2115.nptl.
I just tried the command on a 825 line file.  When scrolled to the
bottom, less reported that there were only 217 lines (3 for the
header, and so only 214 for the rest of the file).
Comment 3 Sjoerd Mullender 2004-03-10 16:13:48 EST
Still present in Fedora Core 1 with kernel-2.4.22-1.2174.nptl.
This is a very annoying bug since I get bit by it almost every when I
try to view annotation from within XEmacs.

I'm also not happy that this bug is over a year old and apparently
nobody has looked at it.  What is the use of submitting bug reports?
Comment 4 Sjoerd Mullender 2004-06-16 17:00:11 EDT
This bug is still present in Fedora Core 2 with kernel-smp-2.6.6-1.435.

This bug was reported a year-and-a-half ago and nobody has even
acknowledged the report yet!  It is trivial to test!
Comment 5 Alan Cox 2004-06-18 19:35:41 EDT
I've never been able to duplicate this for one. Do you use some
particularly weird shell perhaps ?
Comment 6 Sjoerd Mullender 2004-06-19 03:29:15 EDT
I noticed the problem first in XEmacs (vc-annotate command).  No shell
involved there.  I was able to reproduce it in my own private shell
(originally based on the Bourne shell), but I can also reproduce it in
bash.

I hadn't mentioned it, the CVS repositories I use for this are all
remote repositories accessed with SSH.
Comment 7 Alan Cox 2004-06-19 06:58:18 EDT
Can't reproduce it with bash and ssh repositories either. Can you get
me an strace following forks including that of the shell itself ? That
way I can see which task sets the pipe to nonblocking
Comment 8 Sjoerd Mullender 2004-06-19 08:39:44 EDT
Created attachment 101257 [details]
strace output of bash showing problematic invocation

There is an old attachment with strace output of cvs and less.

I'll add a newly created strace output using this command:
strace -o @bash -f bash -c 'cvs annotate xec.c 2>&1 | (sleep 2; less)'

This invokation also resulted in a truncated result.  This was created on FC1.
Comment 9 David Evers 2004-09-28 07:00:12 EDT
I'm chasing a similar effect with truncated output from rsync -n
over ssh.  For what it's worth, in my case the problem is
not kernel-related, but is an unfortunate side-effect of the

  "2>&1 | [something slow, like tee]"

idiom when combined with ssh.

It seems that rsync's child ssh sets its stdERR non-blocking,
and that stderr has been inherited unchanged from the top-level
rsync.  (The rsync has supplied pipes for its child's stdin
and stdout, but left the stderr alone.  The code that does this
is commented as being derived from CVS code, which is a
possible connection to the original report; see
rsync-2.6.2/pipe.c::piped_child())

Because of the "2>&1", the top-level stderr is a dup of
the top-level stdout, so ssh has inadvertantly made rsync's
stdOUT non-blocking.  Rsync is not expecting that, and does
not check the return code from fflush(stdout), so it can
silently drop lines from stdout. (See the end of
rsync-2.6.3/log.c::rwrite())

For me, removing the "2>&1" from the command restores the full
output, at the cost of not capturing stderr.

The original reporter might like to try the same experiment.

(Versions: OpenSSH_3.7.1p2, rsync version 2.6.2)
Comment 11 Sjoerd Mullender 2004-09-29 03:25:15 EDT
Dropping the (equivalent of) 2>&1 has been my workaround for ages.

David's analysis is consistent with what I have seen, so perhaps this
is not a kernel bug after all.
Comment 12 Dave Jones 2004-11-27 18:17:47 EST
I have no clues (assuming it still isn't fixed with latest updates).
Reassigning to 'bash', maybe Tim has some insight.
Comment 13 Eido Inoue 2004-11-29 12:17:55 EST
reflecting change in comment 12
Comment 14 Tim Waugh 2004-11-29 12:22:39 EST
No idea what happened to the comment I already wrote when I reassigned this to
cvs, but here's a summary:

Read the link: http://sources.redhat.com/ml/bug-glibc/2002-08/msg00041.html

Paul Eggart thinks CVS needs changing, and I tend to agree.
Comment 15 Matthew Miller 2005-04-26 11:12:34 EDT
Fedora Core 2 is now maintained by the Fedora Legacy project for
security updates only. If this problem is a security issue, please
reopen and reassign to the Fedora Legacy product. If it is not a
security issue and hasn't been resolved in the current FC3 updates or
in the FC4 test release, reopen and change the version to match.
Comment 16 Sjoerd Mullender 2005-04-26 12:10:15 EDT
I can't reproduce the bug in FC3+updates anymore, so the report can be closed,
as far as I am concerned.

Note You need to log in before you can comment on or make changes to this bug.