Bug 102208 - Bug in sort when merging FIFO's
Bug in sort when merging FIFO's
Product: Red Hat Linux
Classification: Retired
Component: coreutils (Show other bugs)
All Linux
medium Severity medium
: ---
: ---
Assigned To: Tim Waugh
Depends On:
  Show dependency treegraph
Reported: 2003-08-12 12:08 EDT by Kostadin Koruchev
Modified: 2007-04-18 12:56 EDT (History)
0 users

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2004-12-07 06:42:25 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Kostadin Koruchev 2003-08-12 12:08:28 EDT
i386 (all?)

Environ: any

Sporadic -- no. Systematic -- yes.

Possible system crash -- no. 

Possible user programms crash -- yes.


  Merge with fifo as input files always opens 
  a temporary file and can crash the
  programm and overflow the file system.


 mkfifo /tmp/aaa /tmp/aab
 awk 'BEGIN{for(i=0;i<10000000;i++) print i;}' >> /tmp/aab &
 awk 'BEGIN{for(i=0;i<10000000;i++) print i;}' >> /tmp/aaa &
 sort -m -n /tmp/aaa /tmp/aab >/dev/null &

 Well, the actual problem is merging very big files, usually when the result is
 not saved on the disk.

 It seems that RH 6.x had no problems.


 -- normal termination, no result.

Actual results:

 -- No space on  filesystem /tmp/

Lateral effects

 -- can crash any program that use /tmp.
 -- performance impact (file written twice).
 -- if /tmp/ is located on another filesystem the bug can 
    lead to lost of data.

Probable location:

  Buggy check for same file in the input list.
Comment 1 Tim Waugh 2003-08-12 12:20:11 EDT
So, put simply: 'sort uses temporary file'?
Comment 2 Tim Waugh 2003-10-03 05:22:19 EDT
Can you please explain in more detail where you think the bug is?  I'm not sure
what you mean by 'Buggy check for same file in the input list.'.  Thanks.
Comment 3 Kostadin Koruchev 2003-10-14 07:37:55 EDT
Sorry for the delay.

I think that the error is in the function first_same_file() in
combination with the logic of temporaty file creation in merge(),
when using pipes.

The reason of the bug:

When named pipes (pp) are used as an output of sort,
there might be a feedback (so there can exists cat pp > entry_of_sort) 
and the result, according to /bin/sort, 
must be saved in a temporary file in order to avoid the immidiate feedback
that will block the program.

The behaviour is actually not the expected from the user, who usually
uses the pipes as a dynamics structure and expects that the result will
be written directly into the pipe.

It is possible to construct some rather rare examples, where the data
feedback via named pipes is usefull, but the "normal" usage of the pipes is
to use them as unidirectional dataflow. After all it is also possible
to add a buffer to sort as for example sort | buffer.sh > pp and the buffer
will behave exactly as the current version of sort.


     cat > tmpfile
     cat tmpfile

On contrary, it is not possible to avoid the temporary file with the current
logic of the program sort.

More sofisticated logic will be to check the existance of the 
data flow loop, 
but I think that too much sofistication is sort can bring only harm.
At least the moment of the check can not be defined.

Comment 4 Tim Waugh 2004-12-07 06:42:25 EST
This is the sort of thing that needs to be addressed by the upstream maintainers
of coreutils.

Note You need to log in before you can comment on or make changes to this bug.