Bug 103744

Summary: slow-running CGI scripts are buffered too much
Product: [Retired] Red Hat Linux Reporter: Hrunting Johnson <hrunting>
Component: httpdAssignee: Joe Orton <jorton>
Status: CLOSED ERRATA QA Contact: Brian Brock <bbrock>
Severity: medium Docs Contact:
Priority: medium    
Version: 9   
Target Milestone: ---   
Target Release: ---   
Hardware: i386   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2003-12-16 21:45:04 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Hrunting Johnson 2003-09-04 15:29:18 UTC
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5b)
Gecko/20030901 Firebird/0.6.1+

Description of problem:
When running a long-running CGI script, output is buffered until the CGI script
completely finishes.  Only when the CGI exits does the web server actually send
the data.  This worked fine under the previous httpd package (before the latest
errata).

I have verified via strace that the httpd process is reading in the data from
the CGI process, but is not writing anything to the connected socket.  The
output once rendered is still of a chunked form and the server has not
calculated a Content-Length, but it all gets sent at once instead of
line-by-line as it comes in.  No filters are configured on the output.  The
server just doesn't write the data until the process exits.

Version-Release number of selected component (if applicable):
httpd-2.0.40-21.5

How reproducible:
Always

Steps to Reproduce:
1. Run a long running CGI script with unbuffered output
2.
3.
    

Actual Results:  The CGI script's data is sent to the browser once the CGI
script exits.

Expected Results:  The CGI script's data is sent to the browser as it is sent to
the CGI pipe.

Additional info:

#!/usr/bin/perl

use strict;
use CGI;

use constant TRACEROUTE => qw| /usr/sbin/traceroute -w 2 |;
use constant BUF_LENGTH => 1024;

local( $| ) = 1;

my $q = CGI->new();
my $ip = $q->param('ip') || $ENV{'REMOTE_ADDR'};
my $out = $q->param('out');
exit 1 unless $ip;

my $pid = open( TR, '-|' );
local $SIG{'PIPE'} = sub { kill( 'TERM', $pid ); } if $pid;
do {
  open( STDERR, '>/dev/null' );
  exec( TRACEROUTE, $out ? ( '-s', $out ) : (), $ip );
  die( "could not exec traceroute: $!" );
} unless $pid;

print $q->header( 'text/plain' );

my ( $buf, $length );

# do some extra processing on Linux because of the length line output
# print $buf while ( $length = sysread( TR, $buf, BUF_LENGTH ) ) > 0;

while ( ( $length = sysread( TR, $buf, BUF_LENGTH ) ) > 0 ) {
  $buf =~ s/(\d+)\.\d+ (.?s)/$1 $2/sg;
  print $buf;
}

close( TR );
exit 0;

Comment 1 Joe Orton 2003-09-05 06:09:43 UTC
This should be fixed if you up2date to the erratum which was just issued. 
Please reopen bug 97111 if you still have problems.

*** This bug has been marked as a duplicate of 97111 ***

Comment 2 Hrunting Johnson 2003-09-05 12:06:34 UTC
Please reread my bug report.  It was very detailed.  My problem is with the 
latest errata version of Apache and is not a duplicate of bug 97111 (for 
example, my script is not an NPH script; it is a normal script).

Maybe I should have noted at the top that this is not a duplicate of 97111.

Again, this bug only manifests itself in the latest -21.5 release of your 
package.  If I downgrade to -21.3 (the previous errata), the script works 
fine.  Note in my bug report that I specifically say that the server is not 
waiting to calculate Content-Length.  It's chunking the data as if it wants to 
send the data to the browser immediately.  It's just not sending the data.

Comment 3 Hrunting Johnson 2003-09-05 12:11:02 UTC
Correction:
I'm thinking of another bug with the NPH problem.

Still, this script is not waiting to calculate Content-Length (Content-Length 
is not being passed back in the result) and the problem is with the new 09/04 
errata, not a previous version of httpd.  My script cares not about the Content-
Length.

Comment 4 Hrunting Johnson 2003-09-11 17:38:23 UTC
Have you been able to reproduce this problem with the errata httpd package and 
included CGI script?  I can reproduce it every time.  With the older httpd 
package, the problem does not exist.

Comment 5 Joe Orton 2003-09-11 19:00:19 UTC
Yes, I've reproduced it locally, we're investigating the issue.

Thanks very much for the simple repro case, and my apologies for misdiagnosing
the dupe.

Comment 6 Joe Orton 2003-09-11 19:38:39 UTC
Yes, it's a regression caused by the fix for bug 97111 (swapping one bug for
another).  Packages which fix this can be found at:
http://people.redhat.com/jorton/9-httpd/

The fix will be integrated for the next erratum too.  Thanks for the report.

Comment 7 Hrunting Johnson 2003-11-02 00:59:20 UTC
That fix works for this specific problem, but at the same time, the 
CGI process doesn't get a SIGPIPE now when the remote end terminates 
its connection.

Comment 8 Joe Orton 2003-11-03 14:13:12 UTC
No, it will get a SIGTERM.  Are you seeing any problems with the -21.6
packages?

Comment 9 Hrunting Johnson 2003-11-03 14:16:52 UTC
Other than this issue where the process doesn't get any signal when 
the remote end closes its connection, no.  Normally, what I've seen 
is that when the remote end closes the connection, the process 
writing to client (the CGI process) gets a SIGPIPE.  It's not getting 
either signal, though, and continues running until the process it 
reads from terminates.

Comment 10 Joe Orton 2003-11-03 16:10:52 UTC
OK, thanks.  I've also integrated the fixes for that for the next
erratum.  Thanks for testing this out.

Comment 11 Hrunting Johnson 2003-11-03 17:31:59 UTC
Would you have another temporary release, or do you have an erratum 
planned for the next day or two (I know Apache released some security 
updates recently, so perhaps it's a non-issue)?

Comment 12 Joe Orton 2003-11-03 21:32:31 UTC
I'll put the new packages up on "people" when I have them ready for
testing.

Comment 13 Joe Orton 2003-11-04 14:32:37 UTC
Can you try out these packages?

http://people.redhat.com/jorton/9-httpd/



Comment 14 Hrunting Johnson 2003-11-04 14:39:59 UTC
Those packages appear to work excellently.

Comment 15 Mark J. Cox 2003-12-16 21:45:04 UTC
An errata has been issued which should help the problem described in this bug report. 
This report is therefore being closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files, please follow the link below. You may reopen 
this bug report if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2003-320.html