Red Hat Bugzilla – Bug 103744
slow-running CGI scripts are buffered too much
Last modified: 2007-04-18 12:57:22 EDT
From Bugzilla Helper: User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5b) Gecko/20030901 Firebird/0.6.1+ Description of problem: When running a long-running CGI script, output is buffered until the CGI script completely finishes. Only when the CGI exits does the web server actually send the data. This worked fine under the previous httpd package (before the latest errata). I have verified via strace that the httpd process is reading in the data from the CGI process, but is not writing anything to the connected socket. The output once rendered is still of a chunked form and the server has not calculated a Content-Length, but it all gets sent at once instead of line-by-line as it comes in. No filters are configured on the output. The server just doesn't write the data until the process exits. Version-Release number of selected component (if applicable): httpd-2.0.40-21.5 How reproducible: Always Steps to Reproduce: 1. Run a long running CGI script with unbuffered output 2. 3. Actual Results: The CGI script's data is sent to the browser once the CGI script exits. Expected Results: The CGI script's data is sent to the browser as it is sent to the CGI pipe. Additional info: #!/usr/bin/perl use strict; use CGI; use constant TRACEROUTE => qw| /usr/sbin/traceroute -w 2 |; use constant BUF_LENGTH => 1024; local( $| ) = 1; my $q = CGI->new(); my $ip = $q->param('ip') || $ENV{'REMOTE_ADDR'}; my $out = $q->param('out'); exit 1 unless $ip; my $pid = open( TR, '-|' ); local $SIG{'PIPE'} = sub { kill( 'TERM', $pid ); } if $pid; do { open( STDERR, '>/dev/null' ); exec( TRACEROUTE, $out ? ( '-s', $out ) : (), $ip ); die( "could not exec traceroute: $!" ); } unless $pid; print $q->header( 'text/plain' ); my ( $buf, $length ); # do some extra processing on Linux because of the length line output # print $buf while ( $length = sysread( TR, $buf, BUF_LENGTH ) ) > 0; while ( ( $length = sysread( TR, $buf, BUF_LENGTH ) ) > 0 ) { $buf =~ s/(\d+)\.\d+ (.?s)/$1 $2/sg; print $buf; } close( TR ); exit 0;
This should be fixed if you up2date to the erratum which was just issued. Please reopen bug 97111 if you still have problems. *** This bug has been marked as a duplicate of 97111 ***
Please reread my bug report. It was very detailed. My problem is with the latest errata version of Apache and is not a duplicate of bug 97111 (for example, my script is not an NPH script; it is a normal script). Maybe I should have noted at the top that this is not a duplicate of 97111. Again, this bug only manifests itself in the latest -21.5 release of your package. If I downgrade to -21.3 (the previous errata), the script works fine. Note in my bug report that I specifically say that the server is not waiting to calculate Content-Length. It's chunking the data as if it wants to send the data to the browser immediately. It's just not sending the data.
Correction: I'm thinking of another bug with the NPH problem. Still, this script is not waiting to calculate Content-Length (Content-Length is not being passed back in the result) and the problem is with the new 09/04 errata, not a previous version of httpd. My script cares not about the Content- Length.
Have you been able to reproduce this problem with the errata httpd package and included CGI script? I can reproduce it every time. With the older httpd package, the problem does not exist.
Yes, I've reproduced it locally, we're investigating the issue. Thanks very much for the simple repro case, and my apologies for misdiagnosing the dupe.
Yes, it's a regression caused by the fix for bug 97111 (swapping one bug for another). Packages which fix this can be found at: http://people.redhat.com/jorton/9-httpd/ The fix will be integrated for the next erratum too. Thanks for the report.
That fix works for this specific problem, but at the same time, the CGI process doesn't get a SIGPIPE now when the remote end terminates its connection.
No, it will get a SIGTERM. Are you seeing any problems with the -21.6 packages?
Other than this issue where the process doesn't get any signal when the remote end closes its connection, no. Normally, what I've seen is that when the remote end closes the connection, the process writing to client (the CGI process) gets a SIGPIPE. It's not getting either signal, though, and continues running until the process it reads from terminates.
OK, thanks. I've also integrated the fixes for that for the next erratum. Thanks for testing this out.
Would you have another temporary release, or do you have an erratum planned for the next day or two (I know Apache released some security updates recently, so perhaps it's a non-issue)?
I'll put the new packages up on "people" when I have them ready for testing.
Can you try out these packages? http://people.redhat.com/jorton/9-httpd/
Those packages appear to work excellently.
An errata has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on the solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHSA-2003-320.html