Bug 474162 - LWP sometimes truncates long lines fetched over https
Summary: LWP sometimes truncates long lines fetched over https
Keywords:
Status: CLOSED NEXTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: perl-libwww-perl
Version: 10
Hardware: All
OS: Linux
low
medium
Target Milestone: ---
Assignee: Marcela Mašláňová
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2008-12-02 15:41 UTC by Ed Avis
Modified: 2009-01-29 23:01 UTC (History)
3 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2009-01-23 06:17:18 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)

Description Ed Avis 2008-12-02 15:41:07 UTC
Fetching a page using LWP::UserAgent over https can truncate a long line, while the same page over http is fine.  Here is a recipe to reproduce the bug using only a Fedora installation (so both the web server and the client are on the same machine, which is running vanilla Fedora 10).  Please do try and see if you can reproduce the bug.

Install Fedora 10 (I used the i386 live CD and the 'install to local drive' option).  Change the firewall config so that HTTP and HTTPS connections are accepted and make sure Apache is running with SSL:

# yum install mod_ssl
# service httpd stop
# service httpd start

Now create a test file containing a long line of text:

# perl -E 'say "X" x 3059, " ", "X" x 21518' >/var/www/html/o.html

Note the filename o.html.  Not every filename triggers the bug.

Now you should be able to fetch <http://localhost/o.html> and <https://localhost/o.html> with Firefox or wget (after certificate warnings).  Of course, they both return the same content.  However, run this test program:

#!/usr/bin/perl
use warnings;
use strict;
use 5.010;
use LWP::UserAgent;
sub get_length {
    my $uri = shift;
    my $ua = LWP::UserAgent->new;
    die if defined $ua->max_size;
    my $req = new HTTP::Request(GET => $uri);
    my $res = (new LWP::UserAgent)->request($req);
    if (not $res->is_success) {
        my $err = $res->status_line;
        die "could not get Outlook web access page $uri:\n$err\n";
    }
    my $c = $res->content;
    return length $c;
}
my $https_l = get_length "https://localhost/o.html";
my $http_l = get_length "http://localhost/o.html";
die "lengths differ: $https_l vs $http_l\n" if $https_l != $http_l;

You will see that when fetched via https using LWP::UserAgent, the file is truncated.

This is a reasonably minimal test case.  If you're unable to reproduce it, let me know and I will try to get a better one.  If you can reproduce it on a clean Fedora 10 system (as I could) then we can start investigating where the bug lies.  It must be in some Fedora-maintained package, since both the web server and client are on the local machine.  But I suspect that Apache is not at fault, since I have seen the same truncation bug fetching https pages from non-Apache web servers.

Comment 1 Ed Avis 2009-01-21 14:19:24 UTC
Hi, do you think you could try to reproduce this using the recipe given?  Do you need any more information?

Comment 2 Marcela Mašláňová 2009-01-22 12:51:34 UTC
I made an update, you can try it and let me know. I'm sorry it took so long. I hope the latest version will be working.

https://admin.fedoraproject.org/updates/perl-libwww-perl-5.823-1.fc10

Comment 3 Ed Avis 2009-01-22 18:18:27 UTC
I upgraded LWP using the new package you mention (actually, I rebuilt it from the SRPM) and I can confirm it fixes the bug.  Thanks!

Comment 4 Marcela Mašláňová 2009-01-23 06:17:18 UTC
Thank you for your verification.

Comment 5 Fedora Update System 2009-01-29 23:01:06 UTC
perl-libwww-perl-5.823-1.fc10 has been pushed to the Fedora 10 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.