Bug 474162 - LWP sometimes truncates long lines fetched over https
LWP sometimes truncates long lines fetched over https
Product: Fedora
Classification: Fedora
Component: perl-libwww-perl (Show other bugs)
All Linux
low Severity medium
: ---
: ---
Assigned To: Marcela Mašláňová
Fedora Extras Quality Assurance
Depends On:
  Show dependency treegraph
Reported: 2008-12-02 10:41 EST by Ed Avis
Modified: 2009-01-29 18:01 EST (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2009-01-23 01:17:18 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Ed Avis 2008-12-02 10:41:07 EST
Fetching a page using LWP::UserAgent over https can truncate a long line, while the same page over http is fine.  Here is a recipe to reproduce the bug using only a Fedora installation (so both the web server and the client are on the same machine, which is running vanilla Fedora 10).  Please do try and see if you can reproduce the bug.

Install Fedora 10 (I used the i386 live CD and the 'install to local drive' option).  Change the firewall config so that HTTP and HTTPS connections are accepted and make sure Apache is running with SSL:

# yum install mod_ssl
# service httpd stop
# service httpd start

Now create a test file containing a long line of text:

# perl -E 'say "X" x 3059, " ", "X" x 21518' >/var/www/html/o.html

Note the filename o.html.  Not every filename triggers the bug.

Now you should be able to fetch <http://localhost/o.html> and <https://localhost/o.html> with Firefox or wget (after certificate warnings).  Of course, they both return the same content.  However, run this test program:

use warnings;
use strict;
use 5.010;
use LWP::UserAgent;
sub get_length {
    my $uri = shift;
    my $ua = LWP::UserAgent->new;
    die if defined $ua->max_size;
    my $req = new HTTP::Request(GET => $uri);
    my $res = (new LWP::UserAgent)->request($req);
    if (not $res->is_success) {
        my $err = $res->status_line;
        die "could not get Outlook web access page $uri:\n$err\n";
    my $c = $res->content;
    return length $c;
my $https_l = get_length "https://localhost/o.html";
my $http_l = get_length "http://localhost/o.html";
die "lengths differ: $https_l vs $http_l\n" if $https_l != $http_l;

You will see that when fetched via https using LWP::UserAgent, the file is truncated.

This is a reasonably minimal test case.  If you're unable to reproduce it, let me know and I will try to get a better one.  If you can reproduce it on a clean Fedora 10 system (as I could) then we can start investigating where the bug lies.  It must be in some Fedora-maintained package, since both the web server and client are on the local machine.  But I suspect that Apache is not at fault, since I have seen the same truncation bug fetching https pages from non-Apache web servers.
Comment 1 Ed Avis 2009-01-21 09:19:24 EST
Hi, do you think you could try to reproduce this using the recipe given?  Do you need any more information?
Comment 2 Marcela Mašláňová 2009-01-22 07:51:34 EST
I made an update, you can try it and let me know. I'm sorry it took so long. I hope the latest version will be working.

Comment 3 Ed Avis 2009-01-22 13:18:27 EST
I upgraded LWP using the new package you mention (actually, I rebuilt it from the SRPM) and I can confirm it fixes the bug.  Thanks!
Comment 4 Marcela Mašláňová 2009-01-23 01:17:18 EST
Thank you for your verification.
Comment 5 Fedora Update System 2009-01-29 18:01:06 EST
perl-libwww-perl-5.823-1.fc10 has been pushed to the Fedora 10 stable repository.  If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.