Bug 768394 - LWP::UserAgent cuts chunked response sent through HTTPS
Summary: LWP::UserAgent cuts chunked response sent through HTTPS
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: perl-Net-HTTP
Version: 17
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Petr Pisar
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 911713 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-12-16 14:51 UTC by Marcela Mašláňová
Modified: 2014-05-28 20:06 UTC (History)
11 users (show)

Fixed In Version: perl-Net-HTTP-6.05-3.fc19
Clone Of:
Environment:
Last Closed: 2013-04-16 14:57:19 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
Net-HTTP agent (1.12 KB, application/octet-stream)
2011-12-21 14:25 UTC, Petr Pisar
no flags Details
HTTP response (147.70 KB, text/plain)
2011-12-21 14:27 UTC, Petr Pisar
no flags Details


Links
System ID Private Priority Status Summary Last Updated
CPAN 72790 0 None None None Never
CPAN 80591 0 None None None Never
CPAN 80670 0 None None None Never
CPAN 81237 0 None None None Never
Red Hat Bugzilla 824140 1 None None None 2021-01-20 06:05:38 UTC

Internal Links: 824140

Description Marcela Mašláňová 2011-12-16 14:51:44 UTC
Test case:
GET 'https://bugzilla.redhat.com/show_bug.cgi?id=1' | tail

perl5.14.2 and higher:
perlbrew use perl-5.14.2
[marca@localhost skripty]$ GET 'https://bugzilla.redhat.com/show_bug.cgi?id=1' | tail

  <li class="form">
    <span class="separator">| </span>
    <form action="buglist.cgi" method="get"
        onsubmit="if (this.quicksearch.value == '')
                  { alert('Please enter one or more search terms first.');
                    return false; } return true;">
    <input class="txt" type="text" id="quicksearch_top" name="quicksearch" 
           value="">
    <input class="btn" type

perl5.12.4
perlbrew use perl-5.12.4
[marca@localhost perl]$ GET 'https://bugzilla.redhat.com/show_bug.cgi?id=1' | tail
                      Bug</a></li>
  
</ul>


<div class="navigation">
  
  <i><font color="#777777">First</font></i>
  <i><font color="#777777">Last</font></i>
  <i><font color="#777

In both cases is not printed out whole output.

Comment 1 Petr Pisar 2011-12-20 17:21:59 UTC
Resulting HTTP::Message contains headers:

Client-Aborted: die
Client-Transfer-Encoding: chunked
X-Died: Missing newline after chunk data: ' HTML output */' at /usr/share/perl5/vendor_perl/Net/HTTP/Methods.pm line 481.

Last body text is: "  /* Outputs a link to call replyToComment(); used to reduce"

So I guess there is a problem with decoding HTTP/1.1 chunk.

Comment 2 Petr Pisar 2011-12-21 14:25:02 UTC
Created attachment 549038 [details]
Net-HTTP agent

Running this Net-HTTP request through stunnel on localhost:2000 to redirect TLS to bugzilla, Net::HTTP does not raise any exception and complete body is returned to application.

Comment 3 Petr Pisar 2011-12-21 14:27:03 UTC
Created attachment 549039 [details]
HTTP response

This is response by Bugzilla on request:

GET /show_bug.cgi?id=1 HTTP/1.1
TE: deflate,gzip;q=0.3
Connection: TE, close
Host: bugzilla.redhat.com:443
User-Agent: libwww-perl/6.02

sent through gnutls-client. Fetching this response to LWP::UserAgent through netcat does not provoce the error.

Comment 4 Petr Pisar 2011-12-21 14:28:51 UTC
Maybe IO::Socket::SSL cuts the response to early. However I have not yet found how to debug LWP::UserAgent to reveal raw stream from IO::Socket::SSL engine, so I cannot confirm it.

Comment 5 Petr Pisar 2011-12-21 14:53:23 UTC
Comparing other distributions I found using Net-HTTP-6.01 LWP::UserAgent gets Bugzilla response correctly.

Comment 6 Paul Howarth 2011-12-21 14:57:27 UTC
(In reply to comment #4)
> Maybe IO::Socket::SSL cuts the response to early. However I have not yet found
> how to debug LWP::UserAgent to reveal raw stream from IO::Socket::SSL engine,
> so I cannot confirm it.

There was a data truncation issue with IO::Socket::SSL (Bug #751751) but that was fixed a month ago. Marcela, is your IO::Socket::SSL up to date?

Comment 7 Marcela Mašláňová 2011-12-21 15:24:03 UTC
(In reply to comment #6)
> (In reply to comment #4)
> > Maybe IO::Socket::SSL cuts the response to early. However I have not yet found
> > how to debug LWP::UserAgent to reveal raw stream from IO::Socket::SSL engine,
> > so I cannot confirm it.
> 
> There was a data truncation issue with IO::Socket::SSL (Bug #751751) but that
> was fixed a month ago. Marcela, is your IO::Socket::SSL up to date?

Yes, I have perl-IO-Socket-SSL-1.49-1.fc16.noarch.

Imho strange are different results with different releases of Perl. The problem is also on Debian: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=469350

Comment 8 Petr Pisar 2011-12-21 15:26:54 UTC
Reverting the only meaningful change between 6.01 and 6.02 "Don't disable blocking [RT#72580]"(208a2e384a176237a55c066755504cfcad1e5436) fixes this issue.

Comment 9 Sean Carlos 2012-08-13 12:22:30 UTC
I also encountered this problem in Fedora 17 downloading gzip files using www::mechanize. The resultant files are corrupt - using perl-Net-HTTP-6.02-2.fc17.noarch.rpm

Downgrading to perl-Net-HTTP-6.01-2.fc16.noarch.rpm fixed the problem

Comment 10 Fedora End Of Life 2013-01-17 00:08:20 UTC
This message is a reminder that Fedora 16 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 16. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '16'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 16's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 16 is end of life. If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora, you are encouraged to click on 
"Clone This Bug" and open it against that version of Fedora.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 11 Jan Pazdziora (Red Hat) 2013-01-17 09:16:32 UTC
I'm currently using

$ rpm -q perl-Net-HTTP
perl-Net-HTTP-6.02-2.fc17.noarch

and I get

$ GET 'https://bugzilla.redhat.com/show_bug.cgi?id=1' | tail
    var piwikTracker = Piwik.getTracker(pkBaseURL + "piwik.php", 2);
    piwikTracker.setCustomVariable(1, "bz_user", "");
    piwikTracker.trackPageView();
    piwikTracker.enableLinkTracking();
} catch( err ) {}
</script><noscript><p><img src="https://engstats.redhat.com/piwik/piwik.php?idsite=2" style="border:0" alt="" /></p></noscript>
<!-- End Piwik Tracking Code -->

</body>
</html>

So strangely enough it seems to work not, on Fedora 17. Do we know what fixed it?

Comment 12 Petr Pisar 2013-01-17 12:30:26 UTC
We don't know.

Comment 13 Claude Brown 2013-01-18 04:12:16 UTC
I also have experienced this bug in "Net::HTTP" in version 6.02 and after a lot of debugging over the past few days I found the problem.

Sometimes "Net::HTTP->read_entity_body" assumes that some data will be immediately available to "sysread" when it needs it.  For a non-blocking socket this isn't always true.

However, often the code gets away with it because the assumption is made when reading the "chunk data" just after the "chunk length". Web-servers will often send these things pretty quickly and so the assumption holds.

I had a look at Net::HTTP 6.05 and the problem has been resolved by ensuring that "select" is used to wait for the specified timeout period before giving up.

Comment 14 Claude Brown 2013-01-18 04:23:59 UTC
Here is the fix being found and fixed:

https://rt.cpan.org/Public/Bug/Display.html?id=80670

I feel very stupid spending the last few days debugging this when the problem has been known and resolved already!

Comment 15 Jan Pazdziora (Red Hat) 2013-01-18 07:24:12 UTC
Claude, you actually did a great job distilling the info for Fedora's benefit.

Since the last Fedora 17 perl-Net-HTTP version is perl-Net-HTTP-6.02-2.fc17, I'm flipping the version on this bugzilla to 17 because we have a reason to believe the issue is still present on Fedora 17 (even if as mentioned in comment 11 and explained in comment 13, it's not as deterministic as we'd love it to be).

Fedoras 18 and 19 have perl-Net-HTTP-6.05-1 thus they have the fix (https://rt.cpan.org/Public/Bug/Display.html?id=80670 mentions the fix is in 6.04).

What we'd need here is either patch for 6.02 to Fedora 17, or rebase of this package in Fedora 17 to 6.04 or higher. Petr, could you rebase the package?

Comment 16 Petr Pisar 2013-01-18 08:44:38 UTC
I will try the rebase. There are various fixes. I worry about time-out support a little but we will see.

Comment 17 Fedora Update System 2013-01-18 08:59:04 UTC
perl-Net-HTTP-6.05-1.fc17 has been submitted as an update for Fedora 17.
https://admin.fedoraproject.org/updates/perl-Net-HTTP-6.05-1.fc17

Comment 18 Claude Brown 2013-01-18 10:13:52 UTC
Yes, I think there may still be invalid assumptions about an entire chunk being available in a single "sysread". And I think the "read until EOF" is broken when the response has neither a "Content-Length" or a "Transfer-Encoded: chunked" header.

But overall I think this version is better than the one that went before with arbitrary "select(undef, undef, undef, 0.1)" timing hacks included.

I suspect a better approach would be to have several read operations (using "select" for timeout control) that fill a buffer to a given condition:
- until a \n is in the buffer
- until a specific number of bytes are in the buffer
- until EOF is reached

Such buffering would protect against any possible arrangement of "sysread" completion until the required data is available, or a timeout.  And then all returns are satisfied form the buffer.

But this was too big a change so I chickened out :(

Comment 19 Fedora Update System 2013-01-20 02:52:44 UTC
Package perl-Net-HTTP-6.05-1.fc17:
* should fix your issue,
* was pushed to the Fedora 17 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing perl-Net-HTTP-6.05-1.fc17'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-0984/perl-Net-HTTP-6.05-1.fc17
then log in and leave karma (feedback).

Comment 20 Fedora Update System 2013-02-05 03:02:37 UTC
perl-Net-HTTP-6.05-1.fc17 has been pushed to the Fedora 17 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 21 Petr Pisar 2013-02-18 09:07:37 UTC
*** Bug 911713 has been marked as a duplicate of this bug. ***

Comment 22 Petr Pisar 2013-02-18 09:08:29 UTC
And we have hit the dual problem now -- it blocks in select(2) for reading forever (bug #911713, URL <https://www134.safesecureweb.com/jrproperties/default.htm>).

Comment 23 Need Real Name 2013-03-05 20:05:02 UTC
Is there a way to get a replacement of perl-Net-HTTP
because perl-Net-HTTP looks very much unfixable and incompetently maintained.

Very standard task - read/write to sockets, avoiding deadlocks.
well implemented in any programming language except perl.

Can RedHat rewrite perl-Net-HTTP by just calling python or C code directly?

current perl-Net-HTTP is just dead end, to be scrapped and replaced by something that works.

Comment 24 Petr Pisar 2013-03-08 15:24:13 UTC
I believe patch referred from new upstream bug report <https://rt.cpan.org/Public/Bug/Display.html?id=81237> could improve this situation. Patched versions for all Fedoras will follow.

Comment 25 Fedora Update System 2013-03-08 15:47:42 UTC
perl-Net-HTTP-6.05-2.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/perl-Net-HTTP-6.05-2.fc18

Comment 26 Fedora Update System 2013-03-08 15:48:23 UTC
perl-Net-HTTP-6.05-2.fc17 has been submitted as an update for Fedora 17.
https://admin.fedoraproject.org/updates/perl-Net-HTTP-6.05-2.fc17

Comment 27 Fedora Update System 2013-03-11 11:54:10 UTC
perl-Net-HTTP-6.06-1.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/perl-Net-HTTP-6.06-1.fc18

Comment 28 Fedora Update System 2013-03-11 11:55:10 UTC
perl-Net-HTTP-6.06-1.fc17 has been submitted as an update for Fedora 17.
https://admin.fedoraproject.org/updates/perl-Net-HTTP-6.06-1.fc17

Comment 29 Fedora Update System 2013-03-20 21:39:12 UTC
perl-Net-HTTP-6.06-1.fc18 has been pushed to the Fedora 18 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 30 Fedora Update System 2013-03-27 00:48:10 UTC
perl-Net-HTTP-6.06-1.fc17 has been pushed to the Fedora 17 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 31 Juan 2014-05-28 19:59:37 UTC
I'm experiencing this same problem with perl-Net-HTTP 6.06-4.fc20; chunked transfer encoding is broken with https.

Comment 32 Juan 2014-05-28 20:06:45 UTC
Please ignore my previous comment. I was wrong.


Note You need to log in before you can comment on or make changes to this bug.