Test case: GET 'https://bugzilla.redhat.com/show_bug.cgi?id=1' | tail perl5.14.2 and higher: perlbrew use perl-5.14.2 [marca@localhost skripty]$ GET 'https://bugzilla.redhat.com/show_bug.cgi?id=1' | tail <li class="form"> <span class="separator">| </span> <form action="buglist.cgi" method="get" onsubmit="if (this.quicksearch.value == '') { alert('Please enter one or more search terms first.'); return false; } return true;"> <input class="txt" type="text" id="quicksearch_top" name="quicksearch" value=""> <input class="btn" type perl5.12.4 perlbrew use perl-5.12.4 [marca@localhost perl]$ GET 'https://bugzilla.redhat.com/show_bug.cgi?id=1' | tail Bug</a></li> </ul> <div class="navigation"> <i><font color="#777777">First</font></i> <i><font color="#777777">Last</font></i> <i><font color="#777 In both cases is not printed out whole output.
Resulting HTTP::Message contains headers: Client-Aborted: die Client-Transfer-Encoding: chunked X-Died: Missing newline after chunk data: ' HTML output */' at /usr/share/perl5/vendor_perl/Net/HTTP/Methods.pm line 481. Last body text is: " /* Outputs a link to call replyToComment(); used to reduce" So I guess there is a problem with decoding HTTP/1.1 chunk.
Created attachment 549038 [details] Net-HTTP agent Running this Net-HTTP request through stunnel on localhost:2000 to redirect TLS to bugzilla, Net::HTTP does not raise any exception and complete body is returned to application.
Created attachment 549039 [details] HTTP response This is response by Bugzilla on request: GET /show_bug.cgi?id=1 HTTP/1.1 TE: deflate,gzip;q=0.3 Connection: TE, close Host: bugzilla.redhat.com:443 User-Agent: libwww-perl/6.02 sent through gnutls-client. Fetching this response to LWP::UserAgent through netcat does not provoce the error.
Maybe IO::Socket::SSL cuts the response to early. However I have not yet found how to debug LWP::UserAgent to reveal raw stream from IO::Socket::SSL engine, so I cannot confirm it.
Comparing other distributions I found using Net-HTTP-6.01 LWP::UserAgent gets Bugzilla response correctly.
(In reply to comment #4) > Maybe IO::Socket::SSL cuts the response to early. However I have not yet found > how to debug LWP::UserAgent to reveal raw stream from IO::Socket::SSL engine, > so I cannot confirm it. There was a data truncation issue with IO::Socket::SSL (Bug #751751) but that was fixed a month ago. Marcela, is your IO::Socket::SSL up to date?
(In reply to comment #6) > (In reply to comment #4) > > Maybe IO::Socket::SSL cuts the response to early. However I have not yet found > > how to debug LWP::UserAgent to reveal raw stream from IO::Socket::SSL engine, > > so I cannot confirm it. > > There was a data truncation issue with IO::Socket::SSL (Bug #751751) but that > was fixed a month ago. Marcela, is your IO::Socket::SSL up to date? Yes, I have perl-IO-Socket-SSL-1.49-1.fc16.noarch. Imho strange are different results with different releases of Perl. The problem is also on Debian: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=469350
Reverting the only meaningful change between 6.01 and 6.02 "Don't disable blocking [RT#72580]"(208a2e384a176237a55c066755504cfcad1e5436) fixes this issue.
I also encountered this problem in Fedora 17 downloading gzip files using www::mechanize. The resultant files are corrupt - using perl-Net-HTTP-6.02-2.fc17.noarch.rpm Downgrading to perl-Net-HTTP-6.01-2.fc16.noarch.rpm fixed the problem
This message is a reminder that Fedora 16 is nearing its end of life. Approximately 4 (four) weeks from now Fedora will stop maintaining and issuing updates for Fedora 16. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as WONTFIX if it remains open with a Fedora 'version' of '16'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version prior to Fedora 16's end of life. Bug Reporter: Thank you for reporting this issue and we are sorry that we may not be able to fix it before Fedora 16 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged to click on "Clone This Bug" and open it against that version of Fedora. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. The process we are following is described here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
I'm currently using $ rpm -q perl-Net-HTTP perl-Net-HTTP-6.02-2.fc17.noarch and I get $ GET 'https://bugzilla.redhat.com/show_bug.cgi?id=1' | tail var piwikTracker = Piwik.getTracker(pkBaseURL + "piwik.php", 2); piwikTracker.setCustomVariable(1, "bz_user", ""); piwikTracker.trackPageView(); piwikTracker.enableLinkTracking(); } catch( err ) {} </script><noscript><p><img src="https://engstats.redhat.com/piwik/piwik.php?idsite=2" style="border:0" alt="" /></p></noscript> <!-- End Piwik Tracking Code --> </body> </html> So strangely enough it seems to work not, on Fedora 17. Do we know what fixed it?
We don't know.
I also have experienced this bug in "Net::HTTP" in version 6.02 and after a lot of debugging over the past few days I found the problem. Sometimes "Net::HTTP->read_entity_body" assumes that some data will be immediately available to "sysread" when it needs it. For a non-blocking socket this isn't always true. However, often the code gets away with it because the assumption is made when reading the "chunk data" just after the "chunk length". Web-servers will often send these things pretty quickly and so the assumption holds. I had a look at Net::HTTP 6.05 and the problem has been resolved by ensuring that "select" is used to wait for the specified timeout period before giving up.
Here is the fix being found and fixed: https://rt.cpan.org/Public/Bug/Display.html?id=80670 I feel very stupid spending the last few days debugging this when the problem has been known and resolved already!
Claude, you actually did a great job distilling the info for Fedora's benefit. Since the last Fedora 17 perl-Net-HTTP version is perl-Net-HTTP-6.02-2.fc17, I'm flipping the version on this bugzilla to 17 because we have a reason to believe the issue is still present on Fedora 17 (even if as mentioned in comment 11 and explained in comment 13, it's not as deterministic as we'd love it to be). Fedoras 18 and 19 have perl-Net-HTTP-6.05-1 thus they have the fix (https://rt.cpan.org/Public/Bug/Display.html?id=80670 mentions the fix is in 6.04). What we'd need here is either patch for 6.02 to Fedora 17, or rebase of this package in Fedora 17 to 6.04 or higher. Petr, could you rebase the package?
I will try the rebase. There are various fixes. I worry about time-out support a little but we will see.
perl-Net-HTTP-6.05-1.fc17 has been submitted as an update for Fedora 17. https://admin.fedoraproject.org/updates/perl-Net-HTTP-6.05-1.fc17
Yes, I think there may still be invalid assumptions about an entire chunk being available in a single "sysread". And I think the "read until EOF" is broken when the response has neither a "Content-Length" or a "Transfer-Encoded: chunked" header. But overall I think this version is better than the one that went before with arbitrary "select(undef, undef, undef, 0.1)" timing hacks included. I suspect a better approach would be to have several read operations (using "select" for timeout control) that fill a buffer to a given condition: - until a \n is in the buffer - until a specific number of bytes are in the buffer - until EOF is reached Such buffering would protect against any possible arrangement of "sysread" completion until the required data is available, or a timeout. And then all returns are satisfied form the buffer. But this was too big a change so I chickened out :(
Package perl-Net-HTTP-6.05-1.fc17: * should fix your issue, * was pushed to the Fedora 17 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing perl-Net-HTTP-6.05-1.fc17' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2013-0984/perl-Net-HTTP-6.05-1.fc17 then log in and leave karma (feedback).
perl-Net-HTTP-6.05-1.fc17 has been pushed to the Fedora 17 stable repository. If problems still persist, please make note of it in this bug report.
*** Bug 911713 has been marked as a duplicate of this bug. ***
And we have hit the dual problem now -- it blocks in select(2) for reading forever (bug #911713, URL <https://www134.safesecureweb.com/jrproperties/default.htm>).
Is there a way to get a replacement of perl-Net-HTTP because perl-Net-HTTP looks very much unfixable and incompetently maintained. Very standard task - read/write to sockets, avoiding deadlocks. well implemented in any programming language except perl. Can RedHat rewrite perl-Net-HTTP by just calling python or C code directly? current perl-Net-HTTP is just dead end, to be scrapped and replaced by something that works.
I believe patch referred from new upstream bug report <https://rt.cpan.org/Public/Bug/Display.html?id=81237> could improve this situation. Patched versions for all Fedoras will follow.
perl-Net-HTTP-6.05-2.fc18 has been submitted as an update for Fedora 18. https://admin.fedoraproject.org/updates/perl-Net-HTTP-6.05-2.fc18
perl-Net-HTTP-6.05-2.fc17 has been submitted as an update for Fedora 17. https://admin.fedoraproject.org/updates/perl-Net-HTTP-6.05-2.fc17
perl-Net-HTTP-6.06-1.fc18 has been submitted as an update for Fedora 18. https://admin.fedoraproject.org/updates/perl-Net-HTTP-6.06-1.fc18
perl-Net-HTTP-6.06-1.fc17 has been submitted as an update for Fedora 17. https://admin.fedoraproject.org/updates/perl-Net-HTTP-6.06-1.fc17
perl-Net-HTTP-6.06-1.fc18 has been pushed to the Fedora 18 stable repository. If problems still persist, please make note of it in this bug report.
perl-Net-HTTP-6.06-1.fc17 has been pushed to the Fedora 17 stable repository. If problems still persist, please make note of it in this bug report.
I'm experiencing this same problem with perl-Net-HTTP 6.06-4.fc20; chunked transfer encoding is broken with https.
Please ignore my previous comment. I was wrong.