|Summary:||pycurl error during yum|
|Product:||Red Hat Enterprise Linux 6||Reporter:||Jon Disnard <jdisnard>|
|Component:||curl||Assignee:||Kamil Dudka <kdudka>|
|Status:||CLOSED ERRATA||QA Contact:||Jiri Jaburek <jjaburek>|
|Version:||6.2||CC:||jjaburek, ovasik, prc|
|Target Milestone:||rc||Keywords:||EasyFix, Patch|
|Fixed In Version:||curl-7.19.7-31.el6||Doc Type:||Bug Fix|
|Doc Text:||Story Points:||---|
|Last Closed:||2013-02-21 10:09:41 UTC||Type:||Bug|
|oVirt Team:||---||RHEL 7.3 requirements from Atomic Host:|
|Cloudforms Team:||---||Target Upstream Version:|
Description Jon Disnard 2012-04-17 02:02:26 UTC
Description of problem: When performing yum operations sometimes the pycurl bindings error: # yum --noplugins clean all ; yum --noplugins repolist Cleaning repos: base updates Cleaning up Everything http://192.168.1.1:8181/yum/rhel/6.2/i386/repodata/repomd.xml: [Errno 14] PYCURL ERROR 18 - "transfer closed with outstanding read data remaining" [...] Manually using curl we are able to reproduce the problem: # curl -vv http://192.168.1.1:8181/yum/rhel/6.2/i386/repodata/repomd.xml [...] * transfer closed with outstanding read data remaining * Closing connection #0 curl: (18) transfer closed with outstanding read data remaining Version-Release number of selected component (if applicable): [root@bluesky ~]# rpm -q curl libcurl curl-7.19.7-26.el6_2.4.i686 libcurl-7.19.7-26.el6_2.4.i686 How reproducible: The problem is repeatable most of the time. The problem seems to be related to the order in which libcurl receives and processes the last few TCP/HTTP packets, so the issue is not always reproducible unless the other side (the httpd) can always order the packets just so. The problem is not reproducible in upstream version from Fedora 16 (currently 7.21.7-7 as of this writing). Steps to Reproduce: 1. YUM update with pycurl bindings to the poor httpd implementation (uhttpd in this case) 2. 3. Actual results: See the above error 18 from curl. Expected results: Expectation that utilities like python/yum that make use of libcurl to work resiliently without error. Additional info: The problem appears to be related to this debian bug report (contains dialog with upstream maintainers of libcurl): http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=572276
Comment 2 Kamil Dudka 2012-04-17 11:26:25 UTC
(In reply to comment #0) > The problem appears to be related to this debian bug report (contains dialog > with upstream maintainers of libcurl): > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=572276 This is said to be the upstream fix for the issue: https://github.com/bagder/curl/commit/7fd32ce Could you please check whether the fix solves your problem?
Comment 3 Kamil Dudka 2012-04-17 11:39:44 UTC
Created attachment 578004 [details] backport of upstream commit 7fd32ce
Comment 4 Jon Disnard 2012-04-17 16:40:41 UTC
> This is said to be the upstream fix for the issue: > > https://github.com/bagder/curl/commit/7fd32ce > > Could you please check whether the fix solves your problem? Have cloned the upstream github, checked-out the 7fd32ce version, built and installed that version. So far I've been unable to reproduce the problem using that specific version. My testing involves an infinite loop that breaks when curl returns error 18. This test has been running continuously for over an hour without problem. while : ; do /usr/local/bin/curl -v http://192.168.1.1:8181/yum/rhel/6.2/i386/repodata/repomd.xml ; case $? in 0) continue ;; *) break ;; esac ; done Using the system curl 7.19-7 located in /usr/bin/curl we are able to reproduce the problem using the same method very quickly. I feel safe in saying that this solves the problem. Hopefully we can get this back-ported. Thank you.
Comment 5 Kamil Dudka 2012-04-17 21:25:32 UTC
Thanks for the testing, but you tested curl-7_20_1~75, which has not much in common with the version we have in RHEL-6. Could you please test it with libcurl-7.19.7-26.el6_2.4 with the attached patch applied? I can prepare an SRPM for testing purposes if needed.
Comment 6 Jon Disnard 2012-04-18 05:12:02 UTC
Sorry about that, I mistakenly thought you wanted the specific upstream version tested. I installed the src.rpm curl-7.19.7-26.el6_2.4, created the patch, fixed up the spec file, and built the program. Unfortunately that did not solve the problem as my test case still fails using the new binary. Here is some info about what was built: # /root/rpmbuild/BUILDROOT/curl-7.19.7-26.el6.4.i386/usr/bin/curl -V curl 7.19.7 (i686-pc-linux-gnu) libcurl/7.19.7 NSS/126.96.36.199 zlib/1.2.3 libidn/1.18 libssh2/1.2.2 Protocols: tftp ftp telnet dict ldap ldaps http file https ftps scp sftp Features: GSS-Negotiate IDN IPv6 Largefile NTLM SSL libz The above features your attached patch on this bz. Here is my test: while /root/rpmbuild/BUILDROOT/curl-7.19.7-26.el6.4.i386/usr/bin/curl http://192.168.1.1:8181/yum/rhel/6.2/i386/repodata/repomd.xml > /dev/null ; do continue ; done [... about 4 or 5 loop cycles] curl: (18) transfer closed with outstanding read data remaining What kills me is that I built another binary from the same curl-7.19.7-26.el6_2.4.src.rpm that does work every single time, and the above loop repeats forever. I just installed that src.rpm, ran buildconf, configure, and make install. The resulting binary in /usr/local/bin/curl doesn't support many features but it works. Please note this binary works with the transfer.c manually patched or left intact. # /usr/local/bin/curl -V curl 7.19.7 (i686-pc-linux-gnu) libcurl/7.20.1-DEV Protocols: dict file ftp http imap pop3 rtsp smtp telnet tftp Features: IPv6 Largefile To make it even more confusing, when I start to use ftrace in my testing then for some reason curl never fails.
Comment 7 Kamil Dudka 2012-04-18 07:22:14 UTC
(In reply to comment #6) > Here is my test: > > while /root/rpmbuild/BUILDROOT/curl-7.19.7-26.el6.4.i386/usr/bin/curl > http://192.168.1.1:8181/yum/rhel/6.2/i386/repodata/repomd.xml > /dev/null ; > do continue ; > done This ^^^ is not sufficient for the patch to take an effect. You are most likely running a new version of the tool (which does not really differ from the old one) against the old version of the library installed on your system. Please run this command prior to your testing: > export LD_PRELOAD=$(find /root/rpmbuild/BUILDROOT/curl-7.19.7-26.el6.4.i386 -name libcurl.so) ... and double-check that the patched library is being loaded: > ldd /root/rpmbuild/BUILDROOT/curl-7.19.7-26.el6.4.i386/usr/bin/curl
Comment 8 Jon Disnard 2012-04-18 13:55:13 UTC
After using LD_PRELOAD the test ran without fail. Thank you.
Comment 9 Kamil Dudka 2012-04-18 14:33:50 UTC
Great, thanks for testing it!
Comment 12 RHEL Program Management 2012-10-12 11:31:04 UTC
This request was evaluated by Red Hat Product Management for inclusion in a Red Hat Enterprise Linux release. Product Management has requested further review of this request by Red Hat Engineering, for potential inclusion in a Red Hat Enterprise Linux release for currently deployed products. This request is not yet committed for inclusion in a release.
Comment 18 errata-xmlrpc 2013-02-21 10:09:41 UTC
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHBA-2013-0393.html