Bug 643656
Summary: | some FTP servers hang if curl closes the data connection too early | ||
---|---|---|---|
Product: | [Fedora] Fedora | Reporter: | Pasi Karkkainen <pasik> |
Component: | curl | Assignee: | Kamil Dudka <kdudka> |
Status: | CLOSED ERRATA | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
Severity: | medium | Docs Contact: | |
Priority: | low | ||
Version: | 14 | CC: | awilliam, daniel, dcantrell, jlaska, jsmith.fedora, kdudka, notting, paul |
Target Milestone: | --- | ||
Target Release: | --- | ||
Hardware: | All | ||
OS: | Linux | ||
Whiteboard: | |||
Fixed In Version: | curl-7.20.1-5.fc13 | Doc Type: | Bug Fix |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2010-11-28 20:41:49 UTC | Type: | --- |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: | |||
Attachments: |
Description
Pasi Karkkainen
2010-10-16 20:23:49 UTC
More info about the Fedora Anaconda installation failures here: https://bugzilla.redhat.com/show_bug.cgi?id=624431 Basicly Anaconda wants to skip the headers of rpm files using the range method, which makes curl use the ftp REST command, and thus triggers this bug and installation fails. It looks like a server bug to me. The server cowardly refuses to respond if we close the data channel before the server has sent entire file. It does not seem to be anyhow related to REST. $ curl -svo/dev/null --range 0-7 ftp://ftp.funet.fi/pub/mirrors/fedora.redhat.com/pub/fedora/linux/releases/13/Fedora/x86_64/os/Packages/iputils-20071127-10.fc13.x86_64.rpm ... * DO phase is complete * DO-MORE phase starts * FTP RANGE from 0 getting 8 bytes * range-download from 0 to 7, totally 8 bytes > TYPE I * FTP 0x62e4c8 state change from STOP to RETR_TYPE < 200 TYPE is now 8-bit binary > SIZE iputils-20071127-10.fc13.x86_64.rpm * FTP 0x62e4c8 state change from RETR_TYPE to RETR_SIZE < 213 121012 > RETR iputils-20071127-10.fc13.x86_64.rpm * FTP 0x62e4c8 state change from RETR_SIZE to RETR < 150-Accepted data connection < 150 118.2 kbytes to download * Maxdownload = 8 * Getting file with size: 8 * FTP 0x62e4c8 state change from RETR to STOP * DO-MORE phase ends with 0 { [data not shown] * Remembering we are in dir "pub/mirrors/fedora.redhat.com/pub/fedora/linux/releases/13/Fedora/x86_64/os/Packages/" Then it continues to hang with the following backtrace: #0 0x0000003a112d7e13 in poll () from /lib64/libc.so.6 #1 0x00007ffff7dba310 in Curl_socket_ready (readfd=7, writefd=-1, timeout_ms=1000) at select.c:217 #2 0x00007ffff7d83f14 in Curl_GetFTPResponse (nreadp=0x7fffffffd8c8, conn=0x62dff8, ftpcode=0x7fffffffd8f8) at ftp.c:505 #3 0x00007ffff7d89063 in ftp_done (conn=0x62dff8, status=CURLE_OK, premature=false) at ftp.c:3113 #4 0x00007ffff7d97748 in Curl_done (connp=0x7fffffffd990, status=CURLE_OK, premature=false) at url.c:5167 #5 0x00007ffff7da2722 in Curl_do_perform (data=0x623568) at transfer.c:2118 #6 0x00007ffff7da287b in Curl_perform (data=0x623568) at transfer.c:2188 #7 0x00007ffff7da33af in curl_easy_perform (curl=0x623568) at easy.c:567 #8 0x0000000000408e0f in operate (argc=0, argv=<value optimized out>) at main.c:5445 #9 main (argc=0, argv=<value optimized out>) at main.c:5775 If I stop curl for a second just before closing the data connection, the server is happy: (gdb) break ftp.c:3095 (gdb) run Breakpoint 1, ftp_done (conn=0x62dff8, status=CURLE_OK, premature=false) at ftp.c:3095 3095 sclose(conn->sock[SECONDARYSOCKET]); (gdb) list 3090 3091 /* Note that we keep "use" set to TRUE since that (next) connection is 3092 still requested to use SSL */ 3093 } 3094 if(CURL_SOCKET_BAD != conn->sock[SECONDARYSOCKET]) { 3095 sclose(conn->sock[SECONDARYSOCKET]); 3096 conn->sock[SECONDARYSOCKET] = CURL_SOCKET_BAD; 3097 } 3098 } 3099 (gdb) continue < 226-File successfully transferred < 226 0.288 seconds (measured here), 410.81 Kbytes per second * Connection #0 to host ftp.funet.fi left intact > QUIT * FTP 0x62e4c8 state change from STOP to QUIT < 221-Goodbye. You uploaded 0 and downloaded 119 kbytes. < 221 Logout. * FTP 0x62e4c8 state change from QUIT to STOP * Closing connection #0 Thanks for looking at it! What do you suggest to do about it? I'm sure ftp.funet.fi is not the only server running pure-ftpd. I've tried every ftp client I can think of, and I've only seen this problem with curl.. wget, firefox, lftp, ftp, ncftp, etc all work just fine with ftp.funet.fi. Some kind of workaround in curl would be nice.. (In reply to comment #3) > I've tried every ftp client I can think of, and I've only seen this problem > with curl.. wget, firefox, lftp, ftp, ncftp, etc all work just fine with > ftp.funet.fi. How can I limit amount of the data to download in those clients? Well, I don't know if you can limit it actually with those clients. So ok, it might be just a broken server, but still workaround in the client would be nice.. Especially when this issue makes people unable to install Fedora from the official mirror! Sure, workaround would be nice. That's why I am asking how other clients solve the problem. So far I am not aware of any existing solution. The following two choices come into consideration: 1) Download more data than we actually need, in order to get the response from server. This can be as well achieved at application level. 2) Do not wait for response from server if we're downloading only part of file and already have received enough data, and move forward to QUIT command immediately. Such a solution is faster than 1) but isn't RFC compliant and therefore should not be enabled by default. We would need to introduce a new option for it. 3) Any other ideas? Yeah, or just download the whole file in Anaconda installer. Skipping 1483 bytes per rpm doesn't save much when we're installing the whole Fedora distro anyway. I don't know how likely it is to get the Anaconda people to change that behaviour though.. Thinking more about this. During Fedora installation anaconda wants to skip the header from the rpm files, and thus it issues REST command to start the download from specific offset (header size). All of the file after that offset will be downloaded. So there's no end of the range specified. If you test with range "0-8" you get a different problem, no? I tried to play with wget. I downloaded 10 MB from a 100MB file, and then hit ctrl+c to quit the download. Then I did "wget -c" to continue the download, which makes wget use REST to continue from the offset it already has downloaded, but that seemed to work OK without problems. In the wireshark capture of the Fedora Anaconda FTP session I can see it starts the download from offset 1483 with REST, and then it fails after one minute (timeout). That's the case that needs to be worked around. Hopefully I didn't miss something. As I said, REST is not problem here. Problem is that data channel is closed before the server sends what it wanted to send. Pick your original example and remove the upper bound from there: "--range 1384-". Then it works in the same way as wget works. Once again, skipping first N bytes of a file does not seem to be a problem here. Ok, please forget my example, since that's not the exact case what happens with Anaconda! I was just trying to make up a similar easy-to-reproduce example. Here's actual wireshark capture of the failing Anaconda/curl session: http://pasik.reaktio.net/fedora/f13-install-port21.thark Per my understanding it's trying to download the rpm, just skipping first N bytes (the header). And I forgot to mention you should go to the end of that wireshark capture.. the failing rpm (iputils) is there. Skipping first N bytes works for me with curl and the mentioned FTP server. Does Anaconda anyhow limit the amount of data to download? Indeed: curl -svo/dev/null --range 1384-121012 ftp://ftp.funet.fi/pub/mirrors/fedora.redhat.com/pub/fedora/linux/releases/13/Fedora/x86_64/os/Packages/iputils-20071127-10.fc13.x86_64.rpm Seems to work. 121012 is the size of the file in question. So anaconda must do something else.. A possible fix for curl that isn't too shabby would be to send an ABOR command when it has closed the data transfer, as that explicitly instructs the server to close the data connection. Daniel, thanks for looking at the issue. Could we rely on the support of ABOR by server? I know it's in RFC959, but it might cause problems with some other FTP servers. Is it safe to use it by default for range FTP transfers? Created attachment 453972 [details] fix proposed by Daniel Stenberg The attached patch solves the problem from comment #0 and does not seem to break anything else at first glance... curl hasn't used ABOR before so I've not really experienced its use in a wide scale deployment but I'm not aware of any documented or known flaws with it. Pasi, are you able to verify that the patch solves the Anaconda problem? Good question. I guess testing requires building custom Fedora installer with the patched curl included. Or I wonder if I could replace the curl libs from a running anaconda session.. Is there an easier way to try patched curl with anaconda? It should be possible to create a floppy image and pass the "update" during boot. Chris, is it documented somewhere? s/update/updates/ http://fedoraproject.org/wiki/Anaconda/Updates Just create a disk image with the .so files in the root of it, and anaconda will deal with the rest. Created attachment 454179 [details]
lzma compressed updates floppy image for Fedora 14 Anaconda
Thanks Chris. That's exactly what I was looking for. Attached is a floppy image with patched libcurl (x86_64). I didn't test it myself yet...
Proposing as NTH in case we want to pull this into RC1. Can everyone please vote on whether they think we should or not? -- Fedora Bugzappers volunteer triage team https://fedoraproject.org/wiki/BugZappers Created attachment 454433 [details]
gzip compressed cpio archive with patched libcurl (x86_64)
I had created FAT image, which Anaconda is not able to mount. Will try the recommended cpio archive instead...
I'm able to try it tomorrow.. maybe others can try it before that. when I asked about NTH status in #anaconda , clumens said 'seems reasonable'. any other votes? -- Fedora Bugzappers volunteer triage team https://fedoraproject.org/wiki/BugZappers I want to test this, but I'm not sure how: funet.fi doesn't seem to have the stage2 images for F14 on it anywhere, so how can I set up an FTP install of F14 from funet.fi to test? -- Fedora Bugzappers volunteer triage team https://fedoraproject.org/wiki/BugZappers There's F14 beta: ftp://ftp.funet.fi/pub/mirrors/fedora.redhat.com/pub/fedora/linux/releases/test/14-Beta/Fedora/x86_64/os/ Does that help? i'm not sure if anaconda 14.19 stage1 can live with stage2 from whatever anaconda we had on 14 Beta, but I can try, I guess... -- Fedora Bugzappers volunteer triage team https://fedoraproject.org/wiki/BugZappers Created attachment 454451 [details]
gzip compressed cpio archive with patched libcurl (x86_64)
The cpio archive has to be created with 'cpio -H newc', so that Anaconda is able to digest it...
So far I can't confirm the Anaconda problem is fixed. It gives me "(28) FTP response timeout" while downloading ftp://ftp.funet.fi/pub/mirrors/fedora.redhat.com/pub/fedora/linux/releases/test/14-Beta/Fedora/x86_64/os/repodata/repomd.xml in both cases. If I switch to console and fetch the same URL with urlgrabber, it works just fine. I suspect that Anaconda refuses to use my patched library for some reason, but have no idea how to check if it does. Created attachment 454454 [details]
gzip compressed cpio archive with patched libcurl (x86_64)
Created attachment 454456 [details]
/tmp/anaconda.log from my attempt
I just burned F14 Beta x86_64 CD1 to a CDR, and booted with "linux askmethod" and tried installing using URL ftp://ftp.funet.fi/pub/mirrors/fedora.redhat.com/pub/fedora/linux/releases/test/14-Beta/Fedora/x86_64/os/ . The installation failed in the usual way, couldn't fetch rpms from the server. Then I tried the updates.img from comment #32, but anaconda isn't able to mount it. There's an error about unsupported/invalid filesystem type. I tried converting that to ext2 image, and then anaconda is able to mount it, but it doesn't seem to copy the included files to the system.. Should there be usr/lib64/ directory on the image, and the .so files in that directory? /mnt/runtime seems to be mounted read-only, so I cannot manually copy the libs there during installation.. (In reply to comment #34) > Then I tried the updates.img from comment #32, but anaconda isn't able to mount > it. There's an error about unsupported/invalid filesystem type. That's just an error that Anaconda issues to scary users. It worked for me (means unpacked the files) even with that error in the log, as you can see in the attachment #454456 [details] above. > Should there be usr/lib64/ directory on the image, and the .so files in that > directory? Nope, they were in /tmp/updates and the directory was included in LD_LIBRARY_PATH. Hmm, true, files are copied to /tmp/updates/ . running "export" in the shell doesn't show /tmp/updates in LD_LIBRARY_PATH though.. ? Maybe it's different inside the anaconda/python session. I noticed one difference when the updates.img is in use! Anaconda still fails to fetch the rpms, but clicking Retry actually fixes it! It's just a bit annoying to click Retry for every rpm :) Dunno why it doesn't work on the first try.. Anyway, I think that behaviour is different from without updates.img.. there it couldn't fetch the rpms even if I clicked Retry. Running "lsof -n | grep -i curl" shows that anaconda actually is using the library from /tmp/updates/, so that's fine. After clicking Retry around 20 times I got to an rpm that now fails and Retry doesn't help.. so I guess there's still a problem with curl.. It would be good to have a minimal example. It's nearly impossible to debug the Anaconda code in place. The original problem reported in comment #0 is addressed by the update, isn't it? My guess is that Anaconda is doing something different than comment #0 suggests. Rpm fetch errors happen during the "Starting installation process" phase.. clicking Retry for every error seemed to work this time.. I had to click Retry *many* times.. and then F14 installed OK. I have a tshark capture of the installation session here: http://pasik.reaktio.net/fedora/f14/ftp-problem/f14-test1-funet.tshark And shorter with only port 21 traffic: http://pasik.reaktio.net/fedora/f14/ftp-problem/f14-test1-port21.tshark Take a look at that port21.tshark .. search for "rsyslog-4.6.3-1.fc14.x86_64.rpm". First anaconda fetched it starting from offset 1384, and later the full file starting from the beginning .. Also search for "iputils-20100418-3.fc14.x86_64.rpm", or for "iproute-2.6.35-2.fc14.x86_64.rpm". First: 12.370205 192.168.0.100 -> 193.166.3.2 FTP Request: MDTM iproute-2.6.35-2.fc14.x86_64.rpm And then: 13.533264 192.168.0.100 -> 193.166.3.2 FTP Request: MDTM iproute-2.6.35-2.fc14.x86_64.rpm 13.562864 192.168.0.100 -> 193.166.3.2 FTP Request: SIZE iproute-2.6.35-2.fc14.x86_64.rpm 13.568473 193.166.3.2 -> 192.168.0.100 FTP Response: 213 359116 13.568660 192.168.0.100 -> 193.166.3.2 FTP Request: REST 1384 13.574364 193.166.3.2 -> 192.168.0.100 FTP Response: 350 Restarting at 1384 13.574548 192.168.0.100 -> 193.166.3.2 FTP Request: RETR iproute-2.6.35-2.fc14.x86_64.rpm 13.580205 193.166.3.2 -> 192.168.0.100 FTP Response: 150-Accepted data connection 13.593034 192.168.0.100 -> 193.166.3.2 FTP Request: ABOR 13.618404 193.166.3.2 -> 192.168.0.100 FTP Response: 150 0.038 seconds (measured here), 3.30 Mbytes per second 13.618420 193.166.3.2 -> 192.168.0.100 FTP Response: 226 Since you see this ABOR must've succeeded And then later: 149.238505 192.168.0.100 -> 193.166.3.2 FTP Request: MDTM iproute-2.6.35-2.fc14.x86_64.rpm 149.394120 193.166.3.2 -> 192.168.0.100 FTP Response: 213 20100818191014 149.394385 192.168.0.100 -> 193.166.3.2 FTP Request: PASV 149.405007 193.166.3.2 -> 192.168.0.100 FTP Response: 227 Entering Passive Mode (193,166,3,2,153,123) 149.411402 192.168.0.100 -> 193.166.3.2 FTP Request: SIZE iproute-2.6.35-2.fc14.x86_64.rpm 149.417034 193.166.3.2 -> 192.168.0.100 FTP Response: 213 359116 149.417234 192.168.0.100 -> 193.166.3.2 FTP Request: RETR iproute-2.6.35-2.fc14.x86_64.rpm 149.422974 193.166.3.2 -> 192.168.0.100 FTP Response: 150-Accepted data connection 149.466546 193.166.3.2 -> 192.168.0.100 FTP Response: 226-File successfully transferred I wonder if it's that first MDTM that fails and makes Anaconda present that Retry/reboot dialog? I was clicking Retry pretty quickly so the timestamps match with that.. I can't actually remember for sure if I got error dialog about exactly that iproute-2.6.35-2.fc14.x86_64.rpm but probably, since most rpms failed during the "Starting installation process" phase.. (In reply to comment #39) > Rpm fetch errors happen during the "Starting installation process" phase.. You were definitely further. In my case, Anaconda didn't let me to even add the repo. > I wonder if it's that first MDTM that fails and makes Anaconda present that What exactly you mean by "MDTM fails"? Is the error specific to Anaconda? Could it be somehow reproduced with yum only? Attempting to separate out the downloading code is an exercise in madness. What I usually do is first try things using urlgrabber from within python on tty2 and if that fails to show the problem, I start stepping through the code myself. It's extremely tedious. As stated in comment #31 , I already tried urlgrabber on tty2. I picked up the URL that Anaconda GUI was complaining about and gave it to urlgrabber. It worked fine on tty2. Any other ways to debug it? I also tried to set ftp://ftp.funet.fi as update repository on my workstation and perform 'yum update'. It successfully downloaded bunch of files from there without any problems. Kamil: Yeah actually I'm not sure if that "MDTM" fails, but at least that's executed twice.. and time diffence matches what it takes for me to click "Retry". So I was just speculating.. Chris: wget/curl/urlgrabber works OK, because those grab the whole file when you ran them from cmdline. If you check the wireshark capture above (comment 39) you can see the "REST 1384" stuff etc.. so anaconda seems to do various "non-basic" things.. anaconda downloads the package headers before installing the packages in order to do transaction checks, but so does yum. anaconda has very little special downloading code in it - most things are just pass-through to yum or handled by yum entirely. These failures have *never* happened with yum in an already installed system, but they're 100% reproducible with anaconda installer. I don't really know what's the difference.. I tried installing F14 Beta again and now it failed in a different way. During the "Starting installation process" phase I get error about rsyslog and iputils rpms, but both of those worked after clicking Retry button. Then selinux-policy-targeted rpm failed, but Retry didn't help for that one.. I clicked Retry multiple times but anaconda couldn't fetch it. So now the logs are shorter. Maybe someone understands what's happening. "selinux-policy-targeted-3.9.3-1.fc14.noarch.rpm" is the failing rpm that couldn't be fetched, even after multiple Retry attemps. tshark/wireshark capture: http://pasik.reaktio.net/fedora/f14/ftp-problem/f14-test2-all.tshark And shorter version with only port 21 traffic: http://pasik.reaktio.net/fedora/f14/ftp-problem/f14-test2-port21.tshark I boot the test-box from F14 Beta x86_64 CD1 by appending "askmethod updates=http://server/updates.img" to the default boot options. I choose URL install and use ftp://ftp.funet.fi/pub/mirrors/fedora.redhat.com/pub/fedora/linux/releases/test/14-Beta/Fedora/x86_64/os/ as the install URL. Should be easy to reproduce by others. I'm using DHCP for Internet access. Yum doesn't download the pkg headers to do file conflict checking. It just downloads the pkgs. Anaconda downloads the headers and then downloads the pkgs when it needs them. That is, almost undoubtedly, where the difference lies. B/c the pkg hdr is a byte-range. My point is that the header downloading code is not in anaconda. From pyanaconda/yuminstall.py: def downloadHeader(self, po): while True: # retrying version of download header try: YumSorter.downloadHeader(self, po) break except yum.Errors.NoMoreMirrorsRepoError: self._handleFailure(po) except IOError: self._handleFailure(po) except yum.Errors.RepoError, e: continue where YumSorter is just yum.YumBase. Yes - I wasn't saying it was anaconda-specific. I was saying the code which triggers this is only ever exercised by anaconda. Pasi - I can show you what to call from your system via yum to test the same code path. Seth: Ok, feel free :) (In reply to comment #50) > Pasi - I can show you what to call from your system via yum to test the same > code path. Great! That would be much appreciated. I am also interested. TIA. from python or ipython run this (as root) import yum my = yum.YumBase() pkg = my.pkgSack.searchNevra(name='zziplib')[0] # replace zziplib with whatever my.downloadHeader(pkg) print(pkg.localHdr()) Here we go. This is on Fedora 13, no extra patches added to curl. I modified /etc/yum.repos.d/fedora.repo and fedora-updates.repo to only use ftp://ftp.funet.fi mirror. [root@f13 ~]# python Python 2.6.4 (r264:75706, Jun 4 2010, 18:20:31) [GCC 4.4.4 20100503 (Red Hat 4.4.4-2)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import yum >>> my = yum.YumBase() >>> pkg = my.pkgSack.searchNevra(name='iputils')[0] Loaded plugins: refresh-packagekit >>> my.downloadHeader(pkg) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 1917, in downloadHeader raise Errors.RepoError, saved_repo_error yum.Errors.NoMoreMirrorsRepoError: failure: Packages/iputils-20071127-10.fc13.x86_64.rpm from fedora: [Errno 256] No more mirrors to try. >>> print(pkg.localHdr()) /var/cache/yum/x86_64/13/fedora/headers/iputils-20071127-10.fc13.x86_64.hdr >>> [root@f13 ~]# "my.downloadHeader(pkg)" takes around 1 minute before it errors out. wireshark/tshark capture of the failing session here: http://pasik.reaktio.net/fedora/f14/ftp-problem/f13-testyum-all.tshark okay - so we should be able to drill down one layer deeper and make this request happen SOLELY with urlgrabber. I'm going to see if I can make something wonky happen and post the trial code for you to repeat it. thanks okay try this in ipython import urlgrabber ug = urlgrabber.grabber.URLGrabber() help(ug.urlgrab) ug.urlgrab(url='ftp://ftp.funet.fi/ftp/pub/mirrors/ftp.redhat.com/pub/fedora/linux/releases/13/Everything/x86_64/os/Packages/iputils-20071127-10.fc13.x86_64.rpm', reget=None, start=1384, end=18096, size=16712) I did and i got: URLGrabError: [Errno 14] Downloaded more than max size for : 18824 > 16712 Thanks! Could you please explain more verbosely what the code is supposed to do? It does not seem to use CURLOPT_RANGE at all: gdb -q --args python bz643656.py (gdb) break curl_easy_setopt Breakpoint 1 at 0x3db99a8: file easy.c, line 397. (gdb) run [Thread debugging using libthread_db enabled] Breakpoint 1, curl_easy_setopt (curl=0x82604c0, tag=CURLOPT_ERRORBUFFER) at easy.c:397 397 { (gdb) cont Breakpoint 1, curl_easy_setopt (curl=0x82604c0, tag=CURLOPT_PRIVATE) at easy.c:397 397 { (gdb) Breakpoint 1, curl_easy_setopt (curl=0x82604c0, tag=CURLOPT_NOPROGRESS) at easy.c:397 397 { (gdb) Breakpoint 1, curl_easy_setopt (curl=0x82604c0, tag=CURLOPT_VERBOSE) at easy.c:397 397 { (gdb) Breakpoint 1, curl_easy_setopt (curl=0x82604c0, tag=CURLOPT_FTP_ACCOUNT) at easy.c:397 397 { (gdb) Breakpoint 1, curl_easy_setopt (curl=0x82604c0, tag=CURLOPT_USERAGENT) at easy.c:397 397 { (gdb) Breakpoint 1, curl_easy_setopt (curl=0x82604c0, tag=CURLOPT_NOPROGRESS) at easy.c:397 397 { (gdb) Breakpoint 1, curl_easy_setopt (curl=0x82604c0, tag=CURLOPT_NOSIGNAL) at easy.c:397 397 { (gdb) Breakpoint 1, curl_easy_setopt (curl=0x82604c0, tag=CURLOPT_WRITEFUNCTION) at easy.c:397 397 { (gdb) Breakpoint 1, curl_easy_setopt (curl=0x82604c0, tag=CURLOPT_FILE) at easy.c:397 397 { (gdb) Breakpoint 1, curl_easy_setopt (curl=0x82604c0, tag=CURLOPT_HEADERFUNCTION) at easy.c:397 397 { (gdb) Breakpoint 1, curl_easy_setopt (curl=0x82604c0, tag=CURLOPT_WRITEHEADER) at easy.c:397 397 { (gdb) Breakpoint 1, curl_easy_setopt (curl=0x82604c0, tag=CURLOPT_PROGRESSFUNCTION) at easy.c:397 397 { (gdb) Breakpoint 1, curl_easy_setopt (curl=0x82604c0, tag=CURLOPT_PROGRESSDATA) at easy.c:397 397 { (gdb) Breakpoint 1, curl_easy_setopt (curl=0x82604c0, tag=CURLOPT_FAILONERROR) at easy.c:397 397 { (gdb) Breakpoint 1, curl_easy_setopt (curl=0x82604c0, tag=CURLOPT_FILETIME) at easy.c:397 397 { (gdb) Breakpoint 1, curl_easy_setopt (curl=0x82604c0, tag=CURLOPT_FOLLOWLOCATION) at easy.c:397 397 { (gdb) Breakpoint 1, curl_easy_setopt (curl=0x82604c0, tag=CURLOPT_USERAGENT) at easy.c:397 397 { (gdb) Breakpoint 1, curl_easy_setopt (curl=0x82604c0, tag=CURLOPT_FOLLOWLOCATION) at easy.c:397 397 { (gdb) Breakpoint 1, curl_easy_setopt (curl=0x82604c0, tag=CURLOPT_MAXREDIRS) at easy.c:397 397 { (gdb) Breakpoint 1, curl_easy_setopt (curl=0x82604c0, tag=CURLOPT_CONNECTTIMEOUT) at easy.c:397 397 { (gdb) Breakpoint 1, curl_easy_setopt (curl=0x82604c0, tag=CURLOPT_LOW_SPEED_LIMIT) at easy.c:397 397 { (gdb) Breakpoint 1, curl_easy_setopt (curl=0x82604c0, tag=CURLOPT_LOW_SPEED_TIME) at easy.c:397 397 { (gdb) Breakpoint 1, curl_easy_setopt (curl=0x82604c0, tag=CURLOPT_URL) at easy.c:397 397 { (gdb) [New Thread 0xb7d9db70 (LWP 4318)] [Thread 0xb7d9db70 (LWP 4318) exited] [New Thread 0xb7d9db70 (LWP 4319)] [Thread 0xb7d9db70 (LWP 4319) exited] Traceback (most recent call last): File "bz643656.py", line 3, in <module> ug.urlgrab(url='ftp://ftp.funet.fi/ftp/pub/mirrors/ftp.redhat.com/pub/fedora/linux/releases/13/Everything/x86_64/os/Packages/iputils-20071127-10.fc13.x86_64.rpm', reget=None, start=1384, end=18096, size=16712) File "/usr/lib/python2.6/site-packages/urlgrabber/grabber.py", line 982, in urlgrab return self._retry(opts, retryfunc, url, filename) File "/usr/lib/python2.6/site-packages/urlgrabber/grabber.py", line 886, in _retry r = apply(func, (opts,) + args, {}) File "/usr/lib/python2.6/site-packages/urlgrabber/grabber.py", line 968, in retryfunc fo = PyCurlFileObject(url, filename, opts) File "/usr/lib/python2.6/site-packages/urlgrabber/grabber.py", line 1063, in __init__ self._do_open() File "/usr/lib/python2.6/site-packages/urlgrabber/grabber.py", line 1351, in _do_open self._do_grab() File "/usr/lib/python2.6/site-packages/urlgrabber/grabber.py", line 1481, in _do_grab self._do_perform() File "/usr/lib/python2.6/site-packages/urlgrabber/grabber.py", line 1323, in _do_perform raise err urlgrabber.grabber.URLGrabError: [Errno 14] Downloaded more than max size for : 29416 > 16712 Program exited with code 01. okay - I tested multiple other ftp servers for the same file and using the latest version of urlgrabber and they all output the same error - giving back more data than asked for. I'm going to be blunt and maybe not kind here but: Maybe it is time for us to put ftp out to pasture. you're right - that example code was incorrect. I was combining two different interfaces in my head. here's the right example import urlgrabber ug = urlgrabber.grabber.URLGrabber() url = 'ftp://ftp.funet.fi/ftp/pub/mirrors/ftp.redhat.com/pub/fedora/linux/releases/13/Everything/x86_64/os/Packages/iputils-20071127-10.fc13.x86_64.rpm' ug.urlgrab(url=url, reget=None, range=(1384,18096), size=16712) on non-funet ftp servers it appears to work. and for a bit more explanation range=(start, end) sets pycurl.RANGE So comment 58 was incorrect b/c i was running the wrong test. Sorry for that. However, the conclusion in comment 58 may still bear some consideration. Why do we continue supporting ftp? (In reply to comment #60) > here's the right example > > import urlgrabber > > ug = urlgrabber.grabber.URLGrabber() > url = > 'ftp://ftp.funet.fi/ftp/pub/mirrors/ftp.redhat.com/pub/fedora/linux/releases/13/Everything/x86_64/os/Packages/iputils-20071127-10.fc13.x86_64.rpm' > > ug.urlgrab(url=url, reget=None, range=(1384,18096), size=16712) > > > on non-funet ftp servers it appears to work. Right, however that failure seems to be already addressed by attachment #453972 [details] -- I am not able to repeat the failure with updated libcurl: $ curl -s https://bugzilla.redhat.com/attachment.cgi?id=454454 \ | gzip -cd \ | cpio -idv libcurl.so.4.2.0 pkgconfig pkgconfig/libcurl.pc libcurl.so libcurl.so.4 2340 blocks $ LD_PRELOAD=./libcurl.so python /tmp/bz643656.py I have no idea why it works on my box (and Anaconda's tty2), but not with the Anaconda GUI. Any idea? "However, the conclusion in comment 58 may still bear some consideration. Why do we continue supporting ftp?" I guess because it's the worst option except for all the others. there aren't a lot of public rsync servers, and http is hardly a better file transfer protocol than ftp... -- Fedora Bugzappers volunteer triage team https://fedoraproject.org/wiki/BugZappers http is vastly better if only b/c the servers have something that vaguely resembles a standard and handling the error codes from http seems to be at least understood better than ftp. fix proposed upstream: http://curl.haxx.se/mail/lib-2010-10/0196.html Seth: Many mirrors prefer/suggest FTP as the primary/best access method. It's easier to queue multiple files with FTP etc.. http mirrors have their problems aswell. I remember virt-install failing because Fedora mirror had customized http error pages for non-existing files.. Anyway, maybe we should keep on topic :) Kamil: Can we get that proposed fix to curl rpm in Fedora? Is it too late for F14? "Is it too late for F14?" Yes. It went gold yesterday. Practically speaking, we stopped taking non-blocker fixes as soon as RC1 went out. -- Fedora Bugzappers volunteer triage team https://fedoraproject.org/wiki/BugZappers (In reply to comment #68) > Kamil: Can we get that proposed fix to curl rpm in Fedora? Is it too late for > F14? The patch solves the problem you originally reported, so that I considered it useful and proposed it upstream. It's pending a review now. If no problem appears with it, I'll build it for Fedora. As for Anaconda, the patch didn't solve the problem AFAIK. We should track the issue separately, hopefully with some help of Anaconda developers. I am not really a python programmer to dive into its code. fixed in curl-7.21.2-3.fc15 Ok, thanks. I'll try with curl-7.21.2-3.fc15 and see how it goes. Here's my test report. F13 with the following default rpms: # rpm -qa|grep -i curl libcurl-7.20.1-4.fc13.x86_64 curl-7.20.1-4.fc13.x86_64 python-pycurl-7.19.0-7.fc13.x86_64 Seth's python test from comment #60 fails with this default F13 setup. After one minute there's the usual timeout error 28 and download fails. Then I grabbed curl-7.21.2-3.fc15 from koji and rebuilt it for F13: # rpm -qa|grep -i curl libcurl-7.21.2-3.fc13.x86_64 curl-7.21.2-3.fc13.x86_64 python-pycurl-7.19.0-7.fc13.x86_64 And now Seth's test from comment #60 works OK without problems! It downloads 16712 bytes from ftp.funet.fi. So at least one problem fixed. I wonder if we should re-test with anaconda aswell.. Pasi, thank you for testing the package. (In reply to comment #73) > So at least one problem fixed. I wonder if we should re-test with anaconda > aswell.. I am afraid we already did. It didn't solve the problem, at least not through the updates image. So there are two possibilities: 1) The problem was solved, but we tested it in a wrong way. 2) There persists another problem that breaks Anaconda downloads. It may be in Anaconda code, but also in yum, urlgrabber, pycurl, or libcurl. We don't have any standalone test-case for this one. Removing F14 NTH status as F14 is now out. Remaining open nice-to-have issues do NOT automatically become nice-to-have issues for Fedora 15. If you believe a Fedora 14 issue which was accepted as nice-to-have but not resolved in time for release should also qualify for nice-to-have status for Fedora 15, please re-propose it as nice-to-have for Fedora 15. -- Fedora Bugzappers volunteer triage team https://fedoraproject.org/wiki/BugZappers Pasi, could you please retest with attachment #458054 [details] ? It works for me with ftp://ftp.funet.fi this time... Yeah sure, I can test again. Did you try with F14 final? (In reply to comment #77) Did you try with F14 final? yes possibly related issues: bug 649347 bug 650255 fixed in curl-7.21.2-4.fc15 Created attachment 459386 [details]
ext2 floppy image with latest libcurl (x86_64)
I just tried with Fedora 14 (GA) using ftp://ftp.funet.fi. - Default F14 installation: fails. - F14 installation with updates.img from comment #81: works OK! So yes, I can confirm this curl patch fixes the problem with Fedora installation! Great! Thank you for being patient during all those attempts. Since installation images are already out, I'll probably postpone the update for F-14 till bug 650255 is also explained and/or fixed. Please let me know if you need it sooner. Kamil: Big thanks to you aswell! I'm happy to wait until 650255 is sorted out, no hurries.. as long as F15 GA installer will have the fixes included :) curl-7.21.0-6.fc14 has been submitted as an update for Fedora 14. https://admin.fedoraproject.org/updates/curl-7.21.0-6.fc14 curl-7.20.1-5.fc13 has been submitted as an update for Fedora 13. https://admin.fedoraproject.org/updates/curl-7.20.1-5.fc13 curl-7.21.0-6.fc14 has been pushed to the Fedora 14 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with su -c 'yum --enablerepo=updates-testing update curl'. You can provide feedback for this update here: https://admin.fedoraproject.org/updates/curl-7.21.0-6.fc14 curl-7.21.0-6.fc14 has been pushed to the Fedora 14 stable repository. If problems still persist, please make note of it in this bug report. curl-7.20.1-5.fc13 has been pushed to the Fedora 13 stable repository. If problems still persist, please make note of it in this bug report. FYI a new FTP command has been proposed by Anthony Bryan, Tatsuhiro Tsujikawa and Daniel Stenberg for exactly this purpose: http://daniel.haxx.se/blog/2010/12/20/byte-ranges-for-ftp The IETF draft is available at: http://www.ietf.org/id/draft-bryan-ftp-range-00.txt |