Red Hat Bugzilla – Bug 231572
wget with -nc aborts when it finds the first file that already exists
Last modified: 2008-02-07 06:20:43 EST
Description of problem: wget with -nc aborts when it finds the first file that already exists. Start a wget download and interrupt it (because it is too long, or because the machine crashed or because the soup is getting cold). Then try to resume the download using the -nc parameter: wget aborts as soon as it finds an already existing file. The intended semantics is that wget will skip these files and continue downloading those that were not downloaded in the first run. Indeed, anything else makes the program less than useful as it precluded decent resumption of a long task. I compiled the official sources wget-1.10.2.tar.gz from http://ftp.gnu.org/gnu/wget/ and it works fine, just as expected, continuing downloading from the point where it was stopped, skipping existing files. Version-Release number of selected component (if applicable): 1.10.2-8.fc6.1. I used this build on FC5 in order to check whether the wrong behaviour that I found in the build (Fedora Core 5 update) 1.10.2-3.3.fc5 was fixed, but it was not. Both versions are wrong. (surprising the the fc6 RPM installed without complaints in FC5). How reproducible: Always Steps to Reproduce: 1. enter the following command: $ wget -erobots=off -o wget.log -nH -r http://pure-data.cvs.sourceforge.net/pure-data/doc/tutorials/ (one line) and let it run for 2 minutes. 2. ^C to stop it. 3. enter the following command, including -nc -c (to continue download of a partially downloaded file, if any, and new files not-yet downloaded: wget -erobots=off -nH -c -r -nc http://pure-data.cvs.sourceforge.net/pure-data/doc/tutorials/ (in one line). Actual results: $ wget -erobots=off -nH -c -r -nc http://pure-data.cvs.sourceforge.net/pure-data/doc/tutorials/ --18:29:27-- http://pure-data.cvs.sourceforge.net/pure-data/doc/tutorials/ Resolving pure-data.cvs.sourceforge.net... 66.35.250.81 Connecting to pure-data.cvs.sourceforge.net|66.35.250.81|:80... connected. HTTP request sent, awaiting response... 200 OK File `pure-data/doc/tutorials/index.html' already there; not retrieving. Aborted $ ====> notice the "Aborted" line. Expected results: File `pure-data/doc/tutorials/index.html' already there; not retrieving. Warning: wildcards not supported in HTTP. File `*docroot*/styles.css' already there; not retrieving. File `pure-data/index.html' already there; not retrieving. File `pure-data/doc/index.html' already there; not retrieving. .... etc ... This is the output from my locally compiled wget from the GNU sources. Additional info: none
I can reproduce this. I have a core, if anyone is interested. The abort happens at http.c, line 2599. I don't know how it gets there, as hstat.res == -1 and thus we shouldn't be inside the if @ 2559 (and then, inside the if @ 2561). Either it's a compiler bug or the debuginfo package is wrong. hstat = {len = 0, contlen = -1, restval = 0, res = -1, rderrmsg = 0x0, newloc = 0x0, remote_time = 0x0, error = 0x0, statcode = 0, rd_size = 0, dltime = 0, referer = 0x0, local_file = 0x9bce378 "www.redhat.com/robots.txt", timestamp_checked = false, orig_file_name = 0x0, orig_file_size = 0, orig_file_tstamp = 0} rpm -q wget wget-debuginfo --qf '%{name}-%{version}-%{release}.%{arch}\n': wget-1.10.2-8.fc6.1.i386 wget-debuginfo-1.10.2-8.fc6.1.i386
Should this bug be closed? I can no longer reproduce it.
Closing as FC-6 isn't supported anymore and more recent releases don't seem to have this bug.