Bug 235618 - yum restarts interrupted downloads instead of resuming
Summary: yum restarts interrupted downloads instead of resuming
Alias: None
Product: Fedora
Classification: Fedora
Component: python-urlgrabber   
(Show other bugs)
Version: rawhide
Hardware: All Linux
Target Milestone: ---
Assignee: Jeremy Katz
QA Contact: Fedora Extras Quality Assurance
: 245280 (view as bug list)
Depends On:
Blocks: F8Blocker
TreeView+ depends on / blocked
Reported: 2007-04-08 06:13 UTC by ritz
Modified: 2014-01-21 22:57 UTC (History)
5 users (show)

Fixed In Version: 3.0.0-3.fc7
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2007-11-05 21:56:54 UTC
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

Description ritz 2007-04-08 06:13:29 UTC
Description of problem:
yum restarts download from start on connection failure. 

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. start yum for a heavy package ( such as openoffice )
2. wait for connection timeout
3. yum restarts download from another mirror
Actual results:
yum starts download from start

Expected results:
yum should continue download from the same or alternate mirror . This works fine
with wget .

Additional info:
using a dial-up connection. out of office, and out of reach from broadband for ~
a month.

(55/328): kernel-PAE-2.6. 100% |=========================|  11 MB    55:08     
[Errno 4] Socket Error: timed out
Trying other mirror.
(55/328): kernel-PAE-2.6.   3% |                         | 632 kB   263:02 ETA 

using wget as a workaround to download heavy packages, as wget will resume
download on temporary disconnection .

reference : bugzilla#162673

Comment 1 Seth Vidal 2007-04-08 06:19:34 UTC
umm yum does support regets. It uses them by default. It uses http or ftp
byte-ranges to grab them separately. To test this - start a download of
openoffice, for example. Look at the file entry in /var/cache/yum/*/packages and
note the size of the file.

cancel the download

now, start the download again, and note the size of the file now.

Comment 2 ritz 2007-04-08 16:55:27 UTC
No go :(
yum seems to restart the download .

# ls -sh /var/cache/yum/development/packages/cups-1.2.10-5.fc7.i386.rpm 
1.4M /var/cache/yum/development/packages/cups-1.2.10-5.fc7.i386.rpm

# yum -d 10  -y update cups
Loading "fedorakmod" plugin
Loading "changelog" plugin
Loading "installonlyn" plugin
Loading "fastestmirror" plugin
Loading "protectbase" plugin
Loading "skip-broken" plugin
Running "config" handler for "changelog" plugin
Running "config" handler for "installonlyn" plugin
Running "config" handler for "protectbase" plugin
Running "config" handler for "skip-broken" plugin
Running "init" handler for "fedorakmod" plugin
Loading Fedora Extras kernel module support.
Running "init" handler for "fastestmirror" plugin
Running "postreposetup" handler for "changelog" plugin
Running "postreposetup" handler for "fastestmirror" plugin
Total download size: 2.9 M
Downloading Packages:
[Errno 12] Timeout: <urlopen error timed out>
Trying other mirror.
(1/1): cups-1.2.10-5.fc7. 100% |=========================| 112 kB    03:27     
[Errno 4] Socket Error: timed out
Trying other mirror.
[Errno 4] IOError: [Errno ftp error] (104, 'Connection reset by peer')

#  killall -9 yum

# ls -sh /var/cache/yum/development/packages/cups-1.2.10-5.fc7.i386.rpm 
84K /var/cache/yum/development/packages/cups-1.2.10-5.fc7.i386.rpm

Comment 3 ritz 2007-04-10 21:33:57 UTC
using smart as a workaround .

Comment 4 Seth Vidal 2007-04-10 21:37:04 UTC
not really a workaround just a 'work different'. I can't get the above to
replicate on my machine here so until then it's hard to test.

Comment 5 ritz 2007-04-11 04:05:29 UTC
how should i run yum, to provide the required debug logs ?

Comment 6 Seth Vidal 2007-04-11 04:16:09 UTC
2. yum -d 10 yourcommand here

send that output, please.

Comment 7 ritz 2007-04-11 04:57:15 UTC
package is downloaded from start.

]# ls -sh /var/cache/yum/development/packages/gtkhtml3-3.14.1-1.fc7.i386.rpm 
348K /var/cache/yum/development/packages/gtkhtml3-3.14.1-1.fc7.i386.rpm

 Package                 Arch       Version          Repository        Size 
 gtkhtml3                i386       3.14.1-1.fc7     development       923 k
Updating for dependencies:
 gtkhtml3-devel          i386       3.14.1-1.fc7     development        26 k

Transaction Summary
Install      0 Package(s)         
Update       2 Package(s)         
Remove       0 Package(s)         

Total download size: 949 k
Is this ok [y/N]: y
Downloading Packages:
2007-04-11 10:19:13,864 MIRROR: trying Fedora/gtkhtml3-3.14.1-1.fc7.i386.rpm ->
INFO:urlgrabber:MIRROR: trying Fedora/gtkhtml3-3.14.1-1.fc7.i386.rpm ->
2007-04-11 10:19:13,865 attempt 1/10:
INFO:urlgrabber:attempt 1/10:
2007-04-11 10:19:13,869 creating new connection to mirror.hiwaay.net (207654444)
INFO:urlgrabber:creating new connection to mirror.hiwaay.net (207654444)
2007-04-11 10:19:16,151 STATUS: 200, OK
INFO:urlgrabber:STATUS: 200, OK
(1/2): gtkhtml3-3.14.1-1. 100% |=========================| 923 kB    03:28     
2007-04-11 10:22:45,143 success
2007-04-11 10:22:45,144 MIRROR: trying
Fedora/gtkhtml3-devel-3.14.1-1.fc7.i386.rpm ->
INFO:urlgrabber:MIRROR: trying Fedora/gtkhtml3-devel-3.14.1-1.fc7.i386.rpm ->
2007-04-11 10:22:45,146 attempt 1/10:
INFO:urlgrabber:attempt 1/10:
2007-04-11 10:22:45,147 failed to re-use connection to mirror.hiwaay.net (207654444)
INFO:urlgrabber:failed to re-use connection to mirror.hiwaay.net (207654444)
2007-04-11 10:22:45,148 creating new connection to mirror.hiwaay.net (207654444)
INFO:urlgrabber:creating new connection to mirror.hiwaay.net (207654444)
2007-04-11 10:22:51,026 STATUS: 200, OK
INFO:urlgrabber:STATUS: 200, OK
(2/2): gtkhtml3-devel-3.1 100% |=========================|  26 kB    00:05     
2007-04-11 10:22:56,802 success
Running Transaction Test

Comment 8 Seth Vidal 2007-04-11 05:17:04 UTC
so two problems:

1. what the progress bar says about total downloaded is not entirely accurate -
which might be where the confusion is happening. the progress bar number at the
end is reporting the size of the FILE, not the number of bytes downloaded.

2. I just tested this out and I still can't duplicate it. Try this for me:
   a. find a package that's a pretty good size 5-10M
   b. try to install it and ctrl-c the download about halfway through
   c. run watch -n1 ls -lah /var/cache/yum/$thatrepo/packages/ in another terminal
  see if the file doesn't just INCREASE in size, rather than truncate and start

Thank you.

Comment 9 ritz 2007-04-11 08:19:25 UTC
1. the progress displays the size downloaded , i have verified this with watch
-n1 ls -sh /....

2. I have tried this, and file is truncated and dowbload starts over again .

Is there any other componen to debug ?

If required, i can attach output of screen using script.

Comment 10 Thorsten Leemhuis 2007-04-11 16:32:21 UTC
Confirmed on devel.i386: a partly downloaded file gets truncated and download
starts anew again. I actually saw this some days ago already once or twice, but
didn't take a closer look until now.

Comment 11 Seth Vidal 2007-04-11 16:43:31 UTC
looks like this is only happening on systems using python 2.5, afaict.

I'll see if I can chase it down.

Comment 12 Seth Vidal 2007-06-20 16:17:42 UTC
confirmed on f7.

Comment 13 Seth Vidal 2007-06-22 04:56:30 UTC
*** Bug 245280 has been marked as a duplicate of this bug. ***

Comment 14 Jeremy Katz 2007-10-09 21:19:42 UTC
Are you still seeing this?  It's working fine with a simpler urlgrabber test
script.  Will try to reproduce in yum tomorrow.

Comment 15 Jan ONDREJ 2007-10-10 05:21:34 UTC
I still see same problems with yum-3.2.6-1.fc7.

Comment 16 ritz 2007-10-10 08:50:05 UTC
bug still here :(


Comment 17 Jeremy Katz 2007-10-10 15:42:58 UTC
Fixed in urlgrabber-3.0.0-3.  Will push back to F7-updates if someone grabs the
rawhide package and tests it out 

Comment 18 Jan ONDREJ 2007-10-10 16:07:16 UTC
Works for me with latest urlgrabber from koji:

Comment 19 Fedora Update System 2007-10-11 01:47:45 UTC
python-urlgrabber-3.0.0-3.fc7 has been pushed to the Fedora 7 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update python-urlgrabber'

Comment 20 Fedora Update System 2007-11-05 21:56:52 UTC
python-urlgrabber-3.0.0-3.fc7 has been pushed to the Fedora 7 stable repository.  If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.