Bug 235618 - yum restarts interrupted downloads instead of resuming
yum restarts interrupted downloads instead of resuming
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: python-urlgrabber (Show other bugs)
rawhide
All Linux
medium Severity medium
: ---
: ---
Assigned To: Jeremy Katz
Fedora Extras Quality Assurance
:
: 245280 (view as bug list)
Depends On:
Blocks: F8Blocker
  Show dependency treegraph
 
Reported: 2007-04-08 02:13 EDT by ritz
Modified: 2014-01-21 17:57 EST (History)
5 users (show)

See Also:
Fixed In Version: 3.0.0-3.fc7
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2007-11-05 16:56:54 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description ritz 2007-04-08 02:13:29 EDT
Description of problem:
yum restarts download from start on connection failure. 

Version-Release number of selected component (if applicable):
yum-3.1.6-1.fc7

How reproducible:
always

Steps to Reproduce:
1. start yum for a heavy package ( such as openoffice )
2. wait for connection timeout
3. yum restarts download from another mirror
  
Actual results:
yum starts download from start

Expected results:
yum should continue download from the same or alternate mirror . This works fine
with wget .

Additional info:
using a dial-up connection. out of office, and out of reach from broadband for ~
a month.

(55/328): kernel-PAE-2.6. 100% |=========================|  11 MB    55:08     
http://mirrors.kernel.org/fedora/core/development/i386/os/Fedora/kernel-PAE-2.6.20-1.3045.fc7.i686.rpm:
[Errno 4] Socket Error: timed out
Trying other mirror.
(55/328): kernel-PAE-2.6.   3% |                         | 632 kB   263:02 ETA 


using wget as a workaround to download heavy packages, as wget will resume
download on temporary disconnection .

reference : bugzilla#162673
Comment 1 Seth Vidal 2007-04-08 02:19:34 EDT
umm yum does support regets. It uses them by default. It uses http or ftp
byte-ranges to grab them separately. To test this - start a download of
openoffice, for example. Look at the file entry in /var/cache/yum/*/packages and
note the size of the file.

cancel the download

now, start the download again, and note the size of the file now.
Comment 2 ritz 2007-04-08 12:55:27 EDT
No go :(
yum seems to restart the download .


# ls -sh /var/cache/yum/development/packages/cups-1.2.10-5.fc7.i386.rpm 
1.4M /var/cache/yum/development/packages/cups-1.2.10-5.fc7.i386.rpm



# yum -d 10  -y update cups
Loading "fedorakmod" plugin
Loading "changelog" plugin
Loading "installonlyn" plugin
Loading "fastestmirror" plugin
Loading "protectbase" plugin
Loading "skip-broken" plugin
Running "config" handler for "changelog" plugin
Running "config" handler for "installonlyn" plugin
Running "config" handler for "protectbase" plugin
Running "config" handler for "skip-broken" plugin
Running "init" handler for "fedorakmod" plugin
Loading Fedora Extras kernel module support.
Running "init" handler for "fastestmirror" plugin
Running "postreposetup" handler for "changelog" plugin
Running "postreposetup" handler for "fastestmirror" plugin
....
Total download size: 2.9 M
Downloading Packages:
http://mirror.hiwaay.net/redhat/fedora/linux/core/development/i386/os/Fedora/cups-1.2.10-5.fc7.i386.rpm:
[Errno 12] Timeout: <urlopen error timed out>
Trying other mirror.
(1/1): cups-1.2.10-5.fc7. 100% |=========================| 112 kB    03:27     
http://download.fedoraproject.org/pub/fedora/linux/core/development/i386/os/Fedora/cups-1.2.10-5.fc7.i386.rpm:
[Errno 4] Socket Error: timed out
Trying other mirror.
ftp://ftp.funet.fi/pub/mirrors/ftp.redhat.com/pub/fedora/linux/core/development/i386/os/Fedora/cups-1.2.10-5.fc7.i386.rpm:
[Errno 4] IOError: [Errno ftp error] (104, 'Connection reset by peer')

#  killall -9 yum

# ls -sh /var/cache/yum/development/packages/cups-1.2.10-5.fc7.i386.rpm 
84K /var/cache/yum/development/packages/cups-1.2.10-5.fc7.i386.rpm

Comment 3 ritz 2007-04-10 17:33:57 EDT
using smart as a workaround .
Comment 4 Seth Vidal 2007-04-10 17:37:04 EDT
not really a workaround just a 'work different'. I can't get the above to
replicate on my machine here so until then it's hard to test.

Comment 5 ritz 2007-04-11 00:05:29 EDT
how should i run yum, to provide the required debug logs ?
Comment 6 Seth Vidal 2007-04-11 00:16:09 EDT
1. export URLGRABBER_DEBUG=1
2. yum -d 10 yourcommand here


send that output, please.
Comment 7 ritz 2007-04-11 00:57:15 EDT
package is downloaded from start.

]# ls -sh /var/cache/yum/development/packages/gtkhtml3-3.14.1-1.fc7.i386.rpm 
348K /var/cache/yum/development/packages/gtkhtml3-3.14.1-1.fc7.i386.rpm


...
=============================================================================
 Package                 Arch       Version          Repository        Size 
=============================================================================
Updating:
 gtkhtml3                i386       3.14.1-1.fc7     development       923 k
Updating for dependencies:
 gtkhtml3-devel          i386       3.14.1-1.fc7     development        26 k

Transaction Summary
=============================================================================
Install      0 Package(s)         
Update       2 Package(s)         
Remove       0 Package(s)         

Total download size: 949 k
Is this ok [y/N]: y
Downloading Packages:
2007-04-11 10:19:13,864 MIRROR: trying Fedora/gtkhtml3-3.14.1-1.fc7.i386.rpm ->
http://mirror.hiwaay.net/redhat/fedora/linux/core/development/i386/os/Fedora/gtkhtml3-3.14.1-1.fc7.i386.rpm
INFO:urlgrabber:MIRROR: trying Fedora/gtkhtml3-3.14.1-1.fc7.i386.rpm ->
http://mirror.hiwaay.net/redhat/fedora/linux/core/development/i386/os/Fedora/gtkhtml3-3.14.1-1.fc7.i386.rpm
2007-04-11 10:19:13,865 attempt 1/10:
http://mirror.hiwaay.net/redhat/fedora/linux/core/development/i386/os/Fedora/gtkhtml3-3.14.1-1.fc7.i386.rpm
INFO:urlgrabber:attempt 1/10:
http://mirror.hiwaay.net/redhat/fedora/linux/core/development/i386/os/Fedora/gtkhtml3-3.14.1-1.fc7.i386.rpm
2007-04-11 10:19:13,869 creating new connection to mirror.hiwaay.net (207654444)
INFO:urlgrabber:creating new connection to mirror.hiwaay.net (207654444)
2007-04-11 10:19:16,151 STATUS: 200, OK
INFO:urlgrabber:STATUS: 200, OK
(1/2): gtkhtml3-3.14.1-1. 100% |=========================| 923 kB    03:28     
2007-04-11 10:22:45,143 success
INFO:urlgrabber:success
2007-04-11 10:22:45,144 MIRROR: trying
Fedora/gtkhtml3-devel-3.14.1-1.fc7.i386.rpm ->
http://mirror.hiwaay.net/redhat/fedora/linux/core/development/i386/os/Fedora/gtkhtml3-devel-3.14.1-1.fc7.i386.rpm
INFO:urlgrabber:MIRROR: trying Fedora/gtkhtml3-devel-3.14.1-1.fc7.i386.rpm ->
http://mirror.hiwaay.net/redhat/fedora/linux/core/development/i386/os/Fedora/gtkhtml3-devel-3.14.1-1.fc7.i386.rpm
2007-04-11 10:22:45,146 attempt 1/10:
http://mirror.hiwaay.net/redhat/fedora/linux/core/development/i386/os/Fedora/gtkhtml3-devel-3.14.1-1.fc7.i386.rpm
INFO:urlgrabber:attempt 1/10:
http://mirror.hiwaay.net/redhat/fedora/linux/core/development/i386/os/Fedora/gtkhtml3-devel-3.14.1-1.fc7.i386.rpm
2007-04-11 10:22:45,147 failed to re-use connection to mirror.hiwaay.net (207654444)
INFO:urlgrabber:failed to re-use connection to mirror.hiwaay.net (207654444)
2007-04-11 10:22:45,148 creating new connection to mirror.hiwaay.net (207654444)
INFO:urlgrabber:creating new connection to mirror.hiwaay.net (207654444)
2007-04-11 10:22:51,026 STATUS: 200, OK
INFO:urlgrabber:STATUS: 200, OK
(2/2): gtkhtml3-devel-3.1 100% |=========================|  26 kB    00:05     
2007-04-11 10:22:56,802 success
INFO:urlgrabber:success
Running Transaction Test
Comment 8 Seth Vidal 2007-04-11 01:17:04 EDT
so two problems:

1. what the progress bar says about total downloaded is not entirely accurate -
which might be where the confusion is happening. the progress bar number at the
end is reporting the size of the FILE, not the number of bytes downloaded.

2. I just tested this out and I still can't duplicate it. Try this for me:
   a. find a package that's a pretty good size 5-10M
   b. try to install it and ctrl-c the download about halfway through
   c. run watch -n1 ls -lah /var/cache/yum/$thatrepo/packages/ in another terminal
  see if the file doesn't just INCREASE in size, rather than truncate and start
over.


Thank you.

Comment 9 ritz 2007-04-11 04:19:25 EDT
1. the progress displays the size downloaded , i have verified this with watch
-n1 ls -sh /....

2. I have tried this, and file is truncated and dowbload starts over again .


Is there any other componen to debug ?

If required, i can attach output of screen using script.
Comment 10 Thorsten Leemhuis 2007-04-11 12:32:21 EDT
Confirmed on devel.i386: a partly downloaded file gets truncated and download
starts anew again. I actually saw this some days ago already once or twice, but
didn't take a closer look until now.
Comment 11 Seth Vidal 2007-04-11 12:43:31 EDT
looks like this is only happening on systems using python 2.5, afaict.

I'll see if I can chase it down.
Comment 12 Seth Vidal 2007-06-20 12:17:42 EDT
confirmed on f7.
Comment 13 Seth Vidal 2007-06-22 00:56:30 EDT
*** Bug 245280 has been marked as a duplicate of this bug. ***
Comment 14 Jeremy Katz 2007-10-09 17:19:42 EDT
Are you still seeing this?  It's working fine with a simpler urlgrabber test
script.  Will try to reproduce in yum tomorrow.
Comment 15 Jan ONDREJ 2007-10-10 01:21:34 EDT
I still see same problems with yum-3.2.6-1.fc7.
Comment 16 ritz 2007-10-10 04:50:05 EDT
bug still here :(

python-urlgrabber-3.0.0-2.fc8
yum-3.2.6-1.fc8
Comment 17 Jeremy Katz 2007-10-10 11:42:58 EDT
Fixed in urlgrabber-3.0.0-3.  Will push back to F7-updates if someone grabs the
rawhide package and tests it out 
Comment 18 Jan ONDREJ 2007-10-10 12:07:16 EDT
Works for me with latest urlgrabber from koji:
python-urlgrabber-3.0.0-3.fc8
yum-3.2.6-1.fc7
Comment 19 Fedora Update System 2007-10-10 21:47:45 EDT
python-urlgrabber-3.0.0-3.fc7 has been pushed to the Fedora 7 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update python-urlgrabber'
Comment 20 Fedora Update System 2007-11-05 16:56:52 EST
python-urlgrabber-3.0.0-3.fc7 has been pushed to the Fedora 7 stable repository.  If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.