Bug 515744 - RFE: yum should resume downloading meta-data files (e.g. primarydb) when possible
Summary: RFE: yum should resume downloading meta-data files (e.g. primarydb) when poss...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: yum
Version: 11
Hardware: All
OS: Linux
low
medium
Target Milestone: ---
Assignee: Seth Vidal
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2009-08-05 15:16 UTC by Hedayat Vatankhah
Modified: 2014-01-21 23:10 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
Clone Of:
Environment:
Last Closed: 2010-04-28 13:35:16 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)

Description Hedayat Vatankhah 2009-08-05 15:16:23 UTC
Description of problem:
Currently, if yum cannot completely download a metadata file, it'll start downloading the file from scratch from another mirror. (Or if it terminates, it'll start downloading from scratch in the next run). This is really problematic for slow or bad internet connections. Sometimes, the connection times out regularly when yum is downloading a database file, and it starts downloading from scratch each time. It is really frustrating (specially in PackageKit, where you don't see any progress and you wonder why it do not finish downloading repository data). 
Sometimes I'm forced to download those files by another tool and put it in yum cache directory manually.

This is a small feature, but it improves user experience considerably (for users with bad internet connection).

Comment 1 James Antill 2009-08-05 16:37:25 UTC
Are you using the new test version of urlgrabber? As I'm sure the old version would resume for metadata files (when it could).

Comment 2 Hedayat Vatankhah 2009-08-05 16:56:54 UTC
No, I'm using Fedora 11 urlgrabber: python-urlgrabber-3.0.0-15.fc11.noarch.rpm
As far as I remember, it have not resumed for me even once (even in previous Fedora versions). It only works for packages themselves.

Comment 3 Hedayat Vatankhah 2009-08-20 07:20:40 UTC
I'm not sure that this bug was present always (in Fedora 11 and previous versions). Please fix this really annoying bug :(

Comment 4 seth vidal 2009-08-24 17:33:27 UTC
it's not really a bug - we don't reget the repodata b/c, by default, the repodata isn't uniquely named. We could enable it for fedora b/c our repos are using uniquely named repodata, but other repos do not.

it's a bit of a connundrum.

Comment 5 Hedayat Vatankhah 2009-08-25 08:54:09 UTC
So why createrepo does not generate uniquely named repodata files?! It can do it whenever it is creating a repository over an existing repository and/or updating a repository. 
The repodata file names can be as simple as including a sequential version number, or adding the file creating date to the file name or something like current fedora repository files (seem to be hash of something, probably the file itself?!).
This feature is needed for any repository, so I think yum/createrepo should use it by default everywhere. 
Providing a version assignment system for repodata files are (I think) also required for the (hopefully coming) delta meta-data support.

Comment 6 Hedayat Vatankhah 2009-11-08 22:03:39 UTC
Well, resuming the download of repository metadata could be enabled in the configuration file of a repository for repositories with unique repodata file names. It is easy and also it'll not be problematic for repositories without uniquely named repodata.

Comment 7 seth vidal 2010-01-12 21:55:15 UTC
If we make --unique-md-filenames a default in createrepo then I think this is reasonable. And it might be reasonable now since fedora does unique-md-filenames now.

Comment 8 seth vidal 2010-02-08 16:34:42 UTC
a patch to enable regets for metadata downloads has been submitted upstream.

Comment 9 David Culley 2010-04-17 18:45:25 UTC
Hi - I hope I'm not in the wrong place, but I was wondering if this fix has been implemented yet, and if so, how I can activate it.  My office LAN is woefully under-funded, and I can't get more than a few 100kb downloaded before losing the connection.  This has essentially crippled yum during office hours.  Package downloads work fine, because they resume from where they broke, but the metadata is killing me (and requiring hundreds of expensive MB where only a few should really be required).

Thanks

Comment 10 James Antill 2010-04-19 03:58:21 UTC
It has, although the latest fix of it requires size data on the MD.
If you install pycurl from F11, you can probably install the yum from rawhide.

Or download the yum from F12/updates.

Comment 11 Bug Zapper 2010-04-28 09:34:39 UTC
This message is a reminder that Fedora 11 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 11.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '11'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 11's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 11 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 12 James Antill 2010-04-28 13:35:16 UTC
It's fixed for F-13 and F-12 (IIRC). Backporting to F-11 is really unlikely at this point.


Note You need to log in before you can comment on or make changes to this bug.