Bug 484371 - RFE: yum doesn't do automatic dynamic mirror sorting
RFE: yum doesn't do automatic dynamic mirror sorting
Status: NEW
Product: Fedora
Classification: Fedora
Component: yum (Show other bugs)
rawhide
All Linux
low Severity high
: ---
: ---
Assigned To: packaging-team-maint
Fedora Extras Quality Assurance
: FutureFeature, Patch, Triaged
: 450875 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2009-02-06 09:49 EST by Artem S. Tashkinov
Modified: 2016-04-20 08:58 EDT (History)
11 users (show)

See Also:
Fixed In Version:
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)
Implements a slightly less naive approach to mirror speed (1.87 KB, patch)
2009-04-23 15:02 EDT, Andrew McNabb
no flags Details | Diff

  None (edit)
Description Artem S. Tashkinov 2009-02-06 09:49:46 EST
I have a good *fast* Internet connection however yum too often decides to download updates from intolerably slow mirrors - and sometimes yum even chooses to fetch the data from the server which give 1Kbyte/sec speed.

So, these are the major bugs in yum:

1) It doesn't remember mirrors with good speed
2) It doesn't show all available mirrors before downloading updates
3) It doesn't allow to choose the mirrors to download from
4) It doesn't show a mirror from which a current file is being fetched from
5) It doesn't have a key combination to easily switch mirrors while getting the current file (Ctrl+C it really inefficient and slow - and the end result is not defined - it may just decide to stop fetching updates at all)
Comment 1 James Antill 2009-02-06 10:29:41 EST
 You can install yum-fastestmirror ... having this be dynamic is on the TODO list, but is certainly not near the top.
 If you have a fast connection and good bandwidth, you might want to use reposync at night and register that with MirrorManager.
Comment 2 Christopher Beland 2009-04-22 16:30:05 EDT
Just to clarify, it seems that the use of yum-fastestmirror takes care of 1 and 4 directly.  If it automatically picks a mirror it thinks is fast, it seems to me there is no need for 2 or 3.

So is the only remaining request to do 5 automatically instead of manually?  (If a file is being downloaded slowly, try to see if another mirror is faster?)

If this would be an enhancement to yum-fastestmirror, should this bug be reassigned to yum-utils, since it is part of that source package?
Comment 3 James Antill 2009-04-22 22:34:12 EDT
fastestmirror can't do it dynamically ... to do that we'd need to move it into core. fastestmirror basically tests the mirrors at bootup, and then sorts them based on that info.
Comment 4 James Antill 2009-04-23 14:18:33 EDT
*** Bug 450875 has been marked as a duplicate of this bug. ***
Comment 5 Andrew McNabb 2009-04-23 15:02:37 EDT
Created attachment 340988 [details]
Implements a slightly less naive approach to mirror speed

I'm attaching a simple patch that implements a slightly less naive approach to determining the speed of a mirror.  It basically just times how long it takes to download the repomd.xml file (the current behavior just checks how long it takes to connect a socket).  It's still not perfect, but it's an improvement.
Comment 6 Gilbert E. Detillieux 2009-04-24 15:47:11 EDT
I've tried the patch on a Fedora 10 system, and then on several CentOS 5.3 systems (both i386 and x86_64), and it seems to work like a charm!  Finding the fastest mirrors is a bit slower than before, but still acceptably fast.  And the results are much better: closer (or at least more well connected) mirrors seem to consistently come up at the top of the list now.

My test method:
1) yum clean all
2) yum check-update
3) sort -k2n /var/cache/yum/timedhosts.txt
4) (repeat until satisfied)

Before the patch was applied, results (in the timedhosts.txt file) varied from run to run, with distant, slower sites often winning out against local mirrors.  With the patch, I get much more consistent results.  When the sort order is different, it's usually just a few sites with fairly similar timing getting flipped around.

Well done!
Comment 7 Artem S. Tashkinov 2009-04-26 07:32:58 EDT
Thanks a lot for this patch, but it only solves the first outlined problem.

Should I report four new bug reports? ;)
Comment 8 Andrew McNabb 2009-10-22 14:10:17 EDT
Has anyone merged the patch I attached?  It's a pretty simple change, and it seems to work well.  Is there any chance of getting this merged soon?  Thanks.
Comment 9 James Antill 2009-10-22 15:34:45 EDT
We've looked at it, and known we could do that ... but it has problems:

1. urllib isn't thread safe, so you can't do it this way (neither is curl, which is what yum/urlgrabber in F12+ uses).

2. You are now downloading a _lot_ of data, Fedora 11 updates has at least 46 http urls ... that's over 200k. And you are ignoring a lot of config. data from yum (throttling, etc.)

3. It's still unreliable.

4. It still doesn't work with proxies.

5. Mirror manager is very good on it's own, in my experience (there's a reason fastestmirror isn't in the default install). And RHEL doesn't use mirrors.
Comment 10 James Antill 2009-10-22 15:35:25 EDT
moving this to yum, as if/when we do this it's almost certainly going to be in yum core.
Comment 11 Andrew McNabb 2009-10-22 16:34:55 EDT
James, thanks for all of the information.  This is great to know.
Comment 12 Daniel Berrange 2012-04-16 11:23:03 EDT
> 5. Mirror manager is very good on it's own, in my experience (there's a reason
> fastestmirror isn't in the default install). And RHEL doesn't use mirrors.

My experience is entirely the opposite - at work it appears to consistently pick a pretty bad mirror. With plain yum the mirror picked gives me 25 KB/s, once I install yum-plugin-fastestmirror, it picks a mirror giving 1 MB/s download. This gives people a really bad impression of yum by default. I would really like to see the fastestmirror plugin logic (or equivalent) included in YUM by default.

Are there anything preventing us having the fastestmirror plugin included by default in base yum in Fedora ?

Note You need to log in before you can comment on or make changes to this bug.