Bug 999489 - Yum ignores fastestmirror
Summary: Yum ignores fastestmirror
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: yum
Version: 19
Hardware: All
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Packaging Maintenance Team
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-08-21 12:05 UTC by Richard Allen
Modified: 2017-06-06 12:14 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-02-18 14:05:41 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1428210 0 medium CLOSED yum retrieves metadata of the slow repository before detecting the fastest mirror by fastestmirror plugin 2021-02-22 00:41:40 UTC

Internal Links: 1428210

Description Richard Allen 2013-08-21 12:05:57 UTC
Description of problem:
Yum seems to ignore fastestmirror

Version-Release number of selected component (if applicable):
yum-3.4.3-106.fc19.noarch
yum-utils-1.1.31-17.fc19.noarch
yum-plugin-fastestmirror-1.1.31-17.fc19.noarch

How reproducible:
Install yum-plugin-fastestmirror.   Have it determine your local mirror
Run yum update, see yum sometimes talking to other mirrors.


Steps to Reproduce:
1. Install yum-plugin-fastestmirror
2. Run yum update
3. see yum sometimes talking to other mirrors

Actual results:
Yum downloading from remote mirrors


Expected results:
Yum using local (fstest) mirror


Additional info:
Example:
[root@hamburger ~]# yum update
Loaded plugins: fastestmirror, langpacks, refresh-packagekit
adobe-linux-x86_64                                                                               |  951 B  00:00:00     
google-chrome                                                                                    |  951 B  00:00:00     
google-talkplugin                                                                                |  951 B  00:00:00     
rpmfusion-free-updates                                                                           | 3.3 kB  00:00:00     
rpmfusion-nonfree-updates                                                                        | 3.3 kB  00:00:00     
steam                                                                                            | 3.3 kB  00:00:00     
updates/19/x86_64/metalink                                                                       |  29 kB  00:00:00     
updates                                                                                          | 4.6 kB  00:00:00     
updates/19/x86_64/primary_db                                                                     | 7.3 MB  00:00:02     
(1/2): updates/19/x86_64/updateinfo                                                              | 674 kB  00:00:00     
(2/2): updates/19/x86_64/pkgtags                                                                 | 461 kB  00:00:01     
Loading mirror speeds from cached hostfile
 * fedora: www.fedora.is
 * rpmfusion-free: www.fedora.is
 * rpmfusion-free-updates: www.fedora.is
 * rpmfusion-nonfree: www.fedora.is
 * updates: www.fedora.is
Warning: Environment Group gnome-desktop-environment does not exist.
Warning: Environment Group kde-desktop-environment does not exist.
Resolving Dependencies
--> Running transaction check
---> Package fros.noarch 0:1.0-1.fc19 will be updated
---> Package fros.noarch 0:1.0-4.fc19 will be an update
---> Package ibus-typing-booster.noarch 0:1.2.2-1.fc19 will be updated
---> Package ibus-typing-booster.noarch 0:1.2.3-2.fc19 will be an update
[snip]

We can see that www.fedora.is is the fastest mirror.   This is completely correct so far.

[snip]
(26/33): qemu-system-xtensa-1.4.2-6.fc19.x86_64.rpm                                              | 1.8 MB  00:00:00     
(27/33): tcp_wrappers-7.6-75.fc19.x86_64.rpm                                                     |  78 kB  00:00:00     
(28/33): tcp_wrappers-libs-7.6-75.fc19.i686.rpm                                                  |  66 kB  00:00:00     
(29/33): tcp_wrappers-libs-7.6-75.fc19.x86_64.rpm                                                |  66 kB  00:00:00     
(30/33): qemu-system-x86-1.4.2-6.fc19.x86_64.rpm                                                 | 3.0 MB  00:00:01     
(31/33): qemu-user-1.4.2-6.fc19.x86_64.rpm                                                       | 5.9 MB  00:00:03     
(32/33): qemu-system-mips-1.4.2-6.fc19.x86_64.rpm                                                | 5.6 MB  00:00:07     
libisofs-1.3.0-1.fc19.x86_64.r FAILED                                          
http://ftp.pbone.net/pub/fedora/linux/updates/19/x86_64/libisofs-1.3.0-1.fc19.x86_64.rpm: [Errno 12] Timeout on http://ftp.pbone.net/pub/fedora/linux/updates/19/x86_64/libisofs-1.3.0-1.fc19.x86_64.rpm: (28, 'Connection timed out after 30112 milliseconds')
Trying other mirror.
(33/33): libisofs-1.3.0-1.fc19.x86_64.rpm                                                        | 172 kB  00:00:00     
------------------------------------------------------------------------------------------------------------------------
Total                                                                                   1.2 MB/s |  38 MB     00:30     
[snip]

Why is yum talking to ftp.pbone.net?

fedora.is is ok:

[root@hamburger ~]# wget http://www.fedora.is/fedora/updates/19/x86_64/libisofs-1.3.0-1.fc19.x86_64.rpm
--2013-08-21 12:03:38--  http://www.fedora.is/fedora/updates/19/x86_64/libisofs-1.3.0-1.fc19.x86_64.rpm
Resolving www.fedora.is (www.fedora.is)... 194.105.226.20
Connecting to www.fedora.is (www.fedora.is)|194.105.226.20|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 175832 (172K) [application/octet-stream]
Saving to: ‘libisofs-1.3.0-1.fc19.x86_64.rpm’

100%[==============================================================================>] 175,832     --.-K/s   in 0.02s   

2013-08-21 12:03:38 (8.20 MB/s) - ‘libisofs-1.3.0-1.fc19.x86_64.rpm’ saved [175832/175832]


Here in Iceland we have to pay for international bandwidth usage.  Local within the country is free.    This means that when yum fails to use our local mirror it costs us real money.

Comment 1 Richard Allen 2013-08-22 10:56:17 UTC
Today:

[root@hamburger ~]# yum update
Loaded plugins: fastestmirror, langpacks, refresh-packagekit
Loading mirror speeds from cached hostfile
 * fedora: www.fedora.is
 * rpmfusion-free: www.fedora.is
 * rpmfusion-free-updates: www.fedora.is
 * rpmfusion-nonfree: www.fedora.is
 * updates: www.fedora.is
Warning: Environment Group gnome-desktop-environment does not exist.
Warning: Environment Group kde-desktop-environment does not exist.
Resolving Dependencies
--> Running transaction check
[snip]
Downloading packages:
(1/10): control-center-filesystem-3.8.4.1-1.fc19.x86_64.rpm                                      |  15 kB  00:00:00     
(2/10): libcacard-1.4.2-7.fc19.x86_64.rpm                                                        |  70 kB  00:00:00     
evince-libs-3.8.3-2.fc19.x86_6 FAILED                                          
http://fedora-mirror01.rbc.ru/pub/fedora/linux/updates/19/x86_64/evince-libs-3.8.3-2.fc19.x86_64.rpm: [Errno 14] HTTP Error 404 - Not Found
Trying other mirror.
evince-nautilus-3.8.3-2.fc19.x FAILED                                          
http://ftp.byfly.by/pub/fedoraproject.org/linux/updates/19/x86_64/evince-nautilus-3.8.3-2.fc19.x86_64.rpm: [Errno 14] HTTP Error 404 - Not Found
Trying other mirror.
(3/10): python-boto-2.9.6-2.fc19.noarch.rpm                                                      | 1.2 MB  00:00:00     
selinux-policy-3.12.1-71.fc19. FAILED                                                 ] 1.3 MB/s | 5.3 MB  00:00:07 ETA 
ftp://ftp.funet.fi/pub/mirrors/fedora.redhat.com/pub/fedora/linux/updates/19/x86_64/selinux-policy-3.12.1-71.fc19.noarch.rpm: [Errno 14]  Error 550 - Given file does not exist
Trying other mirror.
(4/10): qemu-guest-agent-1.4.2-7.fc19.x86_64.rpm                                                 | 125 kB  00:00:01     
(5/10): evince-3.8.3-2.fc19.x86_64.rpm                                                           | 5.6 MB  00:00:02     
(6/10): evince-nautilus-3.8.3-2.fc19.x86_64.rpm                                                  |  33 kB  00:00:00     
(7/10): selinux-policy-3.12.1-71.fc19.noarch.rpm                                                 | 280 kB  00:00:00     
(8/10): selinux-policy-targeted-3.12.1-71.fc19.noarch.rpm                                        | 4.0 MB  00:00:02     
(9/10): evince-libs-3.8.3-2.fc19.x86_64.rpm                                                      | 300 kB  00:00:01     
(10/10): control-center-3.8.4.1-1.fc19.x86_64.rpm                                                | 4.0 MB  00:00:05     


Again, all packages available on the local mirror.

Comment 2 Zdeněk Pavlas 2013-08-26 10:22:15 UTC
The mirror selection algorithm was changed to mostly work without the fastestimirror plugin, it's probably time to deprecate it. Fastestmirror provides just the initial mirror ordering, but Yum picks the one that has been recently the fastest one. It's possible that the local mirror has lower latency (so fastestmirror picks it), but remote ones have better bandwidth.

Local mirrors (ones marked with mm0:private="True" in metalik.xml) are always used first, before non-local ones, but I assume www.fedora.is is probably not "local" in this sense.

Please grep www.fedora.is and ftp.pbone.net in /var/cache/yum/19/x86_64/updates/metalink.xml and in /var/cache/yum/19/x86_64/timedhosts and post the output, so I can tell what's going on.

Comment 3 Richard Allen 2013-08-26 11:33:51 UTC
[root@hamburger ~]# grep www.fedora.is /var/cache/yum/19/x86_64/updates/metalink.xml 
grep: /var/cache/yum/19/x86_64/updates/metalink.xml: No such file or directory
[root@hamburger ~]# grep www.fedora.is /var/cache/yum/x86_64/19/updates/metalink.xml 
        <url protocol="rsync" type="rsync" location="IS" preference="99" >rsync://www.fedora.is/fedora/updates/19/x86_64/repodata/repomd.xml</url>
        <url protocol="http" type="http" location="IS" preference="99" >http://www.fedora.is/fedora/updates/19/x86_64/repodata/repomd.xml</url>


[root@hamburger ~]# grep ftp.pbone.net /var/cache/yum/x86_64/19/updates/metalink.xml 
        <url protocol="ftp" type="ftp" location="PL" preference="61" >ftp://ftp.pbone.net/pub/fedora/linux/updates/19/x86_64/repodata/repomd.xml</url>
        <url protocol="http" type="http" location="PL" preference="61" >http://ftp.pbone.net/pub/fedora/linux/updates/19/x86_64/repodata/repomd.xml</url>
        <url protocol="rsync" type="rsync" location="PL" preference="61" >rsync://ftp.pbone.net/fedora/linux/updates/19/x86_64/repodata/repomd.xml</url>


Is there any way for clients to lock yum to a specific mirror?
Also, I cant see how any mirror would be faster for me than the one at my ISP (fedora.is at Iceland Telecom)

Comment 4 Richard Allen 2013-08-26 12:02:59 UTC
[root@hamburger ~]# grep ftp.pbone.net /var/cache/yum/x86_64/19/timedhosts*
/var/cache/yum/x86_64/19/timedhosts:ftp.pbone.net 0 1 1377086185
/var/cache/yum/x86_64/19/timedhosts.txt:ftp.pbone.net 1.18315815926


[root@hamburger ~]# grep fedora.is /var/cache/yum/x86_64/19/timedhosts*
/var/cache/yum/x86_64/19/timedhosts:www.fedora.is 991407 0 1373474669
/var/cache/yum/x86_64/19/timedhosts:ftp.fedora.is 708960 0 1373551082
/var/cache/yum/x86_64/19/timedhosts.txt:www.fedora.is 0.00026301765442
/var/cache/yum/x86_64/19/timedhosts.txt:ftp.fedora.is 0.00704216957092

Comment 5 Zdeněk Pavlas 2013-08-26 12:25:23 UTC
> www.fedora.is 991407 0 1373474669

Hmm, I think the problem is that your local mirror bandwidth is slightly below 1MBps, and the "default" speed that Yum uses to estimate bandwidth of hosts it has never downloaded from (or downloaded from them loong ago) is 1MBps.

So while www.fedora.is might be fast, it's not fast enough to convince Yum to stop trying other mirrors from time to time, in a hope it will find something faster.  Unfortunately, there's no yum option to override this, you can only edit /usr/lib/python2.7/site-packages/urlgrabber/grabber.py and change it there, eg.

-        self.default_speed = 1e6 # 1 MBit
+        self.default_speed = .5e6 # 0.5 MBit

I think this is an important option that we should make user-tunable in the future.  I'm leaving this BZ open.

Comment 6 Fedora End Of Life 2015-01-09 22:15:47 UTC
This message is a notice that Fedora 19 is now at end of life. Fedora 
has stopped maintaining and issuing updates for Fedora 19. It is 
Fedora's policy to close all bug reports from releases that are no 
longer maintained. Approximately 4 (four) weeks from now this bug will
be closed as EOL if it remains open with a Fedora 'version' of '19'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 19 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 7 Fedora End Of Life 2015-02-18 14:05:41 UTC
Fedora 19 changed to end-of-life (EOL) status on 2015-01-06. Fedora 19 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.