Bug 999489 - Yum ignores fastestmirror
Yum ignores fastestmirror
Status: CLOSED EOL
Product: Fedora
Classification: Fedora
Component: yum (Show other bugs)
19
All Linux
unspecified Severity medium
: ---
: ---
Assigned To: packaging-team-maint
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-08-21 08:05 EDT by Richard Allen
Modified: 2017-06-06 08:14 EDT (History)
9 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-02-18 09:05:41 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Richard Allen 2013-08-21 08:05:57 EDT
Description of problem:
Yum seems to ignore fastestmirror

Version-Release number of selected component (if applicable):
yum-3.4.3-106.fc19.noarch
yum-utils-1.1.31-17.fc19.noarch
yum-plugin-fastestmirror-1.1.31-17.fc19.noarch

How reproducible:
Install yum-plugin-fastestmirror.   Have it determine your local mirror
Run yum update, see yum sometimes talking to other mirrors.


Steps to Reproduce:
1. Install yum-plugin-fastestmirror
2. Run yum update
3. see yum sometimes talking to other mirrors

Actual results:
Yum downloading from remote mirrors


Expected results:
Yum using local (fstest) mirror


Additional info:
Example:
[root@hamburger ~]# yum update
Loaded plugins: fastestmirror, langpacks, refresh-packagekit
adobe-linux-x86_64                                                                               |  951 B  00:00:00     
google-chrome                                                                                    |  951 B  00:00:00     
google-talkplugin                                                                                |  951 B  00:00:00     
rpmfusion-free-updates                                                                           | 3.3 kB  00:00:00     
rpmfusion-nonfree-updates                                                                        | 3.3 kB  00:00:00     
steam                                                                                            | 3.3 kB  00:00:00     
updates/19/x86_64/metalink                                                                       |  29 kB  00:00:00     
updates                                                                                          | 4.6 kB  00:00:00     
updates/19/x86_64/primary_db                                                                     | 7.3 MB  00:00:02     
(1/2): updates/19/x86_64/updateinfo                                                              | 674 kB  00:00:00     
(2/2): updates/19/x86_64/pkgtags                                                                 | 461 kB  00:00:01     
Loading mirror speeds from cached hostfile
 * fedora: www.fedora.is
 * rpmfusion-free: www.fedora.is
 * rpmfusion-free-updates: www.fedora.is
 * rpmfusion-nonfree: www.fedora.is
 * updates: www.fedora.is
Warning: Environment Group gnome-desktop-environment does not exist.
Warning: Environment Group kde-desktop-environment does not exist.
Resolving Dependencies
--> Running transaction check
---> Package fros.noarch 0:1.0-1.fc19 will be updated
---> Package fros.noarch 0:1.0-4.fc19 will be an update
---> Package ibus-typing-booster.noarch 0:1.2.2-1.fc19 will be updated
---> Package ibus-typing-booster.noarch 0:1.2.3-2.fc19 will be an update
[snip]

We can see that www.fedora.is is the fastest mirror.   This is completely correct so far.

[snip]
(26/33): qemu-system-xtensa-1.4.2-6.fc19.x86_64.rpm                                              | 1.8 MB  00:00:00     
(27/33): tcp_wrappers-7.6-75.fc19.x86_64.rpm                                                     |  78 kB  00:00:00     
(28/33): tcp_wrappers-libs-7.6-75.fc19.i686.rpm                                                  |  66 kB  00:00:00     
(29/33): tcp_wrappers-libs-7.6-75.fc19.x86_64.rpm                                                |  66 kB  00:00:00     
(30/33): qemu-system-x86-1.4.2-6.fc19.x86_64.rpm                                                 | 3.0 MB  00:00:01     
(31/33): qemu-user-1.4.2-6.fc19.x86_64.rpm                                                       | 5.9 MB  00:00:03     
(32/33): qemu-system-mips-1.4.2-6.fc19.x86_64.rpm                                                | 5.6 MB  00:00:07     
libisofs-1.3.0-1.fc19.x86_64.r FAILED                                          
http://ftp.pbone.net/pub/fedora/linux/updates/19/x86_64/libisofs-1.3.0-1.fc19.x86_64.rpm: [Errno 12] Timeout on http://ftp.pbone.net/pub/fedora/linux/updates/19/x86_64/libisofs-1.3.0-1.fc19.x86_64.rpm: (28, 'Connection timed out after 30112 milliseconds')
Trying other mirror.
(33/33): libisofs-1.3.0-1.fc19.x86_64.rpm                                                        | 172 kB  00:00:00     
------------------------------------------------------------------------------------------------------------------------
Total                                                                                   1.2 MB/s |  38 MB     00:30     
[snip]

Why is yum talking to ftp.pbone.net?

fedora.is is ok:

[root@hamburger ~]# wget http://www.fedora.is/fedora/updates/19/x86_64/libisofs-1.3.0-1.fc19.x86_64.rpm
--2013-08-21 12:03:38--  http://www.fedora.is/fedora/updates/19/x86_64/libisofs-1.3.0-1.fc19.x86_64.rpm
Resolving www.fedora.is (www.fedora.is)... 194.105.226.20
Connecting to www.fedora.is (www.fedora.is)|194.105.226.20|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 175832 (172K) [application/octet-stream]
Saving to: ‘libisofs-1.3.0-1.fc19.x86_64.rpm’

100%[==============================================================================>] 175,832     --.-K/s   in 0.02s   

2013-08-21 12:03:38 (8.20 MB/s) - ‘libisofs-1.3.0-1.fc19.x86_64.rpm’ saved [175832/175832]


Here in Iceland we have to pay for international bandwidth usage.  Local within the country is free.    This means that when yum fails to use our local mirror it costs us real money.
Comment 1 Richard Allen 2013-08-22 06:56:17 EDT
Today:

[root@hamburger ~]# yum update
Loaded plugins: fastestmirror, langpacks, refresh-packagekit
Loading mirror speeds from cached hostfile
 * fedora: www.fedora.is
 * rpmfusion-free: www.fedora.is
 * rpmfusion-free-updates: www.fedora.is
 * rpmfusion-nonfree: www.fedora.is
 * updates: www.fedora.is
Warning: Environment Group gnome-desktop-environment does not exist.
Warning: Environment Group kde-desktop-environment does not exist.
Resolving Dependencies
--> Running transaction check
[snip]
Downloading packages:
(1/10): control-center-filesystem-3.8.4.1-1.fc19.x86_64.rpm                                      |  15 kB  00:00:00     
(2/10): libcacard-1.4.2-7.fc19.x86_64.rpm                                                        |  70 kB  00:00:00     
evince-libs-3.8.3-2.fc19.x86_6 FAILED                                          
http://fedora-mirror01.rbc.ru/pub/fedora/linux/updates/19/x86_64/evince-libs-3.8.3-2.fc19.x86_64.rpm: [Errno 14] HTTP Error 404 - Not Found
Trying other mirror.
evince-nautilus-3.8.3-2.fc19.x FAILED                                          
http://ftp.byfly.by/pub/fedoraproject.org/linux/updates/19/x86_64/evince-nautilus-3.8.3-2.fc19.x86_64.rpm: [Errno 14] HTTP Error 404 - Not Found
Trying other mirror.
(3/10): python-boto-2.9.6-2.fc19.noarch.rpm                                                      | 1.2 MB  00:00:00     
selinux-policy-3.12.1-71.fc19. FAILED                                                 ] 1.3 MB/s | 5.3 MB  00:00:07 ETA 
ftp://ftp.funet.fi/pub/mirrors/fedora.redhat.com/pub/fedora/linux/updates/19/x86_64/selinux-policy-3.12.1-71.fc19.noarch.rpm: [Errno 14]  Error 550 - Given file does not exist
Trying other mirror.
(4/10): qemu-guest-agent-1.4.2-7.fc19.x86_64.rpm                                                 | 125 kB  00:00:01     
(5/10): evince-3.8.3-2.fc19.x86_64.rpm                                                           | 5.6 MB  00:00:02     
(6/10): evince-nautilus-3.8.3-2.fc19.x86_64.rpm                                                  |  33 kB  00:00:00     
(7/10): selinux-policy-3.12.1-71.fc19.noarch.rpm                                                 | 280 kB  00:00:00     
(8/10): selinux-policy-targeted-3.12.1-71.fc19.noarch.rpm                                        | 4.0 MB  00:00:02     
(9/10): evince-libs-3.8.3-2.fc19.x86_64.rpm                                                      | 300 kB  00:00:01     
(10/10): control-center-3.8.4.1-1.fc19.x86_64.rpm                                                | 4.0 MB  00:00:05     


Again, all packages available on the local mirror.
Comment 2 Zdeněk Pavlas 2013-08-26 06:22:15 EDT
The mirror selection algorithm was changed to mostly work without the fastestimirror plugin, it's probably time to deprecate it. Fastestmirror provides just the initial mirror ordering, but Yum picks the one that has been recently the fastest one. It's possible that the local mirror has lower latency (so fastestmirror picks it), but remote ones have better bandwidth.

Local mirrors (ones marked with mm0:private="True" in metalik.xml) are always used first, before non-local ones, but I assume www.fedora.is is probably not "local" in this sense.

Please grep www.fedora.is and ftp.pbone.net in /var/cache/yum/19/x86_64/updates/metalink.xml and in /var/cache/yum/19/x86_64/timedhosts and post the output, so I can tell what's going on.
Comment 3 Richard Allen 2013-08-26 07:33:51 EDT
[root@hamburger ~]# grep www.fedora.is /var/cache/yum/19/x86_64/updates/metalink.xml 
grep: /var/cache/yum/19/x86_64/updates/metalink.xml: No such file or directory
[root@hamburger ~]# grep www.fedora.is /var/cache/yum/x86_64/19/updates/metalink.xml 
        <url protocol="rsync" type="rsync" location="IS" preference="99" >rsync://www.fedora.is/fedora/updates/19/x86_64/repodata/repomd.xml</url>
        <url protocol="http" type="http" location="IS" preference="99" >http://www.fedora.is/fedora/updates/19/x86_64/repodata/repomd.xml</url>


[root@hamburger ~]# grep ftp.pbone.net /var/cache/yum/x86_64/19/updates/metalink.xml 
        <url protocol="ftp" type="ftp" location="PL" preference="61" >ftp://ftp.pbone.net/pub/fedora/linux/updates/19/x86_64/repodata/repomd.xml</url>
        <url protocol="http" type="http" location="PL" preference="61" >http://ftp.pbone.net/pub/fedora/linux/updates/19/x86_64/repodata/repomd.xml</url>
        <url protocol="rsync" type="rsync" location="PL" preference="61" >rsync://ftp.pbone.net/fedora/linux/updates/19/x86_64/repodata/repomd.xml</url>


Is there any way for clients to lock yum to a specific mirror?
Also, I cant see how any mirror would be faster for me than the one at my ISP (fedora.is at Iceland Telecom)
Comment 4 Richard Allen 2013-08-26 08:02:59 EDT
[root@hamburger ~]# grep ftp.pbone.net /var/cache/yum/x86_64/19/timedhosts*
/var/cache/yum/x86_64/19/timedhosts:ftp.pbone.net 0 1 1377086185
/var/cache/yum/x86_64/19/timedhosts.txt:ftp.pbone.net 1.18315815926


[root@hamburger ~]# grep fedora.is /var/cache/yum/x86_64/19/timedhosts*
/var/cache/yum/x86_64/19/timedhosts:www.fedora.is 991407 0 1373474669
/var/cache/yum/x86_64/19/timedhosts:ftp.fedora.is 708960 0 1373551082
/var/cache/yum/x86_64/19/timedhosts.txt:www.fedora.is 0.00026301765442
/var/cache/yum/x86_64/19/timedhosts.txt:ftp.fedora.is 0.00704216957092
Comment 5 Zdeněk Pavlas 2013-08-26 08:25:23 EDT
> www.fedora.is 991407 0 1373474669

Hmm, I think the problem is that your local mirror bandwidth is slightly below 1MBps, and the "default" speed that Yum uses to estimate bandwidth of hosts it has never downloaded from (or downloaded from them loong ago) is 1MBps.

So while www.fedora.is might be fast, it's not fast enough to convince Yum to stop trying other mirrors from time to time, in a hope it will find something faster.  Unfortunately, there's no yum option to override this, you can only edit /usr/lib/python2.7/site-packages/urlgrabber/grabber.py and change it there, eg.

-        self.default_speed = 1e6 # 1 MBit
+        self.default_speed = .5e6 # 0.5 MBit

I think this is an important option that we should make user-tunable in the future.  I'm leaving this BZ open.
Comment 6 Fedora End Of Life 2015-01-09 17:15:47 EST
This message is a notice that Fedora 19 is now at end of life. Fedora 
has stopped maintaining and issuing updates for Fedora 19. It is 
Fedora's policy to close all bug reports from releases that are no 
longer maintained. Approximately 4 (four) weeks from now this bug will
be closed as EOL if it remains open with a Fedora 'version' of '19'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 19 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.
Comment 7 Fedora End Of Life 2015-02-18 09:05:41 EST
Fedora 19 changed to end-of-life (EOL) status on 2015-01-06. Fedora 19 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.

Note You need to log in before you can comment on or make changes to this bug.