Bug 580802

Summary:	yum-plugin-fastestmirror doesn't honor changes to mirrorlists
Product:	[Fedora] Fedora	Reporter:	Ralf Corsepius <rc040203>
Component:	yum-utils	Assignee:	Seth Vidal <skvidal>
Status:	CLOSED NOTABUG	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	high	Docs Contact:
Priority:	low
Version:	12	CC:	james.antill, matt_domsch, maxamillion, pmatilai, tim.lauridsen
Target Milestone:	---
Target Release:	---
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2010-04-09 14:07:02 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Ralf Corsepius 2010-04-09 05:29:41 UTC

Description of problem:

yum-plugin-fastestmirror doesn't honor changes to mirrorlists.

Version-Release number of selected component (if applicable):
yum-plugin-fastestmirror-1.1.26-1.fc12.noarch

How reproducible:
Always

Steps to Reproduce:
1. Set up a network of yum repo mirrors, using traditional mirrorlists on the master server. Let the mirrors sync.
2. Let a yum client using yum-plugin-fastestmirror pickup "its fastest mirror".
3. Remove the "fastest mirror" from the mirrorlist on the server.
4. Modify the repo on the master server, rebuild repodata.

Actual results:

Yum picks up a "fastest mirror". When this "fastest mirror" is being removed from the mirrorlist on the servers, yum on clients continues to use this "fastest mirror", i.e. it continues to use obsolete metadata, causing the client to not receive updates.

Scenarios, I encountered this with
* 3rd party repos which use "static mirrorlists", which only change when a mirror becomes unable, e.g. because when a mirror's admin decides to stop mirroring.
* 3rd party repos, which use "dynamic mirrorlists". e.g. master servers which compose mirrorlist files based on checking whether their mirrors are "in-sync".

Expected results:
yum-plugins-fastestmirror to check their "fastest mirror" for "correctness"/"being in sync with the master".

One way to achieve this would be to always download a mirrorlist from the master server, and to check whether the current "fastest mirror" is still in it.

Additional info:
Current yum-plugin-fastestmirror's behavior leads to yum preferring "dead mirrors" (mirrors which are responding to yum queries but have not seen updates), because "dead mirrors" tend to respond fast, because they tend not be running under heavy load.

Comment 1 James Antill 2010-04-09 14:07:02 UTC

fastestmirror _just_ has mirrorlist => speed, data.

The actual mirrorlist that is used always comes from yum. And that expires, like other repodata, at mirrorlist_expire / metadata_expire intervals. You can lower those timeouts, if you need all clients to see changes faster.

Also if you are using metalink instead of mirrorlist, then the metalink is checked at metadata_expire time (before repomd.xml) ... and it's possible to tell
yum which repomd.xml files are good.

Comment 2 Ralf Corsepius 2010-04-09 16:22:50 UTC

(In reply to comment #1)
> fastestmirror _just_ has mirrorlist => speed, data.
> 
> The actual mirrorlist that is used always comes from yum. And that expires,
> like other repodata, at mirrorlist_expire / metadata_expire intervals. You can
> lower those timeouts, if you need all clients to see changes faster.
Does not work: The repos I am able to reproduce this issue with all have
metadata_expire=0

> Also if you are using metalink instead of mirrorlist,
a) This would be only applicable to those repos under my control
b) Is a work-around and not a solution to this problem.

> then the metalink is
> checked at metadata_expire time (before repomd.xml) ... and it's possible to
> tell yum which repomd.xml files are good.    
From what I experienced earlier today, my feel is this doesn't work either.

Also, thanks for once more closing a BZ on an obvious and serious *defect* in yum and it's infrastructure. Thanks for demonstrating why reporting bug to Fedora is a waste of time.

It's experiences like these which are reponsible for considering the "Great Fedora Experience" to be a marketing hoax.

Comment 3 Matt Domsch 2010-04-09 16:31:20 UTC

There is a bug in MM which I was able to discover and fix after Ralf and Chuck Anderson both reported a similar failure last night.  The explanation and patch are here:
https://fedorahosted.org/pipermail/mirrormanager-commits/2010-April/000023.html

I'll get this addressed in production after F13 beta is out and the infrastructure change freeze is lifted.

Comment 4 James Antill 2010-04-09 16:42:08 UTC

> The repos I am able to reproduce this issue with all have metadata_expire=0

 If they are using mirrorlists instead of metalink you need mirrorlist_expire = 0, to expire the mirrorlists quickly.

> From what I experienced earlier today, my feel is this doesn't work either.

Mind sharing that experience? If you had maybe we could have helped you, in fact from thre text of the initial report it seemed very likely you did control the remote repos.

> Also, thanks for once more closing a BZ on an obvious and serious *defect* in
> yum and it's infrastructure. Thanks for demonstrating why reporting bug to
> Fedora is a waste of time.

The bug you reported is obviously not true:

 You said: fastestmirror chooses a mirror and sticks with it, even when mirrors change.

 Reality: fastestmirror mirror just sorts the mirrors it gets.

...I can't read minds to find out if there's an actual bug you may be hitting, so the only options I'm left with are closing the "bug" when it contradicts reality.