Bug 132241 - Download mirror list isn't being checked
Summary: Download mirror list isn't being checked
Status: CLOSED NEXTRELEASE
Alias: None
Product: Fedora Infrastructure
Classification: Retired
Component: website (Show other bugs)
(Show other bugs)
Version: unspecified
Hardware: All Linux
medium
medium
Target Milestone: ---
Assignee: Fedora Websites Team
QA Contact: Fanny Augustin
URL: http://fedora.redhat.com/download/mir...
Whiteboard:
Keywords:
: 173228 (view as bug list)
Depends On:
Blocks: 173126 189533 193052
TreeView+ depends on / blocked
 
Reported: 2004-09-10 07:29 UTC by Need Real Name
Modified: 2007-04-18 17:11 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2006-10-01 21:15:03 UTC
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
traceback (2.03 KB, text/plain)
2004-10-12 11:38 UTC, Need Real Name
no flags Details

Description Need Real Name 2004-09-10 07:29:14 UTC
Description of problem:
Your mirror list isn't being checked.
 http://mirror.eas.muohio.edu/fedora/linux/core/2/$ARCH/os/
has been down for a while now.

Here's the line from /etc/sysconfig/rhn/sources:
 http://fedora.redhat.com/download/up2date-mirrors/fedora-core-2

up2date falls over with a horrible python error whenever it hits this
particular (broken) mirror. It worries my logs.

Comment 1 Lamont Peterson 2004-09-24 06:59:42 UTC
For months, on either x86 or AMD64 systems, if I ever see up2date
trying
http://mirror.eas.muohio.edu/fedora/linux/core/updates/2/x86_64//headers/header.info
to get information, it NEVER works.  I have not once been able to
contact that mirror (neither has anyone I have talked to).

The only recourse is to close up2date (which, many times, requires
crashing it) and restart it, hoping that it will not try
mirror.eas.muohio.edu the next go around.  I think this mirror needs
to be removed.

Another sugestion: if I had a way to blacklist certain mirrors just
for my machines then I would not have to play cat & mouse.

Another approach would be to automatically try another mirror and keep
track of how many times each mirror fails.  If it crosses a threshold
value (say either 3 failures in a row, 5 in a week, 10 in a month,
whatever) then it gets reported to download.fedora.redhat.com,
automatically.  Enough reports, the mirror gets dropped (manually). 
Of course, there are other details that would haee to be decided, in
order to be fair to the mirror oporators.

BTW:  It looks like this bug and bugs 121071 & 140763 are duplicates
of each other, with slightly differing approaches to encountering this
problem.

Is there a better place to report broken mirrors?

Comment 2 Habig, Alec 2004-10-05 14:36:38 UTC
This should be easy for the master repository people to fix, right? 
One of the big advantages of a dynamic mirror list - somebody just
needs to delete one line from the mirrors list which is being
distributed, and viola, the whole world stops having failures every
few runs of update.

The muohio mirror isn't down entirely though - just disallowing http,
but this link works fine:

ftp://mirror.eas.muohio.edu/mirrors/fedora/linux/core/updates/2/i386/

On a related tangent - having up2date spew the whole python stack dump
when it encounters an error is nice for developers, but wow does it
confuse the average user.  Trapping all that crap and instead printing
"mirror mirror.eas.muohio.edu is down, please try again" or "package
blah.rpm is a corrupt download, please remove
/var/spool/up2date/blah.rpm and try again" would be very very nice.


Comment 3 Need Real Name 2004-10-12 11:38:05 UTC
Created attachment 105055 [details]
traceback

Third traceback today. Posting traceback.

Comment 4 Need Real Name 2004-10-29 12:56:33 UTC
Acknowledged!

"We need a good way of verifying mirror content for listing the mirrors as
synchronized and for handing out accurate mirrorlists for yum and up2date. This
should include both pull mechanisms that crawl the mirrors and verify that the
data is accurate and consistent and push mechanism that let mirror admins submit
information about the consistency of their mirror. This is going to take a lot
of cooperation from the mirror admins and some coding to make this
infrastructure live."
 -- http://blog.sethdot.org/index.cgi/172

Comment 5 Need Real Name 2005-04-08 21:31:01 UTC
No activity since October. Closing.

Comment 6 Need Real Name 2005-11-15 22:21:41 UTC
They care! Re-opening.

Comment 7 Need Real Name 2005-11-15 22:23:53 UTC
*** Bug 173228 has been marked as a duplicate of this bug. ***

Comment 8 Patrick Barnes 2005-11-18 16:52:45 UTC
It's time to get something done about this.  It seems a fairly regular
complaint, and it doesn't look very professional to have a bunch of dead mirrors.

I think the best approach to this would be an automated system to check for
request errors at regular intervals, combined with a documented reporting system
and maybe a human review occassionally.

Alternatively, we could try to set up some sort of a subscription system for
mirror providers, requiring them to check in with us occassionally or be removed
from the listing.  That could also be automated.

Finally, setting up a mailing list dedicated to mirror providers and vendors
might be a good idea.

Thoughts?

I'm assigning this to the web group and CC-ing Elliot Lee to get some attentive
eyes on this.


Comment 9 Luke Macken 2005-11-18 17:45:10 UTC
We've been looking into this issue for a while now, and have agreed upon Bouncer
( http://osuosl.org/projects/bouncer/ ) to manage our mirrors.  The downside is
that we only have a dated snapshot of their code, in which one of our community
contributors hacked upon, then went MIA.  Upstream claims that they will be
opening this project up to the public upon the release of Firefox 1.5, so
hopefully the ball will start rolling then.  In the mean time, feel free to hop
in to #osuosl and bug them about it ;)

Comment 10 Patrick Barnes 2005-12-17 20:03:05 UTC
Bouncer is now in a Subversion repository.  Hopefully we can start making some
progress on this now.

svn co https://svn.osuosl.org/public/bouncer/

Comment 11 Need Real Name 2006-03-15 10:41:33 UTC
Is this ready for the FC5 release?

Comment 12 Jesse Keating 2006-03-15 11:55:11 UTC
I do not believe so.  I'll be managing the mirror list to begin with.  Possibly
farmed out to the Fedora Websites group.

Comment 13 Need Real Name 2006-03-15 18:16:36 UTC
(In reply to comment #12)
> I'll be managing the mirror list to begin with.
> Possibly farmed out to the Fedora Websites group.

Oh okay - why not let a perl script manage it?
Checking the mirror exists is not older than a week would be enough to fix the
bulk of the problems with the current system?

Comment 14 Jesse Keating 2006-03-15 18:39:39 UTC
Wait.  I'm thinking of the yum mirror list files.  I have no idea if bouncer is
running for the other download stuff.

Comment 15 Need Real Name 2006-09-02 13:59:00 UTC
Will we have working mirrors for Fedora Core 6?

Comment 16 Need Real Name 2006-09-02 14:01:23 UTC
Please can this be marked as an FC6Blocker or similar.

Comment 17 Need Real Name 2006-09-06 20:37:52 UTC
Looks like you fixed it - thanks :)
https://www.redhat.com/archives/fedora-devel-list/2006-July/msg00409.html

Comment 18 Mike McGrath 2006-09-18 12:31:28 UTC
This is now being done at mirrors.fedoraproject.org

 http://mirrors.fedoraproject.org/mirrorlist?repo=rawhide&arch=i386



Comment 19 Mike McGrath 2006-10-01 21:15:03 UTC
I forgot to actually close this.  Mirrors are being checked.  You can get basic
stats at:

http://mirrors.fedoraproject.org/mirrorlist?repo=rawhide&arch=i386&stats=1&showinvalid=1

The intent is to make this script a bit more feature involved but I want to
ensure stability before I start farking with it.


Note You need to log in before you can comment on or make changes to this bug.