Bug 132241 - Download mirror list isn't being checked
Download mirror list isn't being checked
Status: CLOSED NEXTRELEASE
Product: Fedora Infrastructure
Classification: Retired
Component: website (Show other bugs)
unspecified
All Linux
medium Severity medium
: ---
: ---
Assigned To: Fedora Websites Team
Fanny Augustin
http://fedora.redhat.com/download/mir...
:
: 173228 (view as bug list)
Depends On:
Blocks: 173126 189533 193052
  Show dependency treegraph
 
Reported: 2004-09-10 03:29 EDT by Need Real Name
Modified: 2007-04-18 13:11 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2006-10-01 17:15:03 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
traceback (2.03 KB, text/plain)
2004-10-12 07:38 EDT, Need Real Name
no flags Details

  None (edit)
Description Need Real Name 2004-09-10 03:29:14 EDT
Description of problem:
Your mirror list isn't being checked.
 http://mirror.eas.muohio.edu/fedora/linux/core/2/$ARCH/os/
has been down for a while now.

Here's the line from /etc/sysconfig/rhn/sources:
 http://fedora.redhat.com/download/up2date-mirrors/fedora-core-2

up2date falls over with a horrible python error whenever it hits this
particular (broken) mirror. It worries my logs.
Comment 1 Lamont Peterson 2004-09-24 02:59:42 EDT
For months, on either x86 or AMD64 systems, if I ever see up2date
trying
http://mirror.eas.muohio.edu/fedora/linux/core/updates/2/x86_64//headers/header.info
to get information, it NEVER works.  I have not once been able to
contact that mirror (neither has anyone I have talked to).

The only recourse is to close up2date (which, many times, requires
crashing it) and restart it, hoping that it will not try
mirror.eas.muohio.edu the next go around.  I think this mirror needs
to be removed.

Another sugestion: if I had a way to blacklist certain mirrors just
for my machines then I would not have to play cat & mouse.

Another approach would be to automatically try another mirror and keep
track of how many times each mirror fails.  If it crosses a threshold
value (say either 3 failures in a row, 5 in a week, 10 in a month,
whatever) then it gets reported to download.fedora.redhat.com,
automatically.  Enough reports, the mirror gets dropped (manually). 
Of course, there are other details that would haee to be decided, in
order to be fair to the mirror oporators.

BTW:  It looks like this bug and bugs 121071 & 140763 are duplicates
of each other, with slightly differing approaches to encountering this
problem.

Is there a better place to report broken mirrors?
Comment 2 Habig, Alec 2004-10-05 10:36:38 EDT
This should be easy for the master repository people to fix, right? 
One of the big advantages of a dynamic mirror list - somebody just
needs to delete one line from the mirrors list which is being
distributed, and viola, the whole world stops having failures every
few runs of update.

The muohio mirror isn't down entirely though - just disallowing http,
but this link works fine:

ftp://mirror.eas.muohio.edu/mirrors/fedora/linux/core/updates/2/i386/

On a related tangent - having up2date spew the whole python stack dump
when it encounters an error is nice for developers, but wow does it
confuse the average user.  Trapping all that crap and instead printing
"mirror mirror.eas.muohio.edu is down, please try again" or "package
blah.rpm is a corrupt download, please remove
/var/spool/up2date/blah.rpm and try again" would be very very nice.
Comment 3 Need Real Name 2004-10-12 07:38:05 EDT
Created attachment 105055 [details]
traceback

Third traceback today. Posting traceback.
Comment 4 Need Real Name 2004-10-29 08:56:33 EDT
Acknowledged!

"We need a good way of verifying mirror content for listing the mirrors as
synchronized and for handing out accurate mirrorlists for yum and up2date. This
should include both pull mechanisms that crawl the mirrors and verify that the
data is accurate and consistent and push mechanism that let mirror admins submit
information about the consistency of their mirror. This is going to take a lot
of cooperation from the mirror admins and some coding to make this
infrastructure live."
 -- http://blog.sethdot.org/index.cgi/172
Comment 5 Need Real Name 2005-04-08 17:31:01 EDT
No activity since October. Closing.
Comment 6 Need Real Name 2005-11-15 17:21:41 EST
They care! Re-opening.
Comment 7 Need Real Name 2005-11-15 17:23:53 EST
*** Bug 173228 has been marked as a duplicate of this bug. ***
Comment 8 Patrick Barnes 2005-11-18 11:52:45 EST
It's time to get something done about this.  It seems a fairly regular
complaint, and it doesn't look very professional to have a bunch of dead mirrors.

I think the best approach to this would be an automated system to check for
request errors at regular intervals, combined with a documented reporting system
and maybe a human review occassionally.

Alternatively, we could try to set up some sort of a subscription system for
mirror providers, requiring them to check in with us occassionally or be removed
from the listing.  That could also be automated.

Finally, setting up a mailing list dedicated to mirror providers and vendors
might be a good idea.

Thoughts?

I'm assigning this to the web group and CC-ing Elliot Lee to get some attentive
eyes on this.
Comment 9 Luke Macken 2005-11-18 12:45:10 EST
We've been looking into this issue for a while now, and have agreed upon Bouncer
( http://osuosl.org/projects/bouncer/ ) to manage our mirrors.  The downside is
that we only have a dated snapshot of their code, in which one of our community
contributors hacked upon, then went MIA.  Upstream claims that they will be
opening this project up to the public upon the release of Firefox 1.5, so
hopefully the ball will start rolling then.  In the mean time, feel free to hop
in to #osuosl and bug them about it ;)
Comment 10 Patrick Barnes 2005-12-17 15:03:05 EST
Bouncer is now in a Subversion repository.  Hopefully we can start making some
progress on this now.

svn co https://svn.osuosl.org/public/bouncer/
Comment 11 Need Real Name 2006-03-15 05:41:33 EST
Is this ready for the FC5 release?
Comment 12 Jesse Keating 2006-03-15 06:55:11 EST
I do not believe so.  I'll be managing the mirror list to begin with.  Possibly
farmed out to the Fedora Websites group.
Comment 13 Need Real Name 2006-03-15 13:16:36 EST
(In reply to comment #12)
> I'll be managing the mirror list to begin with.
> Possibly farmed out to the Fedora Websites group.

Oh okay - why not let a perl script manage it?
Checking the mirror exists is not older than a week would be enough to fix the
bulk of the problems with the current system?
Comment 14 Jesse Keating 2006-03-15 13:39:39 EST
Wait.  I'm thinking of the yum mirror list files.  I have no idea if bouncer is
running for the other download stuff.
Comment 15 Need Real Name 2006-09-02 09:59:00 EDT
Will we have working mirrors for Fedora Core 6?
Comment 16 Need Real Name 2006-09-02 10:01:23 EDT
Please can this be marked as an FC6Blocker or similar.
Comment 17 Need Real Name 2006-09-06 16:37:52 EDT
Looks like you fixed it - thanks :)
https://www.redhat.com/archives/fedora-devel-list/2006-July/msg00409.html
Comment 18 Mike McGrath 2006-09-18 08:31:28 EDT
This is now being done at mirrors.fedoraproject.org

 http://mirrors.fedoraproject.org/mirrorlist?repo=rawhide&arch=i386

Comment 19 Mike McGrath 2006-10-01 17:15:03 EDT
I forgot to actually close this.  Mirrors are being checked.  You can get basic
stats at:

http://mirrors.fedoraproject.org/mirrorlist?repo=rawhide&arch=i386&stats=1&showinvalid=1

The intent is to make this script a bit more feature involved but I want to
ensure stability before I start farking with it.

Note You need to log in before you can comment on or make changes to this bug.