Bug 1328929

Summary: Stale EPEL 6 and 7 metalink data being served intermittently
Product: [Fedora] Fedora EPEL Reporter: Paul Clifford <paul.clifford+redhat>
Component: mirrorbrainAssignee: Andrea Veri <andrea.veri>
Status: CLOSED NOTABUG QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: epel7CC: andrea.veri
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-04-20 16:46:01 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Paul Clifford 2016-04-20 15:48:33 UTC
I'm not sure what the correct component should be; mirrorbrain seemed closest.


Description of problem:
Both of the following are occasionally serving up stale metalink data, causing yum to fail when it can't find a matching repodata/repomd.xml file on any mirror:
https://mirrors.fedoraproject.org/metalink?repo=epel-6&arch=x86_64
https://mirrors.fedoraproject.org/metalink?repo=epel-7&arch=x86_64

At the time of writing this, the current EPEL 6 repomd.xml data served in most responses has:
* timestamp: 1461024786 (April 19th)
* SHA256 digest: bd76e6a91810219ac2b9d2be4784ffb64a19328b5da225d2a9ce66f77f93629a

But roughly 10% of the time we get this instead:
* timestamp: 1460759382 (April 15th)
* SHA256 digest: e7a2b656815cb97b9027ee50e1867af04b9eaf71c87478733a57036f88e29676

Similarly, the current EPEL 7 repomd.xml data served in most responses has:
* timestamp: 1461022805 (April 19th)
* SHA256 digest: 503989ef36ac8d893f8674d834f2e9cc0e968633440cec640c05f7d3c21b2e6a

With the occasional stale response:
* timestamp: 1460757417 (April 15th)
* SHA256 digest: 027d4fbe183fe2535ca15737625d9fc18726838c7c12c324f3194681ccb132fc


Version-Release number of selected component (if applicable):
n/a


How reproducible:
On a small sample we seem to be getting the stale responses roughly 10% of the time.


Steps to Reproduce:
Request the above mirrors.fedoraproject.org URLs in a loop and watch the <mm0:timestamp> value sometimes jump back in time to a version of repomd.xml that the mirrors no longer have.


Actual results:
Occasionally stale metalink data is returned, and yum fails to find the matching repodata/repomd.xml file on any mirrors.


Expected results:
The metalink data should be recent.


Additional info:
These requests are being made from machines running in Amazon's eu-west-1 region, in Dublin.

Comment 1 Paul Clifford 2016-04-20 16:46:01 UTC
Fixed with the assistance of nirik in #fedora-admin