Bug 819886 - RFE: Add retry logic for grinder retrieval of repodata
RFE: Add retry logic for grinder retrieval of repodata
Status: CLOSED ERRATA
Product: Red Hat Update Infrastructure for Cloud Providers
Classification: Red Hat
Component: RHUA (Show other bugs)
2.1
Unspecified Unspecified
high Severity high
: ---
: ---
Assigned To: mkovacik
mkovacik
: Triaged
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-05-08 09:36 EDT by Todd Sanders
Modified: 2012-08-24 09:26 EDT (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
A retry logic is needed in the grinder to avoid Akamai issues with retrieving repodata. @Retry decorator was added to the yum piece of the grinder and will be invoked to retry loading whenever yum returns an exception error.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-08-24 07:54:28 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
mkovacik: needinfo+


Attachments (Terms of Use)
Screen log (14.57 KB, text/plain)
2012-07-27 08:31 EDT, mkovacik
no flags Details
Verifying screen log (26.79 KB, text/plain)
2012-07-30 07:35 EDT, mkovacik
no flags Details

  None (edit)
Description Todd Sanders 2012-05-08 09:36:35 EDT
Description of problem:

Currently, grinder provides for retries of package downloads.  However, occasionally we run into issues (Akamai) with retrieval of repodata.  As discussed with jmatthews, I'd like to add retry logic into grinder for repodata retrieval as well.

-Todd
Comment 1 Todd Sanders 2012-05-08 09:37:50 EDT
Sprint 36
Comment 2 John Matthews 2012-06-25 11:11:00 EDT
This was fixed for RHUI with this commit:

http://git.fedorahosted.org/git/?p=grinder.git;a=commitdiff;h=7fad64d3f1f8b76204e5d5d6d75bd0d11bfbab16

It is included in the 'rhui' branch of grinder that was based on grinder 0.0.136.
http://git.fedorahosted.org/git/?p=grinder.git;a=shortlog;h=refs/heads/rhui
Comment 3 mkovacik 2012-07-27 08:31:58 EDT
Created attachment 600764 [details]
Screen log

Hi,

I'm trying to verify the retry logic but it seems the @Retry() decorator isn't used for repomd.xml fetching as in the screen log attached threre are no retry logs dumped by the Grinder subsystem (or I just can't see any).

So I wanted to ask what cases should the @Retry() decorator activated for in terms of pulp-admin (rhui-manager) commands.

BTW my setup uses Squid proxy to simulate errors connecting (urlpath_regex .xml). See the attached screen log...

Thanks!
Comment 4 John Matthews 2012-07-27 09:10:09 EDT
Grinder fetches files in 2 ways:
 - Using yum
 - Using BaseFetch

The @Retry decorator was added to the 'yum' piece.  This gets invoked when the repo is being setup and yum throws an exception.  

This should be the first step grinder runs.  Looking at the logs I see an exception from fetching the distribution files, that falls under BaseFetch, which does not use the @Retry decorator.

Would you verify the version of grinder you are running.


Also for a quick test, create a repo to a BAD URL and attempt to sync it, note if you see retry attempts for this.
Comment 5 mkovacik 2012-07-30 07:35:59 EDT
Created attachment 601220 [details]
Verifying screen log

With custom proxy settings, adding new Red Hat repository causes the grinder to retry downloading repomd.xml. This verifies the @Retry() decorator being called as expected. See the screen log attached ('tail -f /var/log/pulp/grinder.log' is being run in the background).

Grinder Version: 0.0.138
Comment 6 mkovacik 2012-07-30 07:37:29 EDT
Changing to Verified based on comment 5
Comment 7 Julie 2012-08-15 14:25:53 EDT
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
A retry logic is needed in the grinder to avoid Akamai issues with retrieving repodata. @Retry decorator was added to the yum piece of the grinder and will be invoked to retry loading whenever yum returns an exception error.
Comment 9 errata-xmlrpc 2012-08-24 07:54:28 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2012-1205.html

Note You need to log in before you can comment on or make changes to this bug.