Red Hat Bugzilla – Bug 819886
RFE: Add retry logic for grinder retrieval of repodata
Last modified: 2012-08-24 09:26:57 EDT
Description of problem:
Currently, grinder provides for retries of package downloads. However, occasionally we run into issues (Akamai) with retrieval of repodata. As discussed with jmatthews, I'd like to add retry logic into grinder for repodata retrieval as well.
This was fixed for RHUI with this commit:
It is included in the 'rhui' branch of grinder that was based on grinder 0.0.136.
Created attachment 600764 [details]
I'm trying to verify the retry logic but it seems the @Retry() decorator isn't used for repomd.xml fetching as in the screen log attached threre are no retry logs dumped by the Grinder subsystem (or I just can't see any).
So I wanted to ask what cases should the @Retry() decorator activated for in terms of pulp-admin (rhui-manager) commands.
BTW my setup uses Squid proxy to simulate errors connecting (urlpath_regex .xml). See the attached screen log...
Grinder fetches files in 2 ways:
- Using yum
- Using BaseFetch
The @Retry decorator was added to the 'yum' piece. This gets invoked when the repo is being setup and yum throws an exception.
This should be the first step grinder runs. Looking at the logs I see an exception from fetching the distribution files, that falls under BaseFetch, which does not use the @Retry decorator.
Would you verify the version of grinder you are running.
Also for a quick test, create a repo to a BAD URL and attempt to sync it, note if you see retry attempts for this.
Created attachment 601220 [details]
Verifying screen log
With custom proxy settings, adding new Red Hat repository causes the grinder to retry downloading repomd.xml. This verifies the @Retry() decorator being called as expected. See the screen log attached ('tail -f /var/log/pulp/grinder.log' is being run in the background).
Grinder Version: 0.0.138
Changing to Verified based on comment 5
Technical note added. If any revisions are required, please edit the "Technical Notes" field
accordingly. All revisions will be proofread by the Engineering Content Services team.
A retry logic is needed in the grinder to avoid Akamai issues with retrieving repodata. @Retry decorator was added to the yum piece of the grinder and will be invoked to retry loading whenever yum returns an exception error.
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.