Red Hat Bugzilla – Bug 996198
CDS sync should not attempt to sync older packages which have been removed from RHUA by way of 'num_old_packages_keep'
Last modified: 2013-12-17 15:10:10 EST
Description of problem: A RHUA can be configured to not sync a configurable number of older packages, yet the CDS does not honor the same configuration. This results in a large number of 404 messages being logged on RHUA. The RHUA syncs repo metadata from CDN and preserves this metadata with preserve_metadata=True. The CDS syncs the metadata from the RHUA, then uses the package metadata to determine what should be synced from RHUA. In case of the RHUA not syncing old packages the CDS will still attempt to download these missing packages, resulting in excessive 404 messages being logged by Apache. Possible fix is to update CDS to honor the configuration options for number of old packages. We could consider having the RHUA configuration parameters passed to the CDS as part of the sync. Perhaps allowing a CDS configuration file as an additional override, so certain CDSes could be configured for storing even less older packages.
See bug 989506 for more information.
We have modified the RHUA to send the below sync options to the CDS. remove_old_versions num_old_pkgs_keep Additionally we have modified the CDS so it may override any options from the RHUA by looking for these options in "/etc/pulp/cds/cds.conf" Fix is in pulp 0.0.263-37 Below are the associated commits. https://github.com/splice/pulp/commit/f85ef0477fd7bd24de810fc3788d9bc95b843079 https://github.com/splice/pulp/commit/33eb99928c42a0ef44850934238b6a3c8b784b7a https://github.com/splice/pulp/commit/f9d840eb51d7b395af7636901d5f5d204f9e35bd https://github.com/splice/pulp/commit/438e944f22d35393497023fb07cce23e8e9bd366 https://github.com/splice/pulp/commit/a9ce91818dae5e92c623b12ea571d6257869c819 Please note, we ran into an unexpected 'feature' with INIConfig on RHEL-6. Namely, we had to rename the option 'remove_old_versions' to 'cds_remove_old_versions', this is only for /etc/pulp/cds/cds.conf. Reason for rename is: - CDS runs as a Gofer plugin. - Gofer uses INIConfig to parse the config file. - INIConfig honors DOS batch file syntax of 'rem' being a comment Therefore any lines beginning with 'rem' such as 'remove_old_versions. is interpreted to be a comment. - Our workaround is for the CDS to rename 'remove_old_versions' to 'cds_remove_old_versions' QE: Options explained: remove_old_versions: is a boolean, true means we will limit older package synced, false means we will not. num_old_pkgs_keep: is an integer, if remove_old_versions=true, then we will look at this value and keep this many old versions of a package. 0 means we keep none (i.e. only latest package is downloaded, 0 old packages are kept). 1 means we download latest package AND we download 1 old package Below are a few test scenario suggestions 1) RHUA, remove_old_versions=false CDS, remove_old_versions=false 2) RHUA, remove_old_versions=false CDS, remove_old_versions=true 3) RHUA, remove_old_versions=true, num_old_pkgs_keep=4 CDS, remove_old_versions=true, num_old_pkg_keep=1 Note, the below scenario will result in many 404's on RHUA http log, as the CDS will attempt to sync packages that dont exist on the RHUA. RHUA, remove_old_versions=true CDS, remove_old_versions=false
tested in pulp-0.0.263-38.git.1.7f842df.el6.noarch Move to Verified.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHBA-2013-1854.html