Description of problem: Pulp has the capability to ensure that we do not keep around RPMs that are no longer published in a repository or to recover from other issues with the consumed repositories based on repodata changes or otherwise. This feature is called 'remove_missing'. In Satellite 6.2, this is defaulted to on for all individual repositories that are enabled within pulp. There is the ability to signify this on a per-repository basis, or globally via pulp server configuration. I believe this should be in place here to handle changes that will/are occurring on the CDN that could result in customers filesystems filling up or otherwise filling in MongoDB with unnecessary data (as well as the shared storage). Version-Release number of selected component (if applicable): RHUI 3 Steps to Reproduce: 1. Sync any repo. 2. Check config output of any repo in mongo: # mongodb pulp_server > db.repo_importers.findOne() 3. See that there is no config statement for remove_missing. Actual results: When syncing repositories where the repodata checksum changes, or where Red Hat has removed packages from being published due to issues, RHUI does not clean these up or remove unused items. Instead, it duplicates the synced content. Expected results: We should make remove_missing the default for all enabled repositories, or set it globally. Additional info: To set remove_missing globally within a pulp instance, you can modify /etc/pulp/server/plugins.conf.d/yum_importer.json to include: "remove_missing": true, ( a la: sed -i 's/{/{ "remove_missing": true,/' /etc/pulp/server/plugins.conf.d/yum_importer.json ) This goes inside the same json block that is containing the proxy configs. Placing it here will enforce the setting across all repositories.
> 2. Check config output of any repo in mongo: > # mongodb pulp_server Did you mean # mongo pulp_database?
Radek, You are correct, sorry about that.
No worries. Just making sure we've got the right reproducer. Thanks!
Hello Craig, you mention that Satellite 6.2 solved that issue, did they set the global option in the configuration file or was it by passing an option when creating a repository? Second, what would you have us do with already existing instances? Should we attempt to rewrite the configuration on pulp rpm update or, which I think is better, clearly document that only new instances will have the option on and describe the option in the erratum docs. This way we do not change something the customer may want to use (i.e. the current absent default)
Milan, The standard in Satellite 6.2 was to create the repositories with that trait - or to set it true/false when a user modified the 'Mirror-on-Sync' setting within Satellite on a per-repository basis.
Actually, what's the use case in RHUI? RHUI can only sync repos from Red Hat CDN (*), and AFAIK we don't remove packages from CDN; we don't take back what we've once shipped. Therefore, it doesn't seem necessary to remove missing stuff as nothing should become missing. (*) OK, RHUI can also sync Docker containers and OSTree contents, but that's done outside of yum_importer, isn't it? Likewise, you can upload packages from a local directory to a custom repo, but that's not syncing per se.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:2649