Description of problem: After upgrading to v6.9.1, the weekly cron "foreman-rake katello:publish_unpublished_repositories RAILS_ENV=production" from /etc/cron.d/katello fails on the Satellite. (And thus never executes against the rest of the capsules either) Version-Release number of selected component (if applicable): Red Hat Satellite 6.9.1 How reproducible: Unknown if this is widespread or isolated to a single satellite. Steps to Reproduce: 1. Either wait for the weekly execution or invoke the cleanup manually: # foreman-rake katello:publish_unpublished_repositories RAILS_ENV=production 2. Check logs for backtrace (attachment Actual results: Logs report that the pulp3 service on Smart proxy Id 1 may be down "[E|app|] Smart proxy with ID 1 may be down: There was an issue with the backend service pulp3: 503 Service Unavailable" The cleanup never proceeds to clean-up other smart proxies, however, if you specify a single non-satellite capsule with SMART_PROXY_ID= , it will succeed on that capsule. Expected results: Clean-up job succeeds, and proceeds to clean up other capsules. Additional info: To my knowledge, pulp3 should not be expected to be running in 6.9 yet.
Not certain the appropriate component for this rake task; however, Repositories may be a better fit.
Solution for this one may be same as for bug 1957464.
Created redmine issue https://projects.theforeman.org/issues/32623 from this bug
@dgross it isn't clear which rake task this BZ is targeting. In the description you mentioned `katello:publish_unpublished_repositories` but the "Smart proxy with ID 1 may be down" error matches the production log for `katello:delete_orphaned_content`. I see the cause for the `katello:delete_orphaned_content` error so I'll start by fixing that, but if there's also an issue with `katello:publish_unpublished_repositories` we'll need more info about it.
Apologies. Title is incorrect. This is 100% "katello:delete_orphaned_content"
Thanks Dylan. In that case, the PR associated to the redmine attached to this BZ should directly fix the problem.
*** Bug 1957464 has been marked as a duplicate of this bug. ***
Upstream bug assigned to iballou
Moving this bug to POST for triage into Satellite since the upstream issue https://projects.theforeman.org/issues/32623 has been resolved.
Just a follow-up: The upstream fix from https://projects.theforeman.org/issues/32623 does indeed allow the task to be successfully invoked and run to completion in the Satellite (and capsules) where I first encountered the issue.
Verified on 6.9.3 snap 1.0. # systemctl status pulpcore-api ● pulpcore-api.service - Pulp API Server Loaded: loaded (/etc/systemd/system/pulpcore-api.service; disabled; vendor preset: disabled) Active: inactive (dead) # foreman-rake katello:delete_orphaned_content Orphaned content deletion started in background. /var/log/foreman/production.log: 2021-06-07T10:51:31 [I|bac|] Task {label: Actions::Katello::OrphanCleanup::RemoveOrphans, id: 62b7c2cc-1f7c-419e-83a3-a695ed819337, execution_plan_id: 7d212dfa-d991-458c-92ae-d50f69de9f69} state changed: stopped result: success
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Satellite 6.9.3 Async Bug Fix Update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2021:2636