Hide Forgot
Description of problem: When doing a force full Capsule sync, Satellite incorrectly running the regenerate metadata tasks ("Actions::Pulp::Repository::DistributorPublish") on itself instead of running it on the Capsule. See the below output: 274: Actions::Pulp::Consumer::SyncCapsule (success) [ 4113.54s / 803.05s ] Started at: 2019-08-27 02:41:15 UTC Ended at: 2019-08-27 03:49:49 UTC Real time: 4113.54s Execution time (excluding suspended state): 803.05s Input: --- capsule_id: 2 repo_pulp_id: 80bd2431-8c57-45fe-a290-de4f311f6545 sync_options: remove_missing: true force_full: true Output: --- pulp_tasks: - exception: task_type: pulp.server.managers.repo.sync.sync _href: "/pulp/api/v2/tasks/1cb5b017-950e-4356-bbce-d9ed3f53e1a9/" task_id: 1cb5b017-950e-4356-bbce-d9ed3f53e1a9 tags: - pulp:repository:80bd2431-8c57-45fe-a290-de4f311f6545 - pulp:action:sync finish_time: '2019-08-27T02:42:03Z' _ns: task_status start_time: '2019-08-27T02:41:05Z' traceback: spawned_tasks: - _href: "/pulp/api/v2/tasks/81c33bcf-7ea7-4e3d-88d2-b9109fe8840a/" task_id: 81c33bcf-7ea7-4e3d-88d2-b9109fe8840a progress_report: yum_importer: ... queue: reserved_resource_worker-0@hao-capsule.example.com.dq2 <==== state: finished worker_name: reserved_resource_worker-0@hao-capsule.example.com <==== Sync task was running in the Capsule which is correct Input: --- pulp_id: 80bd2431-8c57-45fe-a290-de4f311f6545 distributor_type_id: yum_distributor source_pulp_id: override_config: force_full: true dependency: matching_content: ... Output: --- pulp_tasks: - exception: task_type: pulp.server.managers.repo.publish.publish _href: "/pulp/api/v2/tasks/539f81b0-529a-412d-a0dc-3bcc9b6a9831/" task_id: 539f81b0-529a-412d-a0dc-3bcc9b6a9831 tags: - pulp:repository:80bd2431-8c57-45fe-a290-de4f311f6545 - pulp:action:publish finish_time: '2019-08-27T03:50:07Z' _ns: task_status start_time: '2019-08-27T03:50:05Z' traceback: spawned_tasks: [] progress_report: ... queue: reserved_resource_worker-0@hao-satellite.example.com.dq2 state: finished worker_name: reserved_resource_worker-0@hao-satellite.example.com <====== Regenerate metadata was running in the Satellite. ... Version-Release number of selected component (if applicable): Satellite 6.4 and 6.5 How reproducible: always Steps to Reproduce: 1. Perform full Capsule sync from the Satellite web ui.
Created redmine issue http://projects.theforeman.org/issues/27776 from this bug
Moving this bug to POST for triage into Satellite 6 since the upstream issue https://projects.theforeman.org/issues/27776 has been resolved.
FailedQA. @Satellite 6.6.0 Snap22 tfm-rubygem-katello-3.12.0.27-1.el7sat.noarch Full Capsule sync now ends with the error "undefined method `[]' for nil:NilClass" shown per each repo being synced. Optimized sync works as expected. Not sure if the problem is caused by the fix. Steps to reproduce the new issue: 1) Create a CV and assign a repo to it 2) Publish and promote CV to a LFE 3) Assign a LFE to external capsule 4) Trigger Full Capsule sync Label: Actions::Katello::CapsuleContent::Sync Status: stopped Result: warning _____/Run\__________ 3: Actions::Pulp::Repository::RefreshNeeded (success) [ 0.06s / 0.06s ] 6: Actions::Pulp::Consumer::SyncCapsule (success) [ 11.95s / 0.29s ] 10: Actions::Pulp::Repository::DistributorPublish (skipped) [ 0.05s / 0.03s ]
Actions::Pulp::Repository::DistributorPublish failed with the error Started at: 2019-10-09 16:57:38 UTC Ended at: 2019-10-09 16:57:38 UTC Real time: 0.05s Execution time (excluding suspended state): 0.03s Input: --- smart_proxy_id: 4 options: force: true source_repository: id: 45 name: Capsule_6_6_RHEL7 url: http://<SAT_FQDN>/pulp/repos/Default_Organization/content_views/Capsule_6_6/5.0/custom/Capsule/Capsule_6_6_RHEL7/ matching_content: dependency: repository_creation: false dependency: repository_id: 34 remote_user: admin remote_cp_user: admin current_request_id: current_timezone: Europe/Prague current_user_id: 4 current_organization_id: 1 current_location_id: Output: --- {} Error: NoMethodError undefined method `[]' for nil:NilClass --- - "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.12.0.27/app/services/katello/pulp/repository.rb:168:in `lookup_distributor_id'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.12.0.27/app/services/katello/pulp/repository/yum.rb:57:in `distributors_to_publish'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.12.0.27/app/services/katello/pulp/repository.rb:160:in `distributor_publish'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.12.0.27/app/lib/actions/pulp/repository/distributor_publish.rb:24:in `invoke_external_task'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.2.3/lib/dynflow/action/polling.rb:83:in `initiate_external_action'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.2.3/lib/dynflow/action/polling.rb:18:in `run'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.2.3/lib/dynflow/action/cancellable.rb:13:in `run'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.12.0.27/app/lib/actions/pulp/abstract_async_task.rb:45:in `run'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.2.3/lib/dynflow/action.rb:538:in `block (3 levels) in execute_run'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.2.3/lib/dynflow/middleware/stack.rb:26:in `pass'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.2.3/lib/dynflow/middleware.rb:18:in `pass'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.12.0.27/app/lib/actions/middleware/skip_if_matching_content.rb:5:in `run'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.2.3/lib/dynflow/middleware/stack.rb:22:in `call'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.2.3/lib/dynflow/middleware/stack.rb:26:in `pass'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.2.3/lib/dynflow/middleware.rb:18:in `pass'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.2.3/lib/dynflow/middleware.rb:31:in `run'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.2.3/lib/dynflow/middleware/stack.rb:22:in `call'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.2.3/lib/dynflow/middleware/stack.rb:26:in `pass'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.2.3/lib/dynflow/middleware.rb:18:in `pass'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.12.0.27/app/lib/actions/middleware/remote_action.rb:16:in `block in run'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.12.0.27/app/lib/actions/middleware/remote_action.rb:40:in `block in as_remote_user'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.12.0.27/app/models/katello/concerns/user_extensions.rb:21:in `cp_config'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.12.0.27/app/lib/actions/middleware/remote_action.rb:27:in `as_cp_user'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.12.0.27/app/lib/actions/middleware/remote_action.rb:39:in `as_remote_user'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.12.0.27/app/lib/actions/middleware/remote_action.rb:16:in `run'" ...
VERIFIED. @satellite-6.7.0-4.beta.el7sat.noarch tfm-rubygem-katello-3.14.0-0.4.rc1.el7sat.noarch by the following manual reproducer: 1) Create a CV and assign a repo to it 2) Publish and promote CV to a LFE 3) Assign a LFE to external capsule 4) Trigger Full Capsule sync Label: Actions::Katello::CapsuleContent::Sync Status: stopped Result: success _____/Run\__________ 3: Actions::Pulp::Orchestration::Repository::RefreshRepos (success) [ 0.23s / 0.23s ] 7: Actions::Pulp::Consumer::SyncCapsule (success) [ 17.86s / 1.75s ] 10: Actions::Pulp::Repository::DistributorPublish (success) [ 1.88s / 0.26s ] >>> the action Actions::Pulp::Repository::DistributorPublish succeeded 5) Check where Actions::Pulp::Repository::DistributorPublish was run 10: Actions::Pulp::Repository::DistributorPublish (success) [ 1.88s / 0.26s ] Queue: default Started at: 2019-11-27 13:35:57 UTC Ended at: 2019-11-27 13:35:59 UTC Real time: 1.88s Execution time (excluding suspended state): 0.26s Input: --- smart_proxy_id: 2 options: force: true source_repository: id: 8 name: Red_Hat_Ansible_Engine_2_8_RPMs_for_Red_Hat_Enterprise_Linux_7_Server_x86_64 url: https://satellite.example.com/pulp/repos/Default_Organization/content_views/Test_View/1.0/content/dist/rhel/server/7/7Server/x86_64/ansible/2.8/os/ matching_content: false dependency: repository_creation: false dependency: repository_id: 19 remote_user: admin remote_cp_user: admin current_request_id: current_timezone: Europe/Prague current_user_id: 4 current_organization_id: 1 current_location_id: 2 Output: --- pulp_tasks: - exception: task_type: pulp.server.managers.repo.publish.publish _href: "/pulp/api/v2/tasks/2843b068-933f-4d33-86cd-d5cd28330695/" task_id: 2843b068-933f-4d33-86cd-d5cd28330695 tags: - pulp:repository:1-Test_View-Dev-fbae0800-7fc7-4a64-ad91-c71f8e901d34 - pulp:action:publish finish_time: '2019-11-27T13:35:59Z' _ns: task_status start_time: '2019-11-27T13:35:58Z' traceback: spawned_tasks: [] progress_report: ... queue: reserved_resource_worker-2@capsule.example.com.dq2 state: finished worker_name: reserved_resource_worker-2@capsule.example.com result: result: success ... >>> both queue: and worker_name: indicate that DistributorPublish is being performed on the capsule
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2020:1454