Bug 1745835
| Summary: | Full Capsule sync doesn't fix the broken repository metadata and results in ERROR: "undefined method `[]' for nil:NilClass" | |||
|---|---|---|---|---|
| Product: | Red Hat Satellite | Reporter: | Hao Chang Yu <hyu> | |
| Component: | Capsule - Content | Assignee: | Samir Jha <sajha> | |
| Status: | CLOSED ERRATA | QA Contact: | Lukas Pramuk <lpramuk> | |
| Severity: | high | Docs Contact: | ||
| Priority: | medium | |||
| Version: | 6.5.0 | CC: | bkearney, cmarinea, fperalta, jsherril, mawerner, mmccune, mschibli, pcreech, rraghuwa, sadas, sajha, spetrosi, vsedmik, zhunting | |
| Target Milestone: | 6.7.0 | Keywords: | Regression, Triaged | |
| Target Release: | Unused | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | rubygem-katello-3.12.0.21-1 | Doc Type: | Known Issue | |
| Doc Text: |
Full Capsule synchronization fails and does not fully publish new metadata.
|
Story Points: | --- | |
| Clone Of: | ||||
| : | 1769893 (view as bug list) | Environment: | ||
| Last Closed: | 2020-04-14 13:25:28 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
Created redmine issue http://projects.theforeman.org/issues/27776 from this bug Moving this bug to POST for triage into Satellite 6 since the upstream issue https://projects.theforeman.org/issues/27776 has been resolved. FailedQA. @Satellite 6.6.0 Snap22 tfm-rubygem-katello-3.12.0.27-1.el7sat.noarch Full Capsule sync now ends with the error "undefined method `[]' for nil:NilClass" shown per each repo being synced. Optimized sync works as expected. Not sure if the problem is caused by the fix. Steps to reproduce the new issue: 1) Create a CV and assign a repo to it 2) Publish and promote CV to a LFE 3) Assign a LFE to external capsule 4) Trigger Full Capsule sync Label: Actions::Katello::CapsuleContent::Sync Status: stopped Result: warning _____/Run\__________ 3: Actions::Pulp::Repository::RefreshNeeded (success) [ 0.06s / 0.06s ] 6: Actions::Pulp::Consumer::SyncCapsule (success) [ 11.95s / 0.29s ] 10: Actions::Pulp::Repository::DistributorPublish (skipped) [ 0.05s / 0.03s ] Actions::Pulp::Repository::DistributorPublish failed with the error
Started at: 2019-10-09 16:57:38 UTC
Ended at: 2019-10-09 16:57:38 UTC
Real time: 0.05s
Execution time (excluding suspended state): 0.03s
Input:
---
smart_proxy_id: 4
options:
force: true
source_repository:
id: 45
name: Capsule_6_6_RHEL7
url: http://<SAT_FQDN>/pulp/repos/Default_Organization/content_views/Capsule_6_6/5.0/custom/Capsule/Capsule_6_6_RHEL7/
matching_content:
dependency:
repository_creation: false
dependency:
repository_id: 34
remote_user: admin
remote_cp_user: admin
current_request_id:
current_timezone: Europe/Prague
current_user_id: 4
current_organization_id: 1
current_location_id:
Output:
--- {}
Error:
NoMethodError
undefined method `[]' for nil:NilClass
---
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.12.0.27/app/services/katello/pulp/repository.rb:168:in
`lookup_distributor_id'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.12.0.27/app/services/katello/pulp/repository/yum.rb:57:in
`distributors_to_publish'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.12.0.27/app/services/katello/pulp/repository.rb:160:in
`distributor_publish'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.12.0.27/app/lib/actions/pulp/repository/distributor_publish.rb:24:in
`invoke_external_task'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.2.3/lib/dynflow/action/polling.rb:83:in
`initiate_external_action'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.2.3/lib/dynflow/action/polling.rb:18:in
`run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.2.3/lib/dynflow/action/cancellable.rb:13:in
`run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.12.0.27/app/lib/actions/pulp/abstract_async_task.rb:45:in
`run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.2.3/lib/dynflow/action.rb:538:in
`block (3 levels) in execute_run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.2.3/lib/dynflow/middleware/stack.rb:26:in
`pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.2.3/lib/dynflow/middleware.rb:18:in
`pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.12.0.27/app/lib/actions/middleware/skip_if_matching_content.rb:5:in
`run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.2.3/lib/dynflow/middleware/stack.rb:22:in
`call'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.2.3/lib/dynflow/middleware/stack.rb:26:in
`pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.2.3/lib/dynflow/middleware.rb:18:in
`pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.2.3/lib/dynflow/middleware.rb:31:in
`run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.2.3/lib/dynflow/middleware/stack.rb:22:in
`call'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.2.3/lib/dynflow/middleware/stack.rb:26:in
`pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.2.3/lib/dynflow/middleware.rb:18:in
`pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.12.0.27/app/lib/actions/middleware/remote_action.rb:16:in
`block in run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.12.0.27/app/lib/actions/middleware/remote_action.rb:40:in
`block in as_remote_user'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.12.0.27/app/models/katello/concerns/user_extensions.rb:21:in
`cp_config'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.12.0.27/app/lib/actions/middleware/remote_action.rb:27:in
`as_cp_user'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.12.0.27/app/lib/actions/middleware/remote_action.rb:39:in
`as_remote_user'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.12.0.27/app/lib/actions/middleware/remote_action.rb:16:in
`run'"
...
VERIFIED. @satellite-6.7.0-4.beta.el7sat.noarch tfm-rubygem-katello-3.14.0-0.4.rc1.el7sat.noarch by the following manual reproducer: 1) Create a CV and assign a repo to it 2) Publish and promote CV to a LFE 3) Assign a LFE to external capsule 4) Trigger Full Capsule sync Label: Actions::Katello::CapsuleContent::Sync Status: stopped Result: success _____/Run\__________ 3: Actions::Pulp::Orchestration::Repository::RefreshRepos (success) [ 0.23s / 0.23s ] 7: Actions::Pulp::Consumer::SyncCapsule (success) [ 17.86s / 1.75s ] 10: Actions::Pulp::Repository::DistributorPublish (success) [ 1.88s / 0.26s ] >>> the action Actions::Pulp::Repository::DistributorPublish succeeded 5) Check where Actions::Pulp::Repository::DistributorPublish was run 10: Actions::Pulp::Repository::DistributorPublish (success) [ 1.88s / 0.26s ] Queue: default Started at: 2019-11-27 13:35:57 UTC Ended at: 2019-11-27 13:35:59 UTC Real time: 1.88s Execution time (excluding suspended state): 0.26s Input: --- smart_proxy_id: 2 options: force: true source_repository: id: 8 name: Red_Hat_Ansible_Engine_2_8_RPMs_for_Red_Hat_Enterprise_Linux_7_Server_x86_64 url: https://satellite.example.com/pulp/repos/Default_Organization/content_views/Test_View/1.0/content/dist/rhel/server/7/7Server/x86_64/ansible/2.8/os/ matching_content: false dependency: repository_creation: false dependency: repository_id: 19 remote_user: admin remote_cp_user: admin current_request_id: current_timezone: Europe/Prague current_user_id: 4 current_organization_id: 1 current_location_id: 2 Output: --- pulp_tasks: - exception: task_type: pulp.server.managers.repo.publish.publish _href: "/pulp/api/v2/tasks/2843b068-933f-4d33-86cd-d5cd28330695/" task_id: 2843b068-933f-4d33-86cd-d5cd28330695 tags: - pulp:repository:1-Test_View-Dev-fbae0800-7fc7-4a64-ad91-c71f8e901d34 - pulp:action:publish finish_time: '2019-11-27T13:35:59Z' _ns: task_status start_time: '2019-11-27T13:35:58Z' traceback: spawned_tasks: [] progress_report: ... queue: reserved_resource_worker-2.com.dq2 state: finished worker_name: reserved_resource_worker-2.com result: result: success ... >>> both queue: and worker_name: indicate that DistributorPublish is being performed on the capsule Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2020:1454 |
Description of problem: When doing a force full Capsule sync, Satellite incorrectly running the regenerate metadata tasks ("Actions::Pulp::Repository::DistributorPublish") on itself instead of running it on the Capsule. See the below output: 274: Actions::Pulp::Consumer::SyncCapsule (success) [ 4113.54s / 803.05s ] Started at: 2019-08-27 02:41:15 UTC Ended at: 2019-08-27 03:49:49 UTC Real time: 4113.54s Execution time (excluding suspended state): 803.05s Input: --- capsule_id: 2 repo_pulp_id: 80bd2431-8c57-45fe-a290-de4f311f6545 sync_options: remove_missing: true force_full: true Output: --- pulp_tasks: - exception: task_type: pulp.server.managers.repo.sync.sync _href: "/pulp/api/v2/tasks/1cb5b017-950e-4356-bbce-d9ed3f53e1a9/" task_id: 1cb5b017-950e-4356-bbce-d9ed3f53e1a9 tags: - pulp:repository:80bd2431-8c57-45fe-a290-de4f311f6545 - pulp:action:sync finish_time: '2019-08-27T02:42:03Z' _ns: task_status start_time: '2019-08-27T02:41:05Z' traceback: spawned_tasks: - _href: "/pulp/api/v2/tasks/81c33bcf-7ea7-4e3d-88d2-b9109fe8840a/" task_id: 81c33bcf-7ea7-4e3d-88d2-b9109fe8840a progress_report: yum_importer: ... queue: reserved_resource_worker-0.com.dq2 <==== state: finished worker_name: reserved_resource_worker-0.com <==== Sync task was running in the Capsule which is correct Input: --- pulp_id: 80bd2431-8c57-45fe-a290-de4f311f6545 distributor_type_id: yum_distributor source_pulp_id: override_config: force_full: true dependency: matching_content: ... Output: --- pulp_tasks: - exception: task_type: pulp.server.managers.repo.publish.publish _href: "/pulp/api/v2/tasks/539f81b0-529a-412d-a0dc-3bcc9b6a9831/" task_id: 539f81b0-529a-412d-a0dc-3bcc9b6a9831 tags: - pulp:repository:80bd2431-8c57-45fe-a290-de4f311f6545 - pulp:action:publish finish_time: '2019-08-27T03:50:07Z' _ns: task_status start_time: '2019-08-27T03:50:05Z' traceback: spawned_tasks: [] progress_report: ... queue: reserved_resource_worker-0.com.dq2 state: finished worker_name: reserved_resource_worker-0.com <====== Regenerate metadata was running in the Satellite. ... Version-Release number of selected component (if applicable): Satellite 6.4 and 6.5 How reproducible: always Steps to Reproduce: 1. Perform full Capsule sync from the Satellite web ui.