Hide Forgot
Description of problem: After renaming organization, doing some "get out, get in" safety routine in deleting manifest and reimporting it back, subsequent repository sync from CDN sometimes fails with yum metadata forbidden error. Interestingly, refreshing the manifest does not seem to trigger the bug. Just delete & import the same again. (but since the bz appears with some probability, I might not be (un)lucky enough when testing manifest refresh and repo sync) Version-Release number of selected component (if applicable): Sat 6.1.5 How reproducible: 50% Steps to Reproduce: 1. Have Sat6 in use (with manifest imported, repos enabled and synced) 2. Rename organization 3. Delete manifest, import the same again 4. Sync repositories (already synced before the step 1) - the more the better, since the bug occurs with some probability, randomly for random repos in individual test cases Actual results: some repos sync fails with: 3: Actions::Pulp::Repository::Sync (skipped) [ 35.34s / 2.66s ] Started at: 2016-01-24 10:32:32 UTC Ended at: 2016-01-24 10:33:07 UTC Real time: 35.34s Execution time (excluding suspended state): 2.66s Input: --- pulp_id: Default_Organization-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_7_Server_RPMs_x86_64_7Server task_id: remote_user: admin-9a78af3c remote_cp_user: admin locale: en Output: --- pulp_tasks: - exception: task_type: pulp.server.managers.repo.sync.sync _href: /pulp/api/v2/tasks/0eb2aa56-79f7-46a9-b5d9-7f0f8a40428b/ task_id: 0eb2aa56-79f7-46a9-b5d9-7f0f8a40428b tags: - pulp:repository:Default_Organization-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_7_Server_RPMs_x86_64_7Server - pulp:action:sync finish_time: '2016-01-24T10:33:04Z' _ns: task_status start_time: '2016-01-24T10:33:03Z' traceback: ! "Traceback (most recent call last):\n File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 240, in trace_task\n R = retval = fun(*args, **kwargs)\n File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 328, in __call__\n return super(Task, self).__call__(*args, **kwargs)\n \ File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 437, in __protected_call__\n return self.run(*args, **kwargs)\n File \"/usr/lib/python2.7/site-packages/pulp/server/managers/repo/sync.py\", line 114, in sync\n raise PulpExecutionException(_('Importer indicated a failed response'))\nPulpExecutionException: Importer indicated a failed response\n" spawned_tasks: [] progress_report: yum_importer: content: size_total: 0 items_left: 0 items_total: 0 state: NOT_STARTED size_left: 0 details: rpm_total: 0 rpm_done: 0 drpm_total: 0 drpm_done: 0 error_details: [] comps: state: NOT_STARTED distribution: items_total: 0 state: NOT_STARTED error_details: [] items_left: 0 errata: state: NOT_STARTED metadata: state: FAILED error: Forbidden queue: reserved_resource_worker-2.brq.redhat.com.dq state: error worker_name: reserved_resource_worker-2.brq.redhat.com result: error: code: PLP0000 data: {} description: Importer indicated a failed response sub_errors: [] _id: $oid: 56a4a841c441a805a18c3113 id: 56a4a841c44bb25302c1dd0d poll_attempts: total: 16 failed: 0 Error: Katello::Errors::PulpError PLP0000: Importer indicated a failed response --- - /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.77/app/lib/actions/pulp/abstract_async_task.rb:83:in `block in done?' - /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.77/app/lib/actions/pulp/abstract_async_task.rb:81:in `each' - /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.77/app/lib/actions/pulp/abstract_async_task.rb:81:in `done?' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action/polling.rb:28:in `run' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action/cancellable.rb:9:in `run' - /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.77/app/lib/actions/pulp/abstract_async_task.rb:57:in `run' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:487:in `block (3 levels) in execute_run' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:26:in `call' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:26:in `pass' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware.rb:16:in `pass' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware.rb:25:in `run' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:22:in `call' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:26:in `pass' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware.rb:16:in `pass' - /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.77/app/lib/actions/middleware/remote_action.rb:27:in `block in run' - /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.77/app/lib/actions/middleware/remote_action.rb:57:in `block (2 levels) in as_remote_user' - /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.77/app/lib/katello/util/thread_session.rb:84:in `pulp_config' - /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.77/app/lib/actions/middleware/remote_action.rb:43:in `as_pulp_user' - /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.77/app/lib/actions/middleware/remote_action.rb:56:in `block in as_remote_user' - /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.77/app/lib/katello/util/thread_session.rb:91:in `cp_config' - /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.77/app/lib/actions/middleware/remote_action.rb:38:in `as_cp_user' - /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.77/app/lib/actions/middleware/remote_action.rb:55:in `as_remote_user' - /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.77/app/lib/actions/middleware/remote_action.rb:27:in `run' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:22:in `call' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:26:in `pass' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware.rb:16:in `pass' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action/progress.rb:30:in `with_progress_calculation' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action/progress.rb:16:in `run' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:22:in `call' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:26:in `pass' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware.rb:16:in `pass' - /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.77/app/lib/actions/middleware/keep_locale.rb:23:in `block in run' - /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.77/app/lib/actions/middleware/keep_locale.rb:34:in `with_locale' - /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.77/app/lib/actions/middleware/keep_locale.rb:23:in `run' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/stack.rb:22:in `call' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/middleware/world.rb:30:in `execute' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:486:in `block (2 levels) in execute_run' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:485:in `catch' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:485:in `block in execute_run' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:402:in `call' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:402:in `block in with_error_handling' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:402:in `catch' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:402:in `with_error_handling' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:480:in `execute_run' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/action.rb:262:in `execute' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:9:in `block (2 levels) in execute' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan/steps/abstract.rb:155:in `call' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan/steps/abstract.rb:155:in `with_meta_calculation' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:8:in `block in execute' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:22:in `open_action' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:7:in `execute' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/executors/parallel/worker.rb:20:in `block in on_message' - /opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:859:in `block in assigns' - /opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:858:in `tap' - /opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:858:in `assigns' - /opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:138:in `match_value' - /opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:116:in `block in match' - /opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:115:in `each' - /opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:115:in `match' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/executors/parallel/worker.rb:17:in `on_message' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/micro_actor.rb:82:in `on_envelope' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/micro_actor.rb:72:in `receive' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/micro_actor.rb:99:in `block (2 levels) in run' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/micro_actor.rb:99:in `loop' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/micro_actor.rb:99:in `block in run' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/micro_actor.rb:99:in `catch' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/micro_actor.rb:99:in `run' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.7.9/lib/dynflow/micro_actor.rb:13:in `block in initialize' - /opt/rh/ruby193/root/usr/share/gems/gems/logging-1.8.1/lib/logging/diagnostic_context.rb:323:in `call' - /opt/rh/ruby193/root/usr/share/gems/gems/logging-1.8.1/lib/logging/diagnostic_context.rb:323:in `block in create_with_logging_context' Expected results: no such failure Additional info:
(automated) testing scenario variant with manifest refresh, it never hit the pulp error. So the error appears _only_ when deleting and importing the manifest.
I don't know if this is related, but I recently deleted a manifest and re-imported but set a different CDN URL. When I did this, the UI showed the new URL, but "hammer repository info" showed the old one. This happened while I was going through the ROLE class on Satellite 6.1 and I made a mistake by not editing the CDN entry before import, and thus needed to re-import (well at least I thought I did).
BTW, I was logged in just now with my personal email. I now work at Red Hat and my email is joden.
Moving 6.2 bugs out to sat-backlog.
Thank you for your interest in Satellite 6. We have evaluated this request, and we do not expect this to be implemented in product in the foreseeable future. We are therefore closing this out as WONTFIX. If you have any concerns about this, please feel free to contact Rich Jerrido or Bryan Kearney. Thank you.