Bug 1320794
Summary: | Task fails with "undefined method `cp_config' for" | ||||||||
---|---|---|---|---|---|---|---|---|---|
Product: | Red Hat Satellite | Reporter: | Pradeep Kumar Surisetty <psuriset> | ||||||
Component: | Content Management | Assignee: | Ivan Necas <inecas> | ||||||
Status: | CLOSED ERRATA | QA Contact: | Lukas Zapletal <lzap> | ||||||
Severity: | high | Docs Contact: | |||||||
Priority: | unspecified | ||||||||
Version: | 6.2.0 | CC: | ahuchcha, bbuckingham, brubisch, egolov, ehelms, erik-fedora, ggatward, inecas, jhutar, jmatthew, jmontleo, jsherril, lzap, mburgerh, mmccune, omaciel, oshtaier, peter.vreman, psuriset, sghai, sreber, tpapaioa, xdmoon | ||||||
Target Milestone: | Unspecified | Keywords: | Triaged | ||||||
Target Release: | Unused | ||||||||
Hardware: | Unspecified | ||||||||
OS: | Unspecified | ||||||||
Whiteboard: | |||||||||
Fixed In Version: | rubygem-katello-3.0.0.38-1 | Doc Type: | Bug Fix | ||||||
Doc Text: | Story Points: | --- | |||||||
Clone Of: | Environment: | ||||||||
Last Closed: | 2016-07-27 11:09:45 UTC | Type: | Bug | ||||||
Regression: | --- | Mount Type: | --- | ||||||
Documentation: | --- | CRM: | |||||||
Verified Versions: | Category: | --- | |||||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
Cloudforms Team: | --- | Target Upstream Version: | |||||||
Embargoed: | |||||||||
Bug Depends On: | |||||||||
Bug Blocks: | 1115190, 1122832, 1212602, 1335807 | ||||||||
Attachments: |
|
Description
Pradeep Kumar Surisetty
2016-03-24 02:47:17 UTC
Created attachment 1139819 [details]
dynflow error
Would it be possible to upload the whole sos report or foreman-debug from the issue: the screenshot is too little info to investigate Also, any chance that service restart happened in the meantime? I mean for the foreman-tasks service I havent restarted services. foreman debug , i will get it I was able to reproduce the issue when the task was triggered before the foreman-tasks service started, which lead to the `cp_config` method not being defined, because the starting happened before the class was fully extended. I was able to reproduce this once so far, and also did not restart any services. After the first time I haven't been able to reproduce it though, and I've tried the reproducer steps about a dozen times now. WORKAROUND: The following are a couple of options: - When simultaneously publishing multiple content views, perform them in smaller batches. For example, rather than publish 10 at a time, publish 5. - If the failure raised by this bug does occur, go in to Monitor->Tasks and perform a 'Resume' on any failed content view publish tasks. I've seen this issue as well, when i've seen it it was after a really really long amount of polling such as when a pulp worker is not responding properly or a gofer client picks up the task and doesn't respond (due to another bz). However ori saw this as well almost immediately after starting a CV publish though. So it seems quite random. Created attachment 1152985 [details]
Dynflow crash
This report give us much more info on what's going on there. The suspicious issue seems to be related to this patch in sequel https://github.com/jeremyevans/sequel/pull/1083/commits/6fc930d8a40e6f8b1630866e833d97a6df16c2fd I will try to get a solid reproducer and trying out a sequel upgrade after that to see, if it moves us forward with the issue *** Bug 1331732 has been marked as a duplicate of this bug. *** I've been able to hit the same "undefined method `cp_config'" error but while synchronizing repositories (both custom and RH repositories) on a Satellite 6.2.0 Beta system. Action: Actions::Pulp::Repository::Sync Exception: NoMethodError: undefined method `cp_config' for #<Class:0x00000009631010> Backtrace: /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/dynamic_matchers.rb:26:in `method_missing' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/app/lib/actions/middleware/remote_action.rb:27:in `as_cp_user' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/app/lib/actions/middleware/remote_action.rb:39:in `as_remote_user' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/app/lib/actions/middleware/remote_action.rb:16:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:22:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:26:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware.rb:17:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action/progress.rb:30:in `with_progress_calculation' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action/progress.rb:16:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:22:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:26:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware.rb:17:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/app/lib/actions/middleware/keep_locale.rb:11:in `block in run' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/app/lib/actions/middleware/keep_locale.rb:22:in `with_locale' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/app/lib/actions/middleware/keep_locale.rb:11:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:22:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:26:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware.rb:17:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware.rb:30:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:22:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/world.rb:30:in `execute' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:505:in `block (2 levels) in execute_run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:504:in `catch' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:504:in `block in execute_run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:419:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:419:in `block in with_error_handling' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:419:in `catch' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:419:in `with_error_handling' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:499:in `execute_run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:260:in `execute' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:9:in `block (2 levels) in execute' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/execution_plan/steps/abstract.rb:155:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/execution_plan/steps/abstract.rb:155:in `with_meta_calculation' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:8:in `block in execute' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:22:in `open_action' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:7:in `execute' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/executors/parallel/worker.rb:15:in `block in on_message' /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:74:in `block in assigns' /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:73:in `tap' /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:73:in `assigns' /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:56:in `match_value' /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:36:in `block in match?' /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:35:in `each' /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:35:in `match?' /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:23:in `match' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/executors/parallel/worker.rb:12:in `on_message' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/context.rb:46:in `on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/executes_context.rb:7:in `on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/actor.rb:26:in `on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/awaits.rb:15:in `on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/sets_results.rb:14:in `on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/buffer.rb:38:in `process_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/buffer.rb:31:in `process_envelopes?' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/buffer.rb:20:in `on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/termination.rb:55:in `on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/removes_child.rb:10:in `on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/sets_results.rb:14:in `on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/core.rb:161:in `process_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/core.rb:95:in `block in on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/core.rb:118:in `block (2 levels) in schedule_execution' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `block in synchronize' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `synchronize' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `synchronize' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/core.rb:115:in `block in schedule_execution' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:18:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:18:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:96:in `work' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:77:in `block in call_job' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:333:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:333:in `run_task' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:322:in `block (3 levels) in create_worker' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:305:in `loop' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:305:in `block (2 levels) in create_worker' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:304:in `catch' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:304:in `block in create_worker' /opt/theforeman/tfm/root/usr/share/gems/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in `block in create_with_logging_context' Also seeing this error on 6.2 Beta (customer installation) during sync tasks initiated from a daily sync plan. Seems to be various repos fail to sync with error: undefined method `cp_config' for #<Class:0x00000002ebcc18> Trying to use logger katello/pulp_rest which has not been configured. Output from Errors tab on tasks viewer: Action: Actions::Pulp::Repository::Sync Input: {"pulp_id"=> "Default_Organization-Red_Hat_Satellite_Capsule-Red_Hat_Satellite_Capsule_6_1_for_RHEL_7_Server_RPMs_x86_64", "task_id"=>"7d66cdf0-c60d-453c-90fb-de254868b44a", "source_url"=>nil, "remote_user"=>"admin", "remote_cp_user"=>"admin", "locale"=>"en"} Output: {"contents_changed"=>false, "pulp_tasks"=> [{"exception"=>nil, "task_type"=>"pulp.server.managers.repo.sync.sync", "_href"=>"/pulp/api/v2/tasks/7d66cdf0-c60d-453c-90fb-de254868b44a/", "task_id"=>"7d66cdf0-c60d-453c-90fb-de254868b44a", "tags"=> ["pulp:repository:Default_Organization-Red_Hat_Satellite_Capsule-Red_Hat_Satellite_Capsule_6_1_for_RHEL_7_Server_RPMs_x86_64", "pulp:action:sync"], "finish_time"=>"2016-05-09T14:10:47Z", "_ns"=>"task_status", "start_time"=>"2016-05-09T14:10:39Z", "traceback"=>nil, "spawned_tasks"=> [{"_href"=>"/pulp/api/v2/tasks/7a8c15fa-3e18-41b2-b3ef-31cdf117e6d3/", "task_id"=>"7a8c15fa-3e18-41b2-b3ef-31cdf117e6d3"}], "progress_report"=> {"yum_importer"=> {"content"=> {"items_total"=>0, "state"=>"FINISHED", "error_details"=>[], "details"=> {"rpm_total"=>0, "rpm_done"=>0, "drpm_total"=>0, "drpm_done"=>0}, "size_total"=>0, "size_left"=>0, "items_left"=>0}, "comps"=>{"state"=>"FINISHED"}, "purge_duplicates"=>{"state"=>"FINISHED"}, "distribution"=> {"items_total"=>0, "state"=>"FINISHED", "error_details"=>[], "items_left"=>0}, "errata"=>{"state"=>"FINISHED"}, "metadata"=>{"state"=>"FINISHED"}}}, "queue"=>"reserved_resource_worker-0.org.dq", "state"=>"finished", "worker_name"=>"reserved_resource_worker-0.org", "result"=> {"result"=>"success", "importer_id"=>"yum_importer", "exception"=>nil, "repo_id"=> "Default_Organization-Red_Hat_Satellite_Capsule-Red_Hat_Satellite_Capsule_6_1_for_RHEL_7_Server_RPMs_x86_64", "traceback"=>nil, "started"=>"2016-05-09T14:10:39Z", "_ns"=>"repo_sync_results", "completed"=>"2016-05-09T14:10:47Z", "importer_type_id"=>"yum_importer", "error_message"=>nil, "summary"=> {"content"=>{"state"=>"FINISHED"}, "comps"=>{"state"=>"FINISHED"}, "purge_duplicates"=>{"state"=>"FINISHED"}, "distribution"=>{"state"=>"FINISHED"}, "errata"=>{"state"=>"FINISHED"}, "metadata"=>{"state"=>"FINISHED"}}, "added_count"=>0, "removed_count"=>0, "updated_count"=>0, "id"=>"57309a679d9422570bb514b6", "details"=> {"content"=> {"size_total"=>0, "items_left"=>0, "items_total"=>0, "state"=>"FINISHED", "size_left"=>0, "details"=> {"rpm_total"=>0, "rpm_done"=>0, "drpm_total"=>0, "drpm_done"=>0}, "error_details"=>[]}, "comps"=>{"state"=>"FINISHED"}, "purge_duplicates"=>{"state"=>"FINISHED"}, "distribution"=> {"items_total"=>0, "state"=>"FINISHED", "error_details"=>[], "items_left"=>0}, "errata"=>{"state"=>"FINISHED"}, "metadata"=>{"state"=>"FINISHED"}}}, "error"=>nil, "_id"=>{"$oid"=>"573097e92d723599562be2ef"}, "id"=>"573097e92d723599562be2ef"}, {"exception"=>nil, "task_type"=>"pulp.server.managers.repo.publish.publish", "_href"=>"/pulp/api/v2/tasks/7a8c15fa-3e18-41b2-b3ef-31cdf117e6d3/", "task_id"=>"7a8c15fa-3e18-41b2-b3ef-31cdf117e6d3", "tags"=> ["pulp:repository:Default_Organization-Red_Hat_Satellite_Capsule-Red_Hat_Satellite_Capsule_6_1_for_RHEL_7_Server_RPMs_x86_64", "pulp:action:publish"], "finish_time"=>nil, "_ns"=>"task_status", "start_time"=>nil, "traceback"=>nil, "spawned_tasks"=>[], "progress_report"=>{}, "queue"=>"None.dq", "state"=>"waiting", "worker_name"=>nil, "result"=>nil, "error"=>nil, "_id"=>{"$oid"=>"57309a672d723599562be365"}, "id"=>"57309a672d723599562be365"}], "poll_attempts"=>{"total"=>90, "failed"=>0}} Exception: NoMethodError: undefined method `cp_config' for #<Class:0x00000002ebcc18> Backtrace: /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/dynamic_matchers.rb:26:in `method_missing' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/app/lib/actions/middleware/remote_action.rb:27:in `as_cp_user' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/app/lib/actions/middleware/remote_action.rb:39:in `as_remote_user' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/app/lib/actions/middleware/remote_action.rb:16:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:22:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:26:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware.rb:17:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action/progress.rb:30:in `with_progress_calculation' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action/progress.rb:16:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:22:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:26:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware.rb:17:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/app/lib/actions/middleware/keep_locale.rb:11:in `block in run' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/app/lib/actions/middleware/keep_locale.rb:22:in `with_locale' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/app/lib/actions/middleware/keep_locale.rb:11:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:22:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:26:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware.rb:17:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware.rb:30:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:22:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/world.rb:30:in `execute' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:505:in `block (2 levels) in execute_run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:504:in `catch' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:504:in `block in execute_run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:419:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:419:in `block in with_error_handling' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:419:in `catch' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:419:in `with_error_handling' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:499:in `execute_run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:260:in `execute' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:9:in `block (2 levels) in execute' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/execution_plan/steps/abstract.rb:155:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/execution_plan/steps/abstract.rb:155:in `with_meta_calculation' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:8:in `block in execute' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:22:in `open_action' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:7:in `execute' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/executors/parallel/worker.rb:15:in `block in on_message' /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:74:in `block in assigns' /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:73:in `tap' /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:73:in `assigns' /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:56:in `match_value' /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:36:in `block in match?' /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:35:in `each' /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:35:in `match?' /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:23:in `match' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/executors/parallel/worker.rb:12:in `on_message' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/context.rb:46:in `on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/executes_context.rb:7:in `on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/actor.rb:26:in `on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/awaits.rb:15:in `on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/sets_results.rb:14:in `on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/buffer.rb:38:in `process_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/buffer.rb:31:in `process_envelopes?' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/buffer.rb:20:in `on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/termination.rb:55:in `on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/removes_child.rb:10:in `on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/sets_results.rb:14:in `on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/core.rb:161:in `process_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/core.rb:95:in `block in on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/core.rb:118:in `block (2 levels) in schedule_execution' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `block in synchronize' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `synchronize' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `synchronize' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/core.rb:115:in `block in schedule_execution' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:18:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:18:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:96:in `work' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:77:in `block in call_job' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:333:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:333:in `run_task' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:322:in `block (3 levels) in create_worker' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:305:in `loop' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:305:in `block (2 levels) in create_worker' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:304:in `catch' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:304:in `block in create_worker' /opt/theforeman/tfm/root/usr/share/gems/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in `block in create_with_logging_context' Action: Actions::Pulp::Repository::Sync Input: {"pulp_id"=> "Default_Organization-Red_Hat_Satellite_Capsule-Red_Hat_Satellite_Capsule_6_1_for_RHEL_7_Server_RPMs_x86_64", "task_id"=>"7d66cdf0-c60d-453c-90fb-de254868b44a", "source_url"=>nil, "remote_user"=>"admin", "remote_cp_user"=>"admin", "locale"=>"en"} Output: {"contents_changed"=>false, "pulp_tasks"=> [{"exception"=>nil, "task_type"=>"pulp.server.managers.repo.sync.sync", "_href"=>"/pulp/api/v2/tasks/7d66cdf0-c60d-453c-90fb-de254868b44a/", "task_id"=>"7d66cdf0-c60d-453c-90fb-de254868b44a", "tags"=> ["pulp:repository:Default_Organization-Red_Hat_Satellite_Capsule-Red_Hat_Satellite_Capsule_6_1_for_RHEL_7_Server_RPMs_x86_64", "pulp:action:sync"], "finish_time"=>"2016-05-09T14:10:47Z", "_ns"=>"task_status", "start_time"=>"2016-05-09T14:10:39Z", "traceback"=>nil, "spawned_tasks"=> [{"_href"=>"/pulp/api/v2/tasks/7a8c15fa-3e18-41b2-b3ef-31cdf117e6d3/", "task_id"=>"7a8c15fa-3e18-41b2-b3ef-31cdf117e6d3"}], "progress_report"=> {"yum_importer"=> {"content"=> {"items_total"=>0, "state"=>"FINISHED", "error_details"=>[], "details"=> {"rpm_total"=>0, "rpm_done"=>0, "drpm_total"=>0, "drpm_done"=>0}, "size_total"=>0, "size_left"=>0, "items_left"=>0}, "comps"=>{"state"=>"FINISHED"}, "purge_duplicates"=>{"state"=>"FINISHED"}, "distribution"=> {"items_total"=>0, "state"=>"FINISHED", "error_details"=>[], "items_left"=>0}, "errata"=>{"state"=>"FINISHED"}, "metadata"=>{"state"=>"FINISHED"}}}, "queue"=>"reserved_resource_worker-0.org.dq", "state"=>"finished", "worker_name"=>"reserved_resource_worker-0.org", "result"=> {"result"=>"success", "importer_id"=>"yum_importer", "exception"=>nil, "repo_id"=> "Default_Organization-Red_Hat_Satellite_Capsule-Red_Hat_Satellite_Capsule_6_1_for_RHEL_7_Server_RPMs_x86_64", "traceback"=>nil, "started"=>"2016-05-09T14:10:39Z", "_ns"=>"repo_sync_results", "completed"=>"2016-05-09T14:10:47Z", "importer_type_id"=>"yum_importer", "error_message"=>nil, "summary"=> {"content"=>{"state"=>"FINISHED"}, "comps"=>{"state"=>"FINISHED"}, "purge_duplicates"=>{"state"=>"FINISHED"}, "distribution"=>{"state"=>"FINISHED"}, "errata"=>{"state"=>"FINISHED"}, "metadata"=>{"state"=>"FINISHED"}}, "added_count"=>0, "removed_count"=>0, "updated_count"=>0, "id"=>"57309a679d9422570bb514b6", "details"=> {"content"=> {"size_total"=>0, "items_left"=>0, "items_total"=>0, "state"=>"FINISHED", "size_left"=>0, "details"=> {"rpm_total"=>0, "rpm_done"=>0, "drpm_total"=>0, "drpm_done"=>0}, "error_details"=>[]}, "comps"=>{"state"=>"FINISHED"}, "purge_duplicates"=>{"state"=>"FINISHED"}, "distribution"=> {"items_total"=>0, "state"=>"FINISHED", "error_details"=>[], "items_left"=>0}, "errata"=>{"state"=>"FINISHED"}, "metadata"=>{"state"=>"FINISHED"}}}, "error"=>nil, "_id"=>{"$oid"=>"573097e92d723599562be2ef"}, "id"=>"573097e92d723599562be2ef"}, {"exception"=>nil, "task_type"=>"pulp.server.managers.repo.publish.publish", "_href"=>"/pulp/api/v2/tasks/7a8c15fa-3e18-41b2-b3ef-31cdf117e6d3/", "task_id"=>"7a8c15fa-3e18-41b2-b3ef-31cdf117e6d3", "tags"=> ["pulp:repository:Default_Organization-Red_Hat_Satellite_Capsule-Red_Hat_Satellite_Capsule_6_1_for_RHEL_7_Server_RPMs_x86_64", "pulp:action:publish"], "finish_time"=>nil, "_ns"=>"task_status", "start_time"=>nil, "traceback"=>nil, "spawned_tasks"=>[], "progress_report"=>{}, "queue"=>"None.dq", "state"=>"waiting", "worker_name"=>nil, "result"=>nil, "error"=>nil, "_id"=>{"$oid"=>"57309a672d723599562be365"}, "id"=>"57309a672d723599562be365"}], "poll_attempts"=>{"total"=>90, "failed"=>0}} Action: Actions::Katello::Repository::IndexContent Input: {"id"=>22, "contents_changed"=>Step(3).output[:contents_changed], "current_user_id"=>1, "locale"=>"en"} Output: {} Action: Actions::Katello::Repository::CorrectChecksum Input: {"repo_id"=>22, "locale"=>"en"} Output: {} Exception: RuntimeError: Trying to use logger katello/pulp_rest which has not been configured. Backtrace: /usr/share/foreman/lib/foreman/logging.rb:54:in `logger' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/app/services/katello/pulp/server.rb:14:in `config' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/config/initializers/runcible.rb:3:in `pulp_server' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/app/models/katello/glue/pulp/repo.rb:15:in `block (2 levels) in included' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/app/lib/katello/lazy_accessor.rb:160:in `instance_eval' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/app/lib/katello/lazy_accessor.rb:160:in `run_initializer' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/app/lib/katello/lazy_accessor.rb:139:in `lazy_attribute_get' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/app/lib/katello/lazy_accessor.rb:60:in `block (2 levels) in lazy_accessor' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/app/models/katello/glue/pulp/repo.rb:26:in `block (2 levels) in included' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/app/lib/katello/lazy_accessor.rb:160:in `instance_eval' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/app/lib/katello/lazy_accessor.rb:160:in `run_initializer' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/app/lib/katello/lazy_accessor.rb:139:in `lazy_attribute_get' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/app/lib/katello/lazy_accessor.rb:60:in `block (2 levels) in lazy_accessor' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/app/models/katello/glue/pulp/repo.rb:678:in `find_distributor' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/app/models/katello/glue/pulp/repo.rb:97:in `pulp_checksum_type' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/app/lib/actions/katello/repository/correct_checksum.rb:12:in `finalize' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:528:in `block (2 levels) in execute_finalize' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:26:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:26:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware.rb:17:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action/progress.rb:30:in `with_progress_calculation' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action/progress.rb:22:in `finalize' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:22:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:26:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware.rb:17:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/app/lib/actions/middleware/keep_locale.rb:15:in `block in finalize' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/app/lib/actions/middleware/keep_locale.rb:22:in `with_locale' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/app/lib/actions/middleware/keep_locale.rb:15:in `finalize' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:22:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:26:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware.rb:17:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware.rb:38:in `finalize' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:22:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/world.rb:30:in `execute' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:527:in `block in execute_finalize' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:419:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:419:in `block in with_error_handling' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:419:in `catch' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:419:in `with_error_handling' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:526:in `execute_finalize' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:260:in `execute' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:9:in `block (2 levels) in execute' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/execution_plan/steps/abstract.rb:155:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/execution_plan/steps/abstract.rb:155:in `with_meta_calculation' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:8:in `block in execute' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:22:in `open_action' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:7:in `execute' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/executors/parallel/sequential_manager.rb:68:in `run_step' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/executors/parallel/sequential_manager.rb:53:in `dispatch' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/executors/parallel/sequential_manager.rb:60:in `block in run_in_sequence' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/executors/parallel/sequential_manager.rb:60:in `each' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/executors/parallel/sequential_manager.rb:60:in `all?' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/executors/parallel/sequential_manager.rb:60:in `run_in_sequence' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/executors/parallel/sequential_manager.rb:49:in `dispatch' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/executors/parallel/sequential_manager.rb:27:in `block in finalize' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:26:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:26:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware.rb:17:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware.rb:46:in `finalize_phase' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:22:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:26:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware.rb:17:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware.rb:46:in `finalize_phase' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:22:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:26:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware.rb:17:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware.rb:46:in `finalize_phase' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:22:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:26:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware.rb:17:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware.rb:46:in `finalize_phase' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:22:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:26:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware.rb:17:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/common/transaction.rb:16:in `block in rollback_on_error' /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/abstract/database_statements.rb:201:in `block in transaction' /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/abstract/database_statements.rb:209:in `within_new_transaction' /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/abstract/database_statements.rb:201:in `transaction' /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/transactions.rb:208:in `transaction' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/transaction_adapters/active_record.rb:5:in `transaction' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/common/transaction.rb:15:in `rollback_on_error' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/common/transaction.rb:9:in `finalize_phase' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:22:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/world.rb:30:in `execute' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/executors/parallel/sequential_manager.rb:26:in `finalize' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/executors/parallel/worker.rb:18:in `block in on_message' /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:74:in `block in assigns' /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:73:in `tap' /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:73:in `assigns' /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:56:in `match_value' /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:36:in `block in match?' /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:35:in `each' /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:35:in `match?' /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:23:in `match' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/executors/parallel/worker.rb:12:in `on_message' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/context.rb:46:in `on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/executes_context.rb:7:in `on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/actor.rb:26:in `on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/awaits.rb:15:in `on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/sets_results.rb:14:in `on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/buffer.rb:38:in `process_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/buffer.rb:31:in `process_envelopes?' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/buffer.rb:20:in `on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/termination.rb:55:in `on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/removes_child.rb:10:in `on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/sets_results.rb:14:in `on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/core.rb:161:in `process_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/core.rb:95:in `block in on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/core.rb:118:in `block (2 levels) in schedule_execution' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `block in synchronize' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `synchronize' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `synchronize' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/core.rb:115:in `block in schedule_execution' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:18:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:18:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:96:in `work' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:77:in `block in call_job' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:333:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:333:in `run_task' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:322:in `block (3 levels) in create_worker' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:305:in `loop' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:305:in `block (2 levels) in create_worker' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:304:in `catch' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:304:in `block in create_worker' /opt/theforeman/tfm/root/usr/share/gems/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in `block in create_with_logging_context' Action: Actions::Katello::Repository::UpdateMedia Input: {"repo_id"=>22, "contents_changed"=>Step(3).output[:contents_changed], "locale"=>"en"} Output: {} Action: Actions::Katello::Repository::ErrataMail Input: {"repo"=>22, "last_updated"=>"2016-05-02 06:47:59 UTC", "contents_changed"=>Step(3).output[:contents_changed], "locale"=>"en"} Output: {} Action: Actions::Katello::Repository::ErrataMail Input: {"repo"=>22, "last_updated"=>"2016-05-02 06:47:59 UTC", "contents_changed"=>Step(3).output[:contents_changed], "locale"=>"en"} Output: {} Action: Actions::Katello::Repository::Sync Input: {"repository"=> {"id"=>22, "name"=>"Red Hat Satellite Capsule 6.1 for RHEL 7 Server RPMs x86_64", "label"=>"Red_Hat_Satellite_Capsule_6_1_for_RHEL_7_Server_RPMs_x86_64"}, "product"=> {"id"=>5, "name"=>"Red Hat Satellite Capsule", "label"=>"Red_Hat_Satellite_Capsule", "cp_id"=>"269"}, "provider"=>{"id"=>2, "name"=>"Red Hat"}, "organization"=>{"id"=>1, "name"=>"Default_Organization", "label"=>"Default_Organization"}, "services_checked"=>["pulp", "pulp_auth"], "id"=>22, "sync_result"=>Step(3).output, "user_id"=>1, "contents_changed"=>Step(3).output[:contents_changed], "current_user_id"=>1, "locale"=>"en"} Output: {} Action: Actions::Pulp::Repository::RegenerateApplicability Input: {"pulp_id"=> "Default_Organization-Red_Hat_Satellite_Capsule-Red_Hat_Satellite_Capsule_6_1_for_RHEL_7_Server_RPMs_x86_64", "contents_changed"=>Step(3).output[:contents_changed], "remote_user"=>"admin", "remote_cp_user"=>"admin", "locale"=>"en"} Output: {} Action: Actions::Katello::Repository::ImportApplicability Input: {"repo_id"=>22, "contents_changed"=>Step(3).output[:contents_changed], "locale"=>"en"} Output: {} I also having this issue by RedHat Repository sync of 34 Repositories Additional to the error above there is also an error on the dynflow world: 2016-05-10 13:36:30 [foreman-tasks/dynflow] [W] Error on on_execution_plan_save event | RuntimeError: The Dynflow world was not initialized yet. If your plugin uses it, make sure to call ForemanTasks.dynflow.require! in some initializer | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.14.3/lib/foreman_tasks/dynflow.rb:73:in `world' | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.14.3/app/models/foreman_tasks/task/dynflow_task.rb:48:in `execution_plan' | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.14.3/app/models/foreman_tasks/task/dynflow_task.rb:82:in `main_action' | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.14.3/app/models/foreman_tasks/task/dynflow_task.rb:17:in `update_from_dynflow' | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.14.3/lib/foreman_tasks/dynflow/persistence.rb:46:in `on_execution_plan_save' | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.14.3/lib/foreman_tasks/dynflow/persistence.rb:14:in `block in save_execution_plan' | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.14.3/lib/foreman_tasks/dynflow/persistence.rb:12:in `tap' | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.14.3/lib/foreman_tasks/dynflow/persistence.rb:12:in `save_execution_plan' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/persistence.rb:49:in `save_execution_plan' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/execution_plan.rb:331:in `save' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/execution_plan.rb:83:in `update_state' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/world.rb:294:in `invalidate_execution_lock' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/world.rb:274:in `block (2 levels) in invalidate' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/world.rb:273:in `each' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/world.rb:273:in `block in invalidate' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/coordinator.rb:229:in `call' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/coordinator.rb:229:in `acquire' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/world.rb:266:in `invalidate' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/world.rb:322:in `block in worlds_validity_check' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/world.rb:316:in `each' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/world.rb:316:in `worlds_validity_check' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/world.rb:40:in `initialize' | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.14.3/lib/foreman_tasks/dynflow/configuration.rb:58:in `new' | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.14.3/lib/foreman_tasks/dynflow/configuration.rb:58:in `initialize_world' | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.14.3/lib/foreman_tasks/dynflow.rb:40:in `initialize!' | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.14.3/lib/foreman_tasks/engine.rb:138:in `block in <class:Engine>' I'm trying to reproduce this issue locally. Does this issue occur both on rhel6 and rhel7? (In reply to Ivan Necas from comment #18) > I'm trying to reproduce this issue locally. Does this issue occur both on > rhel6 and rhel7? tried on rhel7.2 only I thought sat6.2 supports only on rhel7 I see the issue on RHEL6.7 Thanks. I have a local reproducer now (it's an intermitted issue, so the reproducer is not 100% but it gives us statistical information if the fix worked). I tried bisecting sequel and pg versions as I had some suspicions around thread safety there, but unfortunately I was able to reproduce this even on the newest sequel and pg versions. I have one more candidate fix to give a try before I search for other candidates for the segmentation fault. I've opened a BZ for the segfault https://bugzilla.redhat.com/show_bug.cgi?id=1335105 The proposed fix is submitted upstream. I've been able to reproduce this against pure dynflow and after the change, I haven't hit the issue with the same reproducer so far, so it looks promising. I will open different fix in foreman-tasks for the cp_config issue, that this BZ is mentioning. In my case the proposed fix https://github.com/Dynflow/dynflow/pull/185/commits/1510982b3d2b641ee4248cf6aa0ad9bd5392f828 solves the problem. `hammer task resume` should resume paused tasks and make continue when it got stuck Created redmine issue http://projects.theforeman.org/issues/15055 from this bug The fix for the undefined method cp_config propsed in foreman-tasks https://github.com/theforeman/foreman-tasks/pull/185 How to reproduce without running into the segfault: 1. `service foreman-tasks stop` 2. trigger repository synchronization 3. wait until it gets to planned/pending stated 4. `service foreman-tasks start` The error might not appear every time with the reproducer, but after serveral times it should show up, when the fix is not applied. The fix makes sure dynflow starts being active after other code is initialized - before the fix, there might be some code not initialize yet, as the cp_config method definition. I just tried to apply your patch against the Sat 6.2 beta, but after restarting everything (with katello-service restart) the webui only shows a fatal exception: 2016-05-17 17:51:08 [app] [W] Failed running foreman-tasks daemon | NameError: uninitialized constant Concerns::PulpDatabaseUnit | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_hooks-0.3.9/lib/foreman_hooks.rb:83:in `load_missing_constant_with_hooks' | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/app/models/katello/erratum.rb:3:in `<class:Erratum>' | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/app/models/katello/erratum.rb:2:in `<module:Katello>' | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.24/app/models/katello/erratum.rb:1:in `<top (required)>' | /opt/rh/rh-ruby22/root/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:54:in `require' <<snip>> The complete trace can be found at https://paste.fedoraproject.org/367629/63500844/raw/ Once I reverted the patch (and restarted everything) the webui became operational again without exceptions Thanks @Erik trying that and reporting feedback, I've tested it with slightly older version of satellite, where this issue wasn't. Here is an additional patch that should address the issue you've described: https://github.com/theforeman/foreman-tasks/pull/185/commits/96fd4af976ff026ec8aad1c8cf0095da66012bb9 More testing revealed other issue that might happen occasionally http://pastie.org/10842225 It's being caused by Katello initializing the plugin later than expected It was introduced in this commit https://github.com/Katello/katello/commit/d637aa2b48ea02b3b73efbd8168eab9629be8afe#diff-cf74e596d7887011413ee9d06b16b753R198 @eric any background on why this change was needed in the first place and if there is another way to handle this. The problem is we need to resume tasks, that are pending execution once the foreman-tasks executor starts and after_initialize is the last instance we can hook in in the load process to start the execution. Since the katello's plugin initializes itself in after_initialize as well, we occasionally hit issues such as pulp logger being not initialized yet by that time. @Ivan: thanks for the updated patch. I've just applied it to my Sat 6.2 beta testing environment and it seems to work fine so far (though more testing is needed). One thing I did notice is that after I applied the patch the Actions::Candlepin::ListenOnCandlepinEvents task was created and started automatically (according to the task history this task wasn't executed before on my local testing environment). It's probably unrelated to this patch, but it's a good thing that this task was created and executed. (recently I was having issues where this specific task was not being created on a Katello 3.0RC environment). The `Actions::Candlepin::ListenOnCandlepinEvents` task is a long-running task listening on QPID for subscription events: it's expected to be running there continuously and doesn't block anything else. I've opened additional PR in Katello to address the loggers not being initialized issue. https://github.com/Katello/katello/pull/6059 Upstream bug component is Content Management Foreman-tasks part of the fix merged https://github.com/theforeman/foreman-tasks/pull/185 Katello's part of the fix got in https://github.com/Katello/katello/pull/6059 @eric: how to make sure both PRs get cherry-picked as part of the fix? *** Bug 1339911 has been marked as a duplicate of this bug. *** *** Bug 1341062 has been marked as a duplicate of this bug. *** I can't see any exceptions when syncing with the upgraded packages on snap 13.1. Just for the record. I can't see the reported issue mentioned in duplicate of this bz 1339911 while performing upgrade from snap 6.1.9 -> sat 6.2 snap14 on rhel7 VERIFIED Synced 42 repositories (all RHEL platforms and Tools) without this issue: [root@ibm-ls22-03 ~]# rpm -q satellite katello tfm-rubygem-foreman-tasks tfm-rubygem-katello satellite-6.2.0-14.2.el7sat.noarch katello-3.0.0-6.el7sat.noarch tfm-rubygem-foreman-tasks-0.7.14.6-1.el7sat.noarch tfm-rubygem-katello-3.0.0.42-2.el7sat.noarch [root@ibm-ls22-03 ~]# grep cp_config /var/log/foreman/production.log Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2016:1501 |