Description of problem: Optimize capsule sync failed undefined method `backend_service' for nil:NilClass undefined method `backend_service' for nil:NilClass undefined method `backend_service' for nil:NilClass undefined method `backend_service' for nil:NilClass OST0005: Fetch summary failed. Reason: GLib.Error('Unacceptable TLS certificate', 'g-tls-error-quark', 2) Dynflow showed 3: Actions::Pulp::Repository::Refresh (skipped) [ 2.64s / 0.06s ] NoMethodError undefined method `backend_service' for nil:NilClass --- - "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.15/app/lib/actions/pulp/repository/refresh.rb:17:in `run'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/action.rb:539:in `block (3 levels) in execute_run'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:26:in `pass'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware.rb:18:in `pass'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware.rb:31:in `run'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:22:in `call'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:26:in `pass'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware.rb:18:in `pass'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.15/app/lib/actions/middleware/remote_action.rb:16:in `block in run'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.15/app/lib/actions/middleware/remote_action.rb:40:in `block in as_remote_user'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.15/app/models/katello/concerns/user_extensions.rb:21:in `cp_config'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.15/app/lib/actions/middleware/remote_action.rb:27:in `as_cp_user'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.15/app/lib/actions/middleware/remote_action.rb:39:in `as_remote_user'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.15/app/lib/actions/middleware/remote_action.rb:16:in `run'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:22:in `call'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:26:in `pass'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware.rb:18:in `pass'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.14.4.3/app/lib/actions/middleware/rails_executor_wrap.rb:14:in `block in run'" - "/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/execution_wrapper.rb:87:in `wrap'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.14.4.3/app/lib/actions/middleware/rails_executor_wrap.rb:13:in `run'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:22:in `call'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:26:in `pass'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware.rb:18:in `pass'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/action/progress.rb:30:in `with_progress_calculation'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/action/progress.rb:16:in `run'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:22:in `call'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:26:in `pass'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware.rb:18:in `pass'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.15/app/lib/actions/middleware/keep_locale.rb:11:in `block in run'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.15/app/lib/actions/middleware/keep_locale.rb:22:in `with_locale'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.15/app/lib/actions/middleware/keep_locale.rb:11:in `run'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:22:in `call'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:26:in `pass'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware.rb:18:in `pass'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.14.4.3/app/lib/actions/middleware/keep_current_timezone.rb:15:in `block in run'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.14.4.3/app/lib/actions/middleware/keep_current_timezone.rb:44:in `restore_curent_timezone'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.14.4.3/app/lib/actions/middleware/keep_current_timezone.rb:15:in `run'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:22:in `call'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:26:in `pass'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware.rb:18:in `pass'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.14.4.3/app/lib/actions/middleware/keep_current_user.rb:15:in `block in run'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.14.4.3/app/lib/actions/middleware/keep_current_user.rb:43:in `restore_curent_user'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.14.4.3/app/lib/actions/middleware/keep_current_user.rb:15:in `run'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:22:in `call'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:26:in `pass'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware.rb:18:in `pass'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.14.4.3/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in `block in run'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.14.4.3/app/lib/actions/middleware/keep_current_taxonomies.rb:44:in `restore_current_taxonomies'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.14.4.3/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in `run'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:22:in `call'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:26:in `pass'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware.rb:18:in `pass'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware.rb:31:in `run'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/stack.rb:22:in `call'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/middleware/world.rb:30:in `execute'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/action.rb:538:in `block (2 levels) in execute_run'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/action.rb:537:in `catch'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/action.rb:537:in `block in execute_run'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/action.rb:451:in `block in with_error_handling'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/action.rb:451:in `catch'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/action.rb:451:in `with_error_handling'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/action.rb:532:in `execute_run'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/action.rb:278:in `execute'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:17:in `block (2 levels) in execute'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/execution_plan/steps/abstract.rb:162:in `with_meta_calculation'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:16:in `block in execute'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:30:in `open_action'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:15:in `execute'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/director.rb:43:in `execute'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/executors/parallel/worker.rb:14:in `block in on_message'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/executors.rb:12:in `run_user_code'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/executors/parallel/worker.rb:13:in `on_message'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/context.rb:46:in `on_envelope'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/executes_context.rb:7:in `on_envelope'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.6/lib/dynflow/actor.rb:26:in `on_envelope'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/awaits.rb:15:in `on_envelope'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/sets_results.rb:14:in `on_envelope'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/buffer.rb:38:in `process_envelope'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/buffer.rb:31:in `process_envelopes?'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/buffer.rb:20:in `on_envelope'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/termination.rb:55:in `on_envelope'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/removes_child.rb:10:in `on_envelope'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/sets_results.rb:14:in `on_envelope'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/core.rb:161:in `process_envelope'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/core.rb:95:in `block in on_envelope'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/core.rb:118:in `block (2 levels) in schedule_execution'" - "/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `block in synchronize'" - "/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `synchronize'" - "/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `synchronize'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/core.rb:115:in `block in schedule_execution'" - "/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/serialized_execution.rb:18:in `call'" - "/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/serialized_execution.rb:96:in `work'" - "/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/serialized_execution.rb:77:in `block in call_job'" - "/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:348:in `run_task'" - "/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:337:in `block (3 levels) in create_worker'" - "/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in `loop'" - "/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in `block (2 levels) in create_worker'" - "/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in `catch'" - "/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in `block in create_worker'" - "/opt/theforeman/tfm/root/usr/share/gems/gems/logging-2.2.2/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'" Version-Release number of selected component (if applicable): Satellite 6.5 snap 13 How reproducible: always Steps to Reproduce: 1. Upgrade from 6.4.1 to 6.5 2. Optimized capsule sync Actual results: Capsule sync failed. Expected results: Successful capsule sync. Additional info:
Additional info : All services are running while before and at time of capsule sync: # hammer ping candlepin: Status: ok Server Response: Duration: 37ms candlepin_auth: Status: ok Server Response: Duration: 28ms pulp: Status: ok Server Response: Duration: 139ms pulp_auth: Status: ok Server Response: Duration: 60ms foreman_tasks: Status: ok Server Response: Duration: 12ms [root@qe-sat6-upgrade-rhel7 ~]# katello-service status -b Redirecting to 'foreman-maintain service' Running Status Services ================================================================================ Get status of applicable services: Displaying the following service(s): rh-mongodb34-mongod, postgresql, qdrouterd, qpidd, squid, pulp_celerybeat, pulp_resource_manager, pulp_streamer, pulp_workers, smart_proxy_dynflow_core, tomcat, dynflowd, elasticsearch, httpd, puppetserver, foreman-proxy | displaying rh-mongodb34-mongod [OK] / displaying postgresql [OK] - displaying qdrouterd [OK] - displaying qpidd [OK] - displaying squid [OK] \ displaying pulp_celerybeat [OK] \ displaying pulp_resource_manager [OK] \ displaying pulp_streamer [OK] | displaying pulp_workers [OK] | displaying smart_proxy_dynflow_core [OK] | displaying tomcat [OK] / displaying dynflowd [OK] / displaying elasticsearch [OK] / displaying httpd [OK] - displaying puppetserver [OK] - displaying foreman-proxy [OK] - All services are running [OK] --------------------------------------------------------------------------------
Created redmine issue https://projects.theforeman.org/issues/25954 from this bug
Upstream bug assigned to jsherril
Moving this bug to POST for triage into Satellite 6 since the upstream issue https://projects.theforeman.org/issues/25954 has been resolved.
VERIFIED. @satellite-6.5.0-6.beta.el7sat.noarch @satellite-capsule-6.5.0-6.beta.el7sat.noarch tfm-rubygem-katello-3.10.0.21-1.el7sat.noarch by the reproducer described in comment #0: 1) Upgrade SAT and Capsule from 6.4 2) Run optimized sync for the capsule >>> Optimized sync for upgraded capsule succeeds
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2019:1222