Description of problem: Performed upgrade from sat6.2 -> 6.2.7 on rhel7 aw/ customer db. Upgrade was completed but I see errors on console at db_seed step : -- Upgrade Step: db_seed... [Sat_SERVER] out: E, [2017-01-13T06:20:01.007010 #16653] ERROR -- /connector-database-core: No executor available (Dynflow::Error) [Sat_SERVER] out: E, [2017-01-13T06:20:01.007245 #16653] ERROR -- /client-dispatcher: No executor available (Dynflow::Error) -- [Sat_SERVER] out: E, [2017-01-13T06:20:01.007010 #16653] ERROR -- /connector-database-core: No executor available (Dynflow::Error) [Sat_SERVER] out: /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.13.3/lib/dynflow/connectors/database.rb:142:in `any_executor' [Sat_SERVER] out: /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.13.3/lib/dynflow/connectors/database.rb:134:in `find_receiver' [Sat_SERVER] out: /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.13.3/lib/dynflow/connectors/database.rb:86:in `handle_envelope' [Sat_SERVER] out: /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.13.3/lib/dynflow/actor.rb:6:in `on_message' [Sat_SERVER] out: /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/context.rb:46:in `on_envelope' [Sat_SERVER] out: /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/executes_context.rb:7:in `on_envelope' [Sat_SERVER] out: E, [2017-01-13T06:20:01.007245 #16653] ERROR -- /client-dispatcher: No executor available (Dynflow::Error) [Sat_SERVER] out: /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.13.3/lib/dynflow/connectors/database.rb:142:in `any_executor' [Sat_SERVER] out: /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.13.3/lib/dynflow/connectors/database.rb:134:in `find_receiver' [Sat_SERVER] out: /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.13.3/lib/dynflow/connectors/database.rb:86:in `handle_envelope' [Sat_SERVER] out: /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.13.3/lib/dynflow/actor.rb:6:in `on_message' [Sat_SERVER] out: /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/context.rb:46:in `on_envelope' [Sat_SERVER] out: /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/executes_context.rb:7:in `on_envelope' Version-Release number of selected component (if applicable): upgrade sat6.2 to 6.2.7 How reproducible: once w/ customer db Steps to Reproduce: 1. used satellite-clone tool 2. 3. Actual results: Upgrade was completed however, got reported errors under "upgrade: db seed" step Expected results: no errors should be raised while performing upgrade Additional info:
Created attachment 1241771 [details] upgrade error dusing db:seed step
Created attachment 1241773 [details] foreman-debug
I was able to reproduce with the cust DB dump and rake db:migrate. I'll update once I have more details.
The customer's DB contains Dynflow tasks that are in running state. We may assume the DB dump was done while foreman-tasks was running and the tasks were not finished properly. During db:seed the Dynflow that is being initiated is trying to recover and is trying to restart the tasks. When no executor is running the error is risen. It is not harmful because the task will be resumed on next executor start, but the error should not be displayed. Changing the componnent to the Tasks plugin.
Created redmine issue http://projects.theforeman.org/issues/18206 from this bug
Minimal reproducer for this issue: 1/ Have Foreman/Satellite with at least one Dynflow task running. (Actions::Candlepin::ListenOnCandlepinEvents in Katello/Satellite is a good candidate) 2/ kill dynflow executor while the task is running: pkill -9 -f dynflow_executor 3/ foreman-rake db:migrate You should see errors in the output E, [2017-01-24T03:00:53.032499 #9544] ERROR -- /client-dispatcher: No executor available (Dynflow::Error)
Moving this bug to POST for triage into Satellite 6 since the upstream issue http://projects.theforeman.org/issues/18206 has been resolved.
*** Bug 1413575 has been marked as a duplicate of this bug. ***
VERIFIED on 6.2.10-2 using the reproducer in Comment #7, I was able to reproduce the issue on 6.2.9 and can confirm, that it no longer occurs on 6.2.10-2
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:1553
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days