Bug 1413966 - Upgrade w/ customer db raises "No executor available (Dynflow::Error)" at db:seed step [NEEDINFO]
Summary: Upgrade w/ customer db raises "No executor available (Dynflow::Error)" at db:...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Tasks Plugin
Version: 6.2.7
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: Unspecified
Assignee: Martin Bacovsky
QA Contact: Roman Plevka
URL:
Whiteboard:
: 1413575 (view as bug list)
Depends On:
Blocks: Sat6_Upgrades
TreeView+ depends on / blocked
 
Reported: 2017-01-17 13:04 UTC by Sachin Ghai
Modified: 2020-08-13 08:49 UTC (History)
10 users (show)

Fixed In Version: rubygem-dynflow-0.8.13.5-1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1446733 (view as bug list)
Environment:
Last Closed: 2017-06-20 17:53:40 UTC
Target Upstream Version:
pmoravec: needinfo? (mmccune)


Attachments (Terms of Use)
upgrade error dusing db:seed step (20.00 KB, text/plain)
2017-01-17 13:06 UTC, Sachin Ghai
no flags Details
foreman-debug (2.84 MB, application/x-xz)
2017-01-17 13:12 UTC, Sachin Ghai
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Foreman Issue Tracker 18206 0 Normal Closed Upgrade raises "No executor available (Dynflow::Error)" at db:seed step 2019-12-27 14:31:24 UTC

Description Sachin Ghai 2017-01-17 13:04:40 UTC
Description of problem:
Performed upgrade from sat6.2 -> 6.2.7 on rhel7 aw/ customer db. Upgrade was completed but I see errors on console at db_seed step :

-- 
Upgrade Step: db_seed...


[Sat_SERVER]  out: E, [2017-01-13T06:20:01.007010 #16653] ERROR -- /connector-database-core: No executor available (Dynflow::Error)


[Sat_SERVER]  out: E, [2017-01-13T06:20:01.007245 #16653] ERROR -- /client-dispatcher: No executor available (Dynflow::Error)
-- 



[Sat_SERVER] out: E, [2017-01-13T06:20:01.007010 #16653] ERROR -- /connector-database-core: No executor available (Dynflow::Error)
[Sat_SERVER] out: /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.13.3/lib/dynflow/connectors/database.rb:142:in `any_executor'
[Sat_SERVER] out: /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.13.3/lib/dynflow/connectors/database.rb:134:in `find_receiver'
[Sat_SERVER] out: /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.13.3/lib/dynflow/connectors/database.rb:86:in `handle_envelope'
[Sat_SERVER] out: /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.13.3/lib/dynflow/actor.rb:6:in `on_message'
[Sat_SERVER] out: /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/context.rb:46:in `on_envelope'
[Sat_SERVER] out: /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/executes_context.rb:7:in `on_envelope'



[Sat_SERVER] out: E, [2017-01-13T06:20:01.007245 #16653] ERROR -- /client-dispatcher: No executor available (Dynflow::Error)
[Sat_SERVER] out: /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.13.3/lib/dynflow/connectors/database.rb:142:in `any_executor'
[Sat_SERVER] out: /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.13.3/lib/dynflow/connectors/database.rb:134:in `find_receiver'
[Sat_SERVER] out: /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.13.3/lib/dynflow/connectors/database.rb:86:in `handle_envelope'
[Sat_SERVER] out: /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.13.3/lib/dynflow/actor.rb:6:in `on_message'
[Sat_SERVER] out: /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/context.rb:46:in `on_envelope'
[Sat_SERVER] out: /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/executes_context.rb:7:in `on_envelope'

Version-Release number of selected component (if applicable):
upgrade sat6.2 to 6.2.7

How reproducible:
once w/ customer db

Steps to Reproduce:
1. used satellite-clone tool
2.
3.

Actual results:
Upgrade was completed however, got reported errors under "upgrade: db seed" step

Expected results:
no errors should be raised while performing upgrade

Additional info:

Comment 2 Sachin Ghai 2017-01-17 13:06:16 UTC
Created attachment 1241771 [details]
upgrade error dusing db:seed step

Comment 3 Sachin Ghai 2017-01-17 13:12:00 UTC
Created attachment 1241773 [details]
foreman-debug

Comment 4 Martin Bacovsky 2017-01-18 14:03:14 UTC
I was able to reproduce with the cust DB dump and rake db:migrate. I'll update once I have more details.

Comment 5 Martin Bacovsky 2017-01-23 16:35:05 UTC
The customer's DB contains Dynflow tasks that are in running state. We may assume the DB dump was done while foreman-tasks was running and the tasks were not finished properly.

During db:seed the Dynflow that is being initiated is trying to recover and is trying to restart the tasks. When no executor is running the error is risen. It is not harmful because the task will be resumed on next executor start, but the error should not be displayed.

Changing the componnent to the Tasks plugin.

Comment 6 Martin Bacovsky 2017-01-23 22:01:02 UTC
Created redmine issue http://projects.theforeman.org/issues/18206 from this bug

Comment 7 Martin Bacovsky 2017-01-24 08:22:40 UTC
Minimal reproducer for this issue:
1/ Have Foreman/Satellite with at least one Dynflow task running.    (Actions::Candlepin::ListenOnCandlepinEvents in Katello/Satellite is a good candidate)
2/ kill dynflow executor while the task is running: pkill -9 -f dynflow_executor
3/ foreman-rake db:migrate

You should see errors in the output

E, [2017-01-24T03:00:53.032499 #9544] ERROR -- /client-dispatcher: No executor available (Dynflow::Error)

Comment 10 pm-sat@redhat.com 2017-01-24 17:20:05 UTC
Moving this bug to POST for triage into Satellite 6 since the upstream issue http://projects.theforeman.org/issues/18206 has been resolved.

Comment 11 Roman Plevka 2017-01-26 09:30:56 UTC
*** Bug 1413575 has been marked as a duplicate of this bug. ***

Comment 14 Roman Plevka 2017-06-01 11:48:34 UTC
VERIFIED
on 6.2.10-2

using the reproducer in Comment #7, I was able to reproduce the issue on 6.2.9 and can confirm, that it no longer occurs on 6.2.10-2

Comment 15 Bryan Kearney 2017-06-20 17:53:40 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:1553


Note You need to log in before you can comment on or make changes to this bug.