Description of problem: Tried to deploy from the GUI: 3 controllers + 1 compute + 1 database role, with no network isolation or any additional features. The deployment hanged in the end so I ran "openstack overcloud status --plan overcloud" but it hanged too. So I ran the workflow manually: curl -g -i -X POST https://192.168.24.2:13989/v2/executions -H "User-Agent: openstacksdk/0.17.2 keystoneauth1/3.10.0 python-requests/2.19.1 CPython/2.7.5" -H "X-Auth-Token: $TOKEN" -H "content-type: application/json" -d '{"input": "{\"plan\": \"overcloud\"}", "workflow_name": "tripleo.deployment.v1.get_deployment_status", "description": ""}' Then I checked the execution status: openstack workflow execution show 18fca**** and saw this traceback: Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/mistral/engine/task_handler.py", line 110, in _on_action_complete task.on_action_complete(action_ex) File "/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 159, in wrapper result = f(*args, **kwargs) File "/usr/lib/python2.7/site-packages/mistral/engine/tasks.py", line 381, in on_action_complete self.complete(state, state_info) File "/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 159, in wrapper result = f(*args, **kwargs) File "/usr/lib/python2.7/site-packages/mistral/engine/tasks.py", line 246, in complete cmds = wf_ctrl.continue_workflow(task_ex=self.task_ex) File "/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 159, in wrapper result = f(*args, **kwargs) File "/usr/lib/python2.7/site-packages/mistral/workflow/base.py", line 134, in continue_workflow return self._find_next_commands(task_ex) File "/usr/lib/python2.7/site-packages/mistral/workflow/direct_workflow.py", line 93, in _find_next_commands cmds.extend(self._find_next_commands_for_task(t_ex)) File "/usr/lib/python2.7/site-packages/mistral/workflow/direct_workflow.py", line 120, in _find_next_commands_for_task for t_n, params, event_name in self._find_next_tasks(task_ex, ctx=ctx): File "/usr/lib/python2.7/site-packages/mistral/workflow/direct_workflow.py", line 284, in _find_next_tasks 'on-complete' File "/usr/lib/python2.7/site-packages/mistral/workflow/direct_workflow.py", line 265, in process_clause task_tuples = self._find_next_tasks_for_clause(clause, ctx_view) File "/usr/lib/python2.7/site-packages/mistral/workflow/direct_workflow.py", line 308, in _find_next_tasks_for_clause if not condition or expr.evaluate(condition, ctx) File "/usr/lib/python2.7/site-packages/mistral/expressions/__init__.py", line 71, in evaluate return evaluator.evaluate(expression, context) File "/usr/lib/python2.7/site-packages/mistral/expressions/yaql_expression.py", line 159, in evaluate cls).evaluate(trim_expr, data_context) File "/usr/lib/python2.7/site-packages/mistral/expressions/yaql_expression.py", line 113, in evaluate ", data=%s]" % (expression, str(e), data_context) YaqlEvaluationException: Can not evaluate YAQL expression [expression=$.status_update = null and $.deployment_status != null, error=u'status_update', data={}] Version-Release number of selected component (if applicable): openstack-tripleo-common-9.4.1-0.20181012010884.el7ost.noarch openstack-mistral-executor:2018-12-11.1 openstack-mistral-engine:2018-12-11.1 openstack-mistral-api:2018-12-11.1 How reproducible: 100% Steps to Reproduce: 1. Start a deployment: 3 controllers + 1 compute + 1 database role, with no network isolation or any additional features. 2. Run "openstack overcloud status --plan overcloud". It should hang. 3. Run the workflow manually: openstack workflow execution create tripleo.deployment.v1.get_deployment_status '{"plan": "overcloud"}' 4. Check if the execution finished: openstack workflow execution show <<ID>> Actual results: YaqlEvaluationException: Can not evaluate YAQL expression [expression=$.status_update = null and $.deployment_status != null, error=u'status_update', data={}] Expected results: The workflow should return the deployment status
DeploymentStatusAction fails with: { "result": "The action raised an exception [action_ex_id=1ae1dc00-31d3-4ebf-9ad3-40cf32d9e30d, action_cls='<class 'mistral.actions.action_factory.DeploymentStatusAction'>', attributes='{}', params='{u'plan': u'overcloud'}']\n 'deployment_status'" } mistral/executor.log: ges/mistralclient/api/httpclient.py:56 2018-12-14 09:10:13.215 1 WARNING mistral.executors.default_executor [req-2ff735ed-daef-4e9d-b278-ea88217b3584 159bf210c99a4740b948016daa51c83b ef1846f91d5742f4821d440252ed3ac5 - default default] The action raised an exception [action_ex_id=None, action_cls='<class 'mistral.actions.action_factory.DeploymentStatusAction'>', attributes='{}', params='{u'plan': u'overcloud'}'] 'deployment_status': KeyError: 'deployment_status' 2018-12-14 09:10:13.215 1 ERROR mistral.executors.default_executor Traceback (most recent call last): 2018-12-14 09:10:13.215 1 ERROR mistral.executors.default_executor File "/usr/lib/python2.7/site-packages/mistral/executors/default_executor.py", line 114, in run_action 2018-12-14 09:10:13.215 1 ERROR mistral.executors.default_executor result = action.run(action_ctx) 2018-12-14 09:10:13.215 1 ERROR mistral.executors.default_executor File "/usr/lib/python2.7/site-packages/tripleo_common/actions/deployment.py", line 379, in run 2018-12-14 09:10:13.215 1 ERROR mistral.executors.default_executor ansible_status = json.loads(cd_exec.output)['deployment_status'] 2018-12-14 09:10:13.215 1 ERROR mistral.executors.default_executor KeyError: 'deployment_status'
(undercloud) [stack@undercloud-0 tmp]$ openstack workflow execution list --filter workflow_name='tripleo.deployment.v1.config_download_deploy' +--------------------------------------+--------------------------------------+----------------------------------------------+--------------------+------------------------+--------------------------------------+--------------------------------------+---------+------------+---------------------+---------------------+ | ID | Workflow ID | Workflow name | Workflow namespace | Description | Task Execution ID | Root Execution ID | State | State info | Created at | Updated at | +--------------------------------------+--------------------------------------+----------------------------------------------+--------------------+------------------------+--------------------------------------+--------------------------------------+---------+------------+---------------------+---------------------+ | 78dfdcc1-c90d-4206-96ab-fd1565d54c67 | b46263e8-5c34-40e8-b627-0e498f14b1f3 | tripleo.deployment.v1.config_download_deploy | | sub-workflow execution | fb4f5a4c-52c7-4aa4-9cde-94c10ab68389 | 5a65ce01-2f6b-4100-895a-1831458783ff | SUCCESS | None | 2018-12-14 06:31:04 | 2018-12-14 07:03:33 | | 41c2ff74-be07-425b-a5dc-c09f7fdcf84a | b46263e8-5c34-40e8-b627-0e498f14b1f3 | tripleo.deployment.v1.config_download_deploy | | sub-workflow execution | 0ec1d37d-d4bb-48dd-a242-433cab4ecec2 | 3e59f126-506c-42fd-8810-7500289f095b | RUNNING | None | 2018-12-14 09:05:55 | 2018-12-14 09:05:55 | +--------------------------------------+--------------------------------------+----------------------------------------------+--------------------+------------------------+--------------------------------------+--------------------------------------+---------+------------+---------------------+---------------------+ (undercloud) [stack@undercloud-0 tmp]$ openstack workflow execution output show 41c2ff74-be07-425b-a5dc-c09f7fdcf84a {} In case when the config-download workflow is still running, the execution output is empty^ so a solution is probably ansible_status = json.loads(cd_exec.output).get('deployment_status', 'RUNNING')
Proposed upstream master fix: https://review.openstack.org/625275 Fix DeploymentStatusAction KeyError
returned success and a response for every command
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2019:0045