Description of problem: openstack stack failures list overcloud --long overcloud.AllNodesDeploySteps.WorkflowTasks_Step2_Execution: resource_type: OS::TripleO::WorkflowSteps physical_resource_id: 52bb4fde-9bf6-49b4-b64f-3bae8adac2a7 status: CREATE_FAILED status_reason: | resources.WorkflowTasks_Step2_Execution: Failure caused by error in tasks: ceph_base_ansible_workflow ceph_base_ansible_workflow [task_ex_id=2ba99c86-1d95-4ce0-896a-7b152c091f5b] -> Failed to handle action completion [error=Can not evaluate YAQL expression [expression=json_parse($.node_data_lookup), error=No JSON object could be decoded, data={}], wf=tripleo.storage.v1.ceph-install, task=parse_node_data_lookup, action=std.noop]: Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/mistral/engine/task_handler.py", line 110, in _on_action_complete task.on_action_complete(action_ex) File "/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 158, in wrapper result = f(*args, **kwargs) File "/usr/lib/python2.7/site-packages/mistral/engine/tasks.py", line 324, in on_action_complete self.complete(state, state_info) File "/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 158, in wrapper result = f(*args, **kwargs) File "/usr/lib/python2.7/site-packages/mistral/engine/tasks.py", line 191, in complete data_flow.publish_variables(self.task_ex, self.task_spec) File "/usr/lib/python2.7/site-packages/mistral/workflow/data_flow.py", line 215, in publish_variables task_ex.published = expr.evaluate_recursively(branch_vars, expr_ctx) File "/usr/lib/python2.7/site-packages/mistral/expressions/__init__.py", line 100, in evaluate_recursively data[key] = _evaluate_item(data[key], context) File "/usr/lib/python2.7/site-packages/mistral/expressions/__init__.py", line 79, in _evaluate_item return evaluate(item, context) File "/usr/lib/python2.7/site-packages/mistral/expressions/__init__.py", line 71, in evaluate return evaluator.evaluate(expression, context) File "/usr/lib/python2.7/site-packages/mistral/expressions/yaql_expression.py", line 119, in evaluate cls).evaluate(trim_expr, data_context) File "/usr/lib/python2.7/site-packages/mistral/expressions/yaql_expression.py", line 73, in evaluate ", data=%s]" % (expression, str(e), data_context) YaqlEvaluationException: Can not evaluate YAQL expression [expression=json_parse($.node_data_lookup), error=No JSON object could be decoded, data={}] [wf_ex_id=c68bb3e0-6ae4-42d0-8651-0a3a93e31fb4, idx=0]: Failed to handle action completion [error=Can not evaluate YAQL expression [expression=json_parse($.node_data_lookup), error=No JSON object could be decoded, data={}], wf=tripleo.storage.v1.ceph-install, task=parse_node_data_lookup, action=std.noop]: Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/mistral/engine/task_handler.py", line 110, in _on_action_complete task.on_action_complete(action_ex) File "/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 158, in wrapper result = f(*args, **kwargs) File "/usr/lib/python2.7/site-packages/mistral/engine/tasks.py", line 324, in on_action_complete self.complete(state, state_info) File "/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 158, in wrapper result = f(*args, **kwargs) File "/usr/lib/python2.7/site-packages/mistral/engine/tasks.py", line 191, in complete data_flow.publish_variables(self.task_ex, self.task_spec) File "/usr/lib/python2.7/site-packages/mistral/workflow/data_flow.py", line 215, in publish_variables task_ex.published = expr.evaluate_recursively(branch_vars, expr_ctx) File "/usr/lib/python2.7/site-packages/mistral/expressions/__init__.py", line 100, in evaluate_recursively data[key] = _evaluate_item(data[key], context) File "/usr/lib/python2.7/site-packages/mistral/expressions/__init__.py", line 79, in _evaluate_item return evaluate(item, context) File "/usr/lib/python2.7/site-packages/mistral/expressions/__init__.py", line 71, in evaluate return evaluator.evaluate(expression, context) File "/usr/lib/python2.7/site-packages/mistral/expressions/yaql_expression.py", line 119, in evaluate cls).evaluate(trim_expr, data_context) File "/usr/lib/python2.7/site-packages/mistral/expressions/yaql_expression.py", line 73, in evaluate ", data=%s]" % (expression, str(e), data_context) YaqlEvaluationException: Can not evaluate YAQL expression [expression=json_parse($.node_data_lookup), error=No JSON object could be decoded, data={}] The error looks similar to bug 1570050 in RHOSP13 Version-Release number of selected component (if applicable): openstack-tripleo-heat-templates-8.3.1-18.el7ost.noarch
I see you are using node specific overrides as described in the following document: https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/13/html/deploying_an_overcloud_with_containerized_red_hat_ceph/configuring_ceph_storage_cluster_settings#map_disk_layout_non-homogen_ceph As per the same document the JSON which is embedded in your YAML needs to be valid json and you can verify this with jq. In this scenario the ceph1-config.yaml is not valid JSON and that's why you're having this issue. Here's how I determined this as documented above. 1. remove the yaml from the file [fultonj@skagra bz1730921]$ diff -u ceph1-config.yaml.bak ceph1-config.yaml --- ceph1-config.yaml.bak 2019-08-06 09:25:49.643880008 -0400 +++ ceph1-config.yaml 2019-08-06 09:26:24.044136893 -0400 @@ -1,16 +1,3 @@ -parameter_defaults: - CephConfigOverrides: - journal_size: 102400 - max_open_files: 131072 - - CephAnsibleDisksConfig: - osd_scenario: non-collocated - - CephPoolDefaultPgNum: 64 - ManilaCephFSDataPoolPGNum: 64 - ManilaCephFSMetadataPoolPGNum: 64 - - NodeDataLookup: | { "39373638-3935-584d-5139-31383030574a": { "devices": [ [fultonj@skagra bz1730921]$ 2. use jq to parse the JSON [fultonj@skagra bz1730921]$ cat ceph1-config.yaml | jq parse error: Invalid numeric literal at line 24, column 32 [fultonj@skagra bz1730921]$ 3. Change the " on line 24, column 32 to a ' and try again [fultonj@skagra bz1730921]$ cat ceph1-config.yaml | jq { "39373638-3935-584d-5139-31383030574a": { "devices": [ "/dev/disk/by-path/pci-0000:5c:00.0-sas-0x31402ec012d557c2-lun-0", "/dev/disk/by-path/pci-0000:5c:00.0-sas-0x31402ec012d557c3-lun-0" ], "dedicated_devices": [ "/dev/disk/by-path/pci-0000:5c:00.0-sas-0x31402ec012d557c4-lun-0", "/dev/disk/by-path/pci-0000:5c:00.0-sas-0x31402ec012d557c4-lun-0" ] }, "39373638-3935-584d-5139-313830305748": { "devices": [ "/dev/disk/by-path/pci-0000:5c:00.0-sas-0x31402ec012d65b82-lun-0", "/dev/disk/by-path/pci-0000:5c:00.0-sas-0x31402ec012d65b83-lun-0" ], "dedicated_devices": [ "/dev/disk/by-path/pci-0000:5c:00.0-sas-0x31402ec012d65b84-lun-0", "/dev/disk/by-path/pci-0000:5c:00.0-sas-0x31402ec012d65b84-lun-0" ] }, "39373638-3935-584d-5139-313830305744": { "devices": [ "/dev/disk/by-path/pci-0000:5c:00.0-sas-0x31402ec012d654b2-lun-0", "/dev/disk/by-path/pci-0000:5c:00.0-sas-0x31402ec012d654b3-lun-0" ], "dedicated_devices": [ "/dev/disk/by-path/pci-0000:5c:00.0-sas-0x31402ec012d654b4-lun-0", "/dev/disk/by-path/pci-0000:5c:00.0-sas-0x31402ec012d654b4-lun-0" ] } } [fultonj@skagra bz1730921]$ 4. Add the YAML which was removed in step1 back to the file You should now be able to redeploy using the new ceph1-config.yaml
Because this was a syntax issue for the program input (" vs ') I'm closing this as not a bug.
Closed NOTABUG, nothing to test/automate per close loop process.