Description of problem: Following the instructions in https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.1/html/advanced_overcloud_customization/chap-configuration_hooks#sect-Customizing_Hieradata_for_Individual_Nodes leads to a json decoding error. Steps to Reproduce: 1. Create a template with the following: resource_registry: OS::TripleO::ComputeExtraConfigPre: /usr/share/openstack-tripleo-heat-templates/puppet/extraconfig/pre_deploy/per_node.yaml parameter_defaults: NodeDataLookup: {"C06C60C2-EC19-4237-B7EB-507F9E303EFD": {"nova::compute::cpu_dedicated_set": [ "1-4" ]}} 2. Run a stack update Actual results: (heat-config) [INFO] b\\'\\'\\n[2021-02-03 23:55:58,781] (heat-config) [DEBUG] b\\'Traceback (most recent call last):\\\\n File \"<string>\", line 5, in <module>\\\\n File \"/usr/ lib64/python3.6/json/__init__.py\", line 354, in loads\\\\n return _default_decoder.decode(s)\\\\n File \"/usr/lib64/python3.6/json/decoder.py\", line 339, in decode\\\\n obj, end = self.raw_decode(s, idx=_w(s, 0).end())\\\\n File \"/usr/lib64/python3.6/json/decoder.py\", line 355, in raw_decode\\\\n obj, end = self.scan_once(s, idx)\\\\njson.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)\\\\nTraceback (most r ecent call last):\\\\n File \"<string>\", line 5, in <module>\\\\n File \"/usr/lib64/python3.6/json/__init__.py\", line 354, in loads\\\\n return _default_decoder.decode(s)\\\\n File \"/usr/lib64/python3.6/json/decoder.py\", line 339 , in decode\\\\n obj, end = self.raw_decode(s, idx=_w(s, 0).end())\\\\n File \"/usr/lib64/python3.6/json/decoder.py\", line 355, in raw_decode\\\\n obj, end = self.scan_once(s, idx)\\\\njson.decoder.JSONDecodeError: Expecting proper ty name enclosed in double quotes: line 1 column 2 (char 1)\\\\n\\'\\n Expected results: A file located at /etc/puppet/hieradata/<uuid>.json is created with the following content: {"nova::compute::cpu_dedicated_set": [ "1-4" ]}} Additional info: It seems like the input to the script is coming in as a map/dictionary (using single quotes) rather than a proper json string, which causes json.loads to fail with the aforementioned decoding error. inputs: - name: node_lookup description: type: String value: |- {'C06C60C2-EC19-4237-B7EB-507F9E303EFD': {'nova::compute::cpu_dedicated_set': ['1-4']}} Workaround: Replacing json.loads(input) with ast.literal_eval(input) looks like it will properly create a dictionary object in python whether it's a json string or the single-quoted mapping style that heat is using. Here's a patch demonstrating the changes: --- a/usr/share/openstack-tripleo-heat-templates/puppet/extraconfig/pre_deploy/per_node.yaml 2020-08-21 20:32:02.000000000 -0400 +++ b/usr/share/openstack-tripleo-heat-templates/puppet/extraconfig/pre_deploy/per_node.yaml 2021-02-03 19:28:20.090743051 -0500 @@ -47,16 +47,18 @@ echo $node_lookup | $(get_python) -c " import json import sys + import ast input = sys.stdin.readline() or '{}' - cnt = json.loads(input) + cnt = ast.literal_eval(input) print(json.dumps(cnt.get('${node_id}', {}))) " > /etc/puppet/hieradata/${node_id}.json # handle upper case node id LP#1816652 echo $node_lookup | $(get_python) -c " import json import sys + import ast input = sys.stdin.readline() or '{}' - cnt = json.loads(input) + cnt = ast.literal_eval(input) print(json.dumps(cnt.get('${node_id_upper}', {}))) " > /etc/puppet/hieradata/${node_id_upper}.json
Kellen, Thanks for flagging this and to providing the patch. [1] Reproduced this I've submitted this patch [a] upstream and I will backport it to train to address this bug. DVD [a] https://review.opendev.org/774323 [1] ~~~ [2021-02-05 21:48:08,975] (heat-config) [INFO] deploy_action=CREATE\ [2021-02-05 21:48:08,975] (heat-config) [INFO] deploy_stack_id=overcloud-Compute-ylhk4xulanbt-0-iq4yrmavu2fr-ComputeExtraConfigPre-h3sri74r3k2n-NodeSpecificDeployment-ugiv4nr3g27o/697ed555-b51c-425d-ba0b-32ef1e149d87\ [2021-02-05 21:48:08,976] (heat-config) [INFO] deploy_resource_name=TripleOSoftwareDeployment\ [2021-02-05 21:48:08,976] (heat-config) [INFO] deploy_signal_transport=NO_SIGNAL\ [2021-02-05 21:48:08,976] (heat-config) [DEBUG] Running /var/lib/heat-config/heat-config-script/67fa688f-c518-49af-9835-8d8640948c05\ [2021-02-05 21:48:09,147] (heat-config) [INFO] b\\'\\'\ [2021-02-05 21:48:09,147] (heat-config) [DEBUG] b\\'Traceback (most recent call last):\\\ File \"<string>\", line 5, in <module>\\\ File \"/usr/lib64/python3.6/json/__init__.py\", line 354, in loads\\\ return _default_decoder.decode(s)\\\ File \"/usr/lib64/python3.6/json/decoder.py\", line 339, in decode\\\ obj, end = self.raw_decode(s, idx=_w(s, 0).end())\\\ File \"/usr/lib64/python3.6/json/decoder.py\", line 355, in raw_decode\\\ obj, end = self.scan_once(s, idx)\\\ json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)\\\ Traceback (most recent call last):\\\ File \"<string>\", line 5, in <module>\\\ File \"/usr/lib64/python3.6/json/__init__.py\", line 354, in loads\\\ return _default_decoder.decode(s)\\\ File \"/usr/lib64/python3.6/json/decoder.py\", line 339, in decode\\\ obj, end = self.raw_decode(s, idx=_w(s, 0).end())\\\ File \"/usr/lib64/python3.6/json/decoder.py\", line 355, in raw_decode\\\ obj, end = self.scan_once(s, idx)\\\ json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)\\\ \\'\ ~~~
*** Bug 1941097 has been marked as a duplicate of this bug. ***
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Red Hat OpenStack Platform 16.1.6 bug fix and enhancement advisory), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2021:2097