Description of problem: /usr/libexec/os-refresh-config/configure.d/40-hiera-datafiles executed at every configuration phase. For example when you just change /usr/share/openstack-tripleo-heat-templates/environments/config-debug.yaml ConfigDebug parameter it is executed 4 times. At the first deployment, it is expected to run even more times. In my small deployment it takes 22 sec to execute, the execution includes 1081 * jq command and 40 * os-apply-config. Both external command is for json(/yaml) manipulation, the os-apply-config tooks ~0.5 sec, 0.25 sec just to load the required python modules. The jq is a relative fast C program 0.001..0.01 sec, but it is executed 1000 times, and while the process is under strace it's costs even more. Version-Release number of selected component (if applicable): openstack-tripleo-puppet-elements-0.0.2-1.el7ost.noarch How reproducible: always Steps to Reproduce: 1. 3 controller, one compute, isolated network default deployment 2. rerun the overcloud deploy and include (or change) /usr/share/openstack-tripleo-heat-templates/environments/config-debug.yaml For ex.: $ source /home/stack/stackrc; time openstack overcloud deploy --log-file overcloud_deployment_48.log --templates --libvirt-type=qemu --neutron-network-type vxlan --neutron-tunnel-types vxlan --ntp-server 42.42.42.42 --control-scale 3 --compute-scale 1 --ceph-storage-scale 0 --block-storage-scale 0 --swift-storage-scale 0 --control-flavor baremetal --compute-flavor baremetal --ceph-storage-flavor baremetal --block-storage-flavor baremetal --swift-storage-flavor baremetal --timeout=240 -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/net-single-nic-with-vlans.yaml -e ~/network-environment.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/puppet-pacemaker.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/config-debug.yaml 3. wait a lot 4. login to one controller as root 5. time /usr/libexec/os-refresh-config/configure.d/40-hiera-datafiles # -> 22 sec 6. time strace -Ff -e execve,clone,fork -v -ttt -s 4096 -o /tmp/execve-hiera /usr/libexec/os-refresh-config/configure.d/40-hiera-datafiles # -> 44 sec Actual results: 40-hiera-datafiles executed in 22 sec without tracing, 44 sec with trace Expected results: 40-hiera-datafiles executes in 2.2 sec. Additional info: 40-hiera-datafiles cloud be written in language which does not needs multiple external python process to complete. For example it can be written in python with anyjson, ujson usage. Alternatively the external command os-apply-config must be significantly faster, for ex. written in C or go. Because at one execution, the external commands frequently reads the same file content multiple times, makes more sense to implement 40-hiera-datafiles in way which avoids unnecessary rereads.
This bug did not make the OSP 8.0 release. It is being deferred to OSP 10.
Obsolete, please re-open if re-appears.