Bug 1291307 - 40-hiera-datafiles is slow
40-hiera-datafiles is slow
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-puppet-elements (Show other bugs)
8.0 (Liberty)
Unspecified Unspecified
unspecified Severity unspecified
: ---
: 10.0 (Newton)
Assigned To: RHOS Maint
Shai Revivo
Depends On:
  Show dependency treegraph
Reported: 2015-12-14 09:34 EST by Attila Fazekas
Modified: 2016-12-05 05:05 EST (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2016-10-09 23:22:31 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

External Trackers
Tracker ID Priority Status Summary Last Updated
Launchpad 1596373 None None None 2016-12-05 05:05 EST

  None (edit)
Description Attila Fazekas 2015-12-14 09:34:01 EST
Description of problem:
/usr/libexec/os-refresh-config/configure.d/40-hiera-datafiles executed at every configuration phase. For example when you just change /usr/share/openstack-tripleo-heat-templates/environments/config-debug.yaml ConfigDebug parameter it is executed 4 times. At the first deployment, it is expected to run even more times.

In my small deployment it takes 22 sec to execute,
the execution includes 1081 * jq command and 40 * os-apply-config.

Both external command is for json(/yaml) manipulation,
the os-apply-config tooks ~0.5 sec, 0.25 sec just to load the required python

The jq is a relative fast C program 0.001..0.01 sec, but it is executed 1000 times, and while the process is under strace it's costs even more.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. 3 controller, one compute, isolated network default deployment
2. rerun the overcloud deploy and include (or change) /usr/share/openstack-tripleo-heat-templates/environments/config-debug.yaml 

For ex.:
$ source /home/stack/stackrc; time openstack overcloud deploy --log-file overcloud_deployment_48.log --templates --libvirt-type=qemu --neutron-network-type vxlan --neutron-tunnel-types vxlan --ntp-server --control-scale 3 --compute-scale 1 --ceph-storage-scale 0 --block-storage-scale 0 --swift-storage-scale 0 --control-flavor baremetal --compute-flavor baremetal --ceph-storage-flavor baremetal --block-storage-flavor baremetal --swift-storage-flavor baremetal --timeout=240 -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/net-single-nic-with-vlans.yaml -e ~/network-environment.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/puppet-pacemaker.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/config-debug.yaml

3. wait a lot
4. login to one controller as root
5. time /usr/libexec/os-refresh-config/configure.d/40-hiera-datafiles # -> 22 sec
6. time strace -Ff -e execve,clone,fork -v -ttt -s 4096  -o /tmp/execve-hiera  /usr/libexec/os-refresh-config/configure.d/40-hiera-datafiles # -> 44 sec

Actual results:
40-hiera-datafiles executed in 22 sec without tracing, 44 sec with trace

Expected results:
40-hiera-datafiles executes in 2.2 sec.

Additional info:
40-hiera-datafiles cloud be written in language which does not needs multiple external python process to complete. For example it can be written in python with anyjson, ujson usage.
Alternatively the external command os-apply-config must be significantly faster,
for ex. written in C or go.

Because at one execution, the external commands frequently reads the same file content multiple times, makes more sense to implement 40-hiera-datafiles in way which avoids unnecessary rereads.
Comment 2 Mike Burns 2016-04-07 17:00:12 EDT
This bug did not make the OSP 8.0 release.  It is being deferred to OSP 10.
Comment 4 Jaromir Coufal 2016-10-09 23:22:31 EDT
Obsolete, please re-open if re-appears.

Note You need to log in before you can comment on or make changes to this bug.