Bug 1243971 - RFE: Enable local hieradata overrides
Summary: RFE: Enable local hieradata overrides
Keywords:
Status: CLOSED DEFERRED
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: rhosp-director
Version: unspecified
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: z1
: Director
Assignee: Jay Dobies
QA Contact: yeylon@redhat.com
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-07-16 17:47 UTC by Steven Hardy
Modified: 2016-04-18 06:51 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-08-05 18:01:19 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Steven Hardy 2015-07-16 17:47:44 UTC
Description of problem:
Currently there's no easy way to pass in additional data to be consumed by puppet, for example to set a specific config value either globally or per service.

This is a common requirement, where operators need flexibility to provide site specific configuration data, ideally without hacking the templates or static hieradata files in the tripleo-heat-templates (packaged) location.

There are workarounds, e.g those described in https://review.gerrithub.io/#/c/240057/, but really we need a way to easily pass in hieradata overrides via template parameters.

Version-Release number of selected component (if applicable):
openstack-tripleo-heat-templates-0.8.6-41.el7ost.noarch

How reproducible:
Always

Steps to Reproduce:
1. Do a deployment, then attempt to modify, say nova::scheduler::filter::ram_allocation_ratio to influence the nova.conf

Actual results:
Doing this is possible but inconvenient

Expected results:
Convenient interface for hieradata overrides

Additional info:
I've got a series of patches in-progress upstream which enable this:

https://review.openstack.org/#/c/196046
https://review.openstack.org/#/c/196059/
https://review.openstack.org/#/c/202693/
https://review.openstack.org/#/c/202694/
https://review.openstack.org/#/c/202695/

These also require these two patches which I think aren't in the downstream builds atm:
https://review.openstack.org/#/c/197908/
https://review.openstack.org/#/c/188772/

The last patch will require an update to the deployed overcloud-full image

Comment 3 Marios Andreou 2015-07-29 12:23:15 UTC
I tried this today and it worked really well for me. On my deploy I mistakenly omitted the class (I used the ram_allocation_ratio example) in the hieradata I passed. I then used the same mechanism and did a stack update. The update completed and the changes were effected in the conf file(s).  Some notes in case useful for anyone else looking to test/verify:

SETUP
=====
All of the changes mentioned above are merged upstream but afaics, none downstream (on openstack-tripleo-heat-templates-0.8.6-45.el7ost.noarch none are applied). I cherry picked all the tht changes in the order above - mild merge conflict for the last one (/#/c/197908/, git mergetool). I then grabbed the tripleo-puppet-change like

sudo su; cd /usr/share/tripleo-puppet-elements/hiera/os-refresh-config/configure.d/
mv 40-hiera-datafiles 40-hiera-datafiles.OLD
curl "https://review.openstack.org/gitweb?p=openstack/tripleo-puppet-elements.git;a=blob_plain;f=elements/hiera/os-refresh-config/configure.d/40-hiera-datafiles;h=9be07a92fc7bc5ef7dded8bff1f1415ec64f2785;hb=5d2d14f69085b3a7dac8474aa00b197e3521e4d0" > 40-hiera-datafiles
exit

Rebuilt the overcloud-full image and reloaded the tuskar roles using the procedure at https://github.com/rdo-management/instack-undercloud/blob/c072ac1e16f3f75dc229c7cffae8acab0a52c1c9/doc/source/advanced_deployment/reload_roles_and_plan.rst

TEST
====
Set some ExtraConfig for Compute and Controller:

openstack management plan set 7cfaa92c-26c3-49a6-85dc-76d1317143bf -P 'Controller-1::ControllerExtraConfig={"neutron::dhcp_agents_per_network": 2}'

openstack management plan set 7cfaa92c-26c3-49a6-85dc-76d1317143bf  -P 'Compute-1::NovaComputeExtraConfig={"nova::scheduler::filter::ram_allocation_ratio": 1.8}'

Note here the second example is wrong, since I don't also pass the required class (we don't define/include nova::scheduler::filer otherwise). This is important because I was successfully able to update this after the create_complete of the deploy using this same arbitrary hiera data passing mechanism.

After CREATE_COMPLETE:

[stack@instack ~]$ node_ips=`nova list | grep Running | awk '{print $12}' | tr "ctlplane=" "\n"`
[stack@instack ~]$ for ip in ${node_ips[@]}; do ssh heat-admin@$ip "hostname; sudo grep -rni 'ram_allocation\|dhcp_agents' /etc/*"; done

on the controllers:
/etc/neutron/neutron.conf:243:dhcp_agents_per_network = 2

on computes:
overcloud-compute-0.localdomain
/etc/nova/nova.conf:1597:#ram_allocation_ratio=1.5
/etc/puppet/hieradata/compute_extraconfig.yaml:2:nova::scheduler::filter::ram_allocation_ratio: 1.8

Note ^^^ the value is passed in the hierdata correctly, but not effected in nova.conf since I didn't include the class. 

Update using the same mechanism:

openstack management plan set 7cfaa92c-26c3-49a6-85dc-76d1317143bf  -P 'Compute-1::NovaComputeExtraConfig={"nova::scheduler::filter::ram_allocation_ratio": 1.8, "compute_classes": ["::nova::scheduler::filter"] }'

openstack overcloud deploy --plan overcloud

after UPDATE_COMPLETE, on the computes:

overcloud-compute-1.localdomain
/etc/nova/nova.conf:1604:ram_allocation_ratio=1.8
/etc/puppet/hieradata/compute_extraconfig.yaml:5:nova::scheduler::filter::ram_allocation_ratio: 1.8

Comment 4 chris alfonso 2015-08-05 18:01:19 UTC
We are tracking this outside of bugzilla with other features, so closing this RFE.


Note You need to log in before you can comment on or make changes to this bug.