Description of problem: Per node hiera data doesn't get applied in osp 13 as defined: https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/13/html-single/advanced_overcloud_customization/ Version-Release number of selected component (if applicable): OSP 13 current. Not sure on the rpm # rpm -qf /var/lib/docker-puppet/docker-puppet.py file /var/lib/docker-puppet/docker-puppet.py is not owned by any package How reproducible: 100% Steps to Reproduce: 1. Deploy with the following per node config resource_registry: OS::TripleO::ComputeExtraConfigPre: /usr/share/openstack-tripleo-heat-templates/puppet/extraconfig/pre_deploy/per_node.yaml parameter_defaults: NodeDataLookup: | {"A8FE66FF-3C42-42FB-804B-7612BA92AF41": {"nova::debug": "True"}} 2. Verify that debug does not get set in /var/lib/config-data/puppet-generated/nova_libvirt/etc/nova/nova.conf on the node with uuid A8FE66FF-3C42-42FB-804B-7612BA92AF41 Actual results: no effect Expected results: work like pre-containerized versions Additional info: It looks /var/lib/docker-puppet/docker-puppet.py replaces the facter uuid with the word docker used for other configuration. From: /var/lib/docker-puppet/docker-puppet.py echo "{\\"step\\": $STEP}" > /etc/puppet/hieradata/docker.json FACTER_hostname=$HOSTNAME FACTER_uuid=docker /usr/bin/puppet apply --summarize \ Also it looks like `facter uuid` doesnt work inside the container to apply per node hiera even if docker-puppet.py was not overriding.
The hiera data file is created correctly in /etc/puppet/hieradata/. The issue is it is unused when configuring the containers. /var/lib/docker-puppet/docker-puppet.py re-proposes the uuid fact: From /var/lib/docker-puppet/docker-puppet.py: sh_script = '/var/lib/docker-puppet/docker-puppet.sh' with open(sh_script, 'w') as script_file: os.chmod(script_file.name, 0755) script_file.write("""#!/bin/bash set -ex mkdir -p /etc/puppet cp -a /tmp/puppet-etc/* /etc/puppet rm -Rf /etc/puppet/ssl # not in use and causes permission errors echo "{\\"step\\": $STEP}" > /etc/puppet/hieradata/docker.json TAGS="" if [ -n "$PUPPET_TAGS" ]; then TAGS="--tags \"$PUPPET_TAGS\"" fi # Create a reference timestamp to easily find all files touched by # puppet. The sync ensures we get all the files we want due to # different timestamp. origin_of_time=/var/lib/config-data/${NAME}.origin_of_time touch $origin_of_time sync set +e FACTER_hostname=$HOSTNAME FACTER_uuid=docker /usr/bin/puppet apply --summarize \ --detailed-exitcodes --color=false --logdest syslog --logdest console --modulepath=/etc/puppet/modules:/usr/share/openstack-puppet/modules $TAGS /etc/config.pp
Ah you need https://review.openstack.org/#/c/607647/. Let me check if it's backported downstream yet.
I used to the following change as a test to work around the issue. Not sure if this is viable but it correctly applied per node settings. $ git diff . diff --git a/openstack-tripleo-heat-templates/puppet/extraconfig/pre_deploy/per_node.yaml b/openstack-tripleo-heat-templates/puppet/extraconfig/pre_deploy/per_no index c39b485..39f8cd6 100644 --- a/openstack-tripleo-heat-templates/puppet/extraconfig/pre_deploy/per_node.yaml +++ b/openstack-tripleo-heat-templates/puppet/extraconfig/pre_deploy/per_node.yaml @@ -40,7 +40,7 @@ resources: input = sys.stdin.readline() or '{}' cnt = json.loads(input) print json.dumps(cnt.get('${node_id}', {})) - " > /etc/puppet/hieradata/${node_id}.json + " > /etc/puppet/hieradata/per_node.json NodeSpecificDeployment: type: OS::Heat::SoftwareDeployment diff --git a/openstack-tripleo-heat-templates/puppet/role.role.j2.yaml b/openstack-tripleo-heat-templates/puppet/role.role.j2.yaml index e9b208b..eba9d41 100644 --- a/openstack-tripleo-heat-templates/puppet/role.role.j2.yaml +++ b/openstack-tripleo-heat-templates/puppet/role.role.j2.yaml @@ -532,6 +532,7 @@ resources: config: hierarchy: - '"%{::uuid}"' + - per_node - heat_config_%{::deploy_config_name} - config_step - {{role.name.lower()}}_extraconfig
Duplicate of : https://bugzilla.redhat.com/show_bug.cgi?id=1635334 Fixed-In: puppet-tripleo-8.3.4-13.el7ost openstack-tripleo-heat-templates-8.0.4-36.el7ost I'll mark it duplicate once confirmed.
Thanks Rabi; thought I was working with the current release. I will confirm; sorry for the noise.
Looks like there is still an issue with the current version. rpm -q openstack-tripleo-heat-templates openstack-tripleo-heat-templates-8.0.7-4.el7ost.noarch As I saw yesterday it appears the uuid fact does not resolve inside a container so the "%{::uuid}" hierarchy does not work even now that it's not being replaced. To show the issue using Steve's awesome blog on debugging containerized deployments (http://hardysteven.blogspot.com/2018/06/tripleo-containerized-deployments.html). On a compute node we see that debug did not get set to true as defined in the per node config. [root@overcloud-compute-0 ~]# facter uuid a8fe66ff-3c42-42fb-804b-7612ba92af41 [root@overcloud-compute-0 ~]# cat /etc/puppet/hieradata/a8fe66ff-3c42-42fb-804b-7612ba92af41.json {"nova::debug": "True"} [root@overcloud-compute-0 ~]# crudini --get /var/lib/config-data/puppet-generated/nova_libvirt/etc/nova/nova.conf DEFAULT debug False We make a small change to docker-puppet.py to show ::uuid does not work inside the container. [root@overcloud-compute-0 docker-puppet]# git diff docker-puppet.py diff --git a/docker-puppet.py b/docker-puppet.py index 8f0736c..8157f3e 100644 --- a/docker-puppet.py +++ b/docker-puppet.py @@ -225,6 +225,7 @@ with open(sh_script, 'w') as script_file: set +e # $::deployment_type in puppet-tripleo export FACTER_deployment_type=containers + echo "uuid set to: $(facter uuid)" FACTER_hostname=$HOSTNAME /usr/bin/puppet apply --summarize \ --detailed-exitcodes --color=false --logdest syslog --logdest console --modulepath=/etc/puppet/modules:/usr/share/openstack-puppet/modules $TAGS /etc/config.pp rc=$? We re-apply the containerized puppet and show that uuid is not available to facter: [root@overcloud-compute-0 ~]# jq '[.[]|select(.config_volume | contains("nova"))]' /var/lib/docker-puppet/docker-puppet.json | tee /tmp/nova.json [ { "config_image": "172.16.6.1:8787/rhosp13/openstack-nova-compute:13.0-70", "step_config": "# TODO(emilien): figure how to deal with libvirt profile.\n# We'll probably treat it like we do with Neutron plugins.\n# Until then, just include it in the default nova-compute role.\ninclude tripleo::profile::base::nova::compute::libvirt\n\ninclude ::tripleo::profile::base::database::mysql::client", "config_volume": "nova_libvirt", "puppet_tags": "nova_config,nova_paste_api_ini" }, { "config_image": "172.16.6.1:8787/rhosp13/openstack-nova-compute:13.0-70", "step_config": "include tripleo::profile::base::nova::libvirt\n\ninclude ::tripleo::profile::base::database::mysql::client", "config_volume": "nova_libvirt", "puppet_tags": "libvirtd_config,nova_config,file,libvirt_tls_password" }, { "step_config": "include ::tripleo::profile::base::sshd\ninclude tripleo::profile::base::nova::migration::target", "config_volume": "nova_libvirt", "config_image": "172.16.6.1:8787/rhosp13/openstack-nova-compute:13.0-70" } ] export NET_HOST='true' export DEBUG='true' export PROCESS_COUNT=1 export CONFIG=/tmp/nova.json [root@overcloud-compute-0 docker-puppet]# python /var/lib/docker-puppet/docker-puppet.py |grep 'uuid set to' 2018-11-29 13:09:51,088 DEBUG: 89842 -- uuid set to: + echo 'uuid set to: '
Verified that openstack-tripleo-heat-templates-8.0.7-19.el7ost resolves the issue for me. -- Reproduce the issue with older templates (undercloud) [stack@undercloud13 templates]$ rpm -q openstack-tripleo-heat-templates openstack-tripleo-heat-templates-8.0.7-4.el7ost.noarch openstack overcloud deploy --stack overcloud \ [..] -e /home/stack/templates/per_node_hiera.yaml (undercloud) [stack@undercloud13 templates]$ cat /home/stack/templates/per_node_hiera.yaml resource_registry: OS::TripleO::ComputeExtraConfigPre: /home/stack/templates/openstack-tripleo-heat-templates/puppet/extraconfig/pre_deploy/per_node.yaml OS::TripleO::ControllerExtraConfigPre: /home/stack/templates/openstack-tripleo-heat-templates/puppet/extraconfig/pre_deploy/per_node.yaml parameter_defaults: NodeDataLookup: | {"a8fe66ff-3c42-42fb-804b-7612ba92af41": {"nova::debug": "True"}, "55907dbc-250b-42a8-a67c-1489cdfecd7f": {"nova::debug": "True"}} [root@overcloud-compute-0 ~]# dmidecode |grep -i uuid UUID: a8fe66ff-3c42-42fb-804b-7612ba92af41 [root@overcloud-controller-0 ~]# dmidecode |grep -i uuid UUID: 55907dbc-250b-42a8-a67c-1489cdfecd7f [root@overcloud-compute-0 ~]# cat /etc/puppet/hieradata/a8fe66ff-3c42-42fb-804b-7612ba92af41.json {"nova::debug": "True"} [root@overcloud-compute-0 ~]# crudini --get /var/lib/config-data/puppet-generated/nova_libvirt/etc/nova/nova.conf DEFAULT debug False [root@overcloud-controller-0 ~]# cat /etc/puppet/hieradata/55907dbc-250b-42a8-a67c-1489cdfecd7f.json {"nova::debug": "True"} [root@overcloud-controller-0 ~]# crudini --get /var/lib/config-data/puppet-generated/nova/etc/nova/nova.conf DEFAULT debug False -- Upgrade templates and verify fix. (undercloud) [stack@undercloud13 templates]$ sudo rpm -Uvh openstack-tripleo-heat-templates-8.0.7-23.el7ost.noarch.rpm Preparing... ################################# [100%] Updating / installing... 1:openstack-tripleo-heat-templates-################################# [ 50%] Cleaning up / removing... 2:openstack-tripleo-heat-templates-################################# [100%] cp -R /usr/share/openstack-tripleo-heat-templates /home/stack/templates/openstack-tripleo-heat-templates openstack overcloud deploy --stack overcloud \ [..] -e /home/stack/templates/per_node_hiera.yaml [root@overcloud-compute-0 ~]# cat /etc/puppet/hieradata/a8fe66ff-3c42-42fb-804b-7612ba92af41.json {"nova::debug": "True"} [root@overcloud-compute-0 ~]# crudini --get /var/lib/config-data/puppet-generated/nova_libvirt/etc/nova/nova.conf DEFAULT debug True [root@overcloud-controller-0 ~]# cat /etc/puppet/hieradata/55907dbc-250b-42a8-a67c-1489cdfecd7f.json {"nova::debug": "True"} [root@overcloud-controller-0 ~]# crudini --get /var/lib/config-data/puppet-generated/nova/etc/nova/nova.conf DEFAULT debug True
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:0068