+++ This bug was initially created as a clone of Bug #1976919 +++

Description of problem:

Command openstack overcloud deploy fails. The ansible.log is attached.
I presume that error is connected with hiera and puppet, I found these in ansible.log:


2021-06-25 10:07:10,549 p=1044 u=mistral n=ansible | 2021-06-25 10:07:10.547788 | fa163eb3-9b85-a684-abe1-000000000826 |      FATAL | Run puppet on the host to apply IPtables rules | rhosp16-overcloud-cont-18492-1 | error={"changed": true, "cmd": "set +e
puppet apply --debug --verbose --detailed-exitcodes --summarize --color=false    --modulepath '/etc/puppet/modules:/opt/stack/puppet-modules:/usr/share/openstack-puppet/modules' --tags 'tripleo::firewall::rule' -e 'if hiera('enable_load_balancer', true) { class {'::tripleo::haproxy': use_internal_certificates => false, manage_firewall => hiera('tripleo::firewall::manage_firewall', true), }}'
rc=$?
set -e
set +ux
if [ $rc -eq 2 -o $rc -eq 0 ]; then
    exit 0
fi
exit $rc
", "delta": "0:00:09.624070", "end": "2021-06-25 10:07:10.481612", "msg": "non-zero return code", "rc": 1, "start": "2021-06-25 10:07:00.857542", "stderr": "Warning: The function 'hiera' is deprecated in favor of using 'lookup'. See https://puppet.com/docs/puppet/5.5/deprecated_language.html\
   (file & line not available)
Warning: /etc/puppet/hiera.yaml: Use of 'hiera.yaml' version 3 is deprecated. It should be converted to version 5
   (file: /etc/puppet/hiera.yaml)
Error: Evaluation Error: Error while evaluating a Function Call, Lookup of key 'lookup_options' failed:  The Lookup Configuration at '/etc/puppet/hiera.yaml' has wrong type, unrecognized key 'changed'
  The Lookup Configuration at '/etc/puppet/hiera.yaml' has wrong type, unrecognized key 'skip_reason'
  The Lookup Configuration at '/etc/puppet/hiera.yaml' has wrong type, unrecognized key 'skipped' (line: 1, column: 4) on node 

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

--- Additional comment from Alex Schultz on 2021-06-28 15:05:46 UTC ---

We don't have a reference to lookup_options in puppet-tripleo.  Can you run with ConfigDebug: True and provide additional logs? Additionally can you provide rpm versions and a sosreport?

--- Additional comment from Alex Schultz on 2021-06-28 15:21:17 UTC ---

I've identified the issue.  Working on a fix.  The issue is that an ansible task is overwriting the var we used to generate the hiera.yaml configuration.