== Info about my Environment == I'm using a test lab with an Undercloud and a 2-node Overcloud (1 Compute and 1 Controller). The Overcloud is using NFS shares for storage and the details for it are included the storage-environment.yaml. I'm not using network isolation. == Where I'm up to == I'm doing a test undercloud and overcloud upgrade and following instructions here: http://etherpad.corp.redhat.com/osp-d-upgrade-kbase I've successfully upgraded the Undercloud and imported the new Overcloud images. == The problem == I've created the 'rhos-release-8.yaml' template and am running the deploy command on line 106. However, I've modified it to not include network isolation files and instead include storage environment files: $ openstack overcloud deploy --templates /home/stack/templates/tripleo-heat-templates \ -e /home/stack/templates/tripleo-heat-templates/overcloud-resource-registry-puppet.yaml \ -e /home/stack/templates/tripleo-heat-templates/environments/puppet-pacemaker.yaml \ -e /home/stack/templates/storage-environment.yaml \ -e /home/stack/templates/tripleo-heat-templates/environments/major-upgrade-pacemaker-init.yaml \ -e /home/stack/templates/rhos-release-8.yaml \ --ntp-server pool.ntp.org --neutron-network-type vxlan --neutron-tunnel-types vxlan The command returns the following: Deploying templates in the directory /home/stack/templates/tripleo-heat-templates ERROR: The Parameter (GlanceFilePcmkFstype) was not defined in template. If I omit '-e /home/stack/templates/storage-environment.yaml', I still get the same error. I originally got the same error regarding 'GlanceFilePcmkManage' but I remember that for OSP 8 you need to change the section for the storage-environment.yaml from 'parameters' to 'parameter_defaults'. The 'GlanceFilePcmkManage' error is gone but replaced with the one for 'GlanceFilePcmkFstype'. The weird thing is the parameter seems to exist in puppet/controller.yaml but for some reason Heat isn't picking up on it.
From Jiri Stransky: If it's certain that `GlanceFilePcmkFstype` is not under `parameters` anywhere now, the only other issue i can think of is that the values previously passed as `parameters` when deploying 7.3 could be still in effect even for the stack-update. Can you please report the behavior you're seeing as a bug? There's a fix to prevent this from happening for the parameters that are being set from tripleoclient directly [1], but there's no fix for arbitrary parameters that got in through environment files, and i think tripleoclient won't allow to pass-through --clear-parameters to heat stack-update at the moment. This is a fairly generic issue -- now that we're using `parameter_defaults` for more things, we will not be able to override any parameters that were previously set using `parameters`. We're hitting this with NFS first, but there's going to be another significant hit of this issue when the "composable services within roles" spec gets implemented, as that would most probably involve switching from `parameters` to `parameter_defaults` for many things. We might need to allow tripleoclient to pass through --clear-parameters to Heat in order to work through these issues manually on a case-by-case basis. Another option might be to compute `clear_parameters` in tripleoclient in a smarter way, e.g. looking at the final single environment file, taking a set of its `parameter_defaults`, subtracting the set of its `parameters` from it and passing that to `clear_parameters`. I'm not certain this solution is free of undesirable consequences, but at the moment i can't think of any specific ones for TripleO. Anyone please chime in if you have some feedback on these ideas. Jirka [1] https://review.openstack.org/#/c/291690/
The likelihood is that it's looking for the parameter in the *old* version of the template, and thus not finding it. After a previous update fails, the content of the old template can become mixed with the new one, sometimes leading to situations like this - there was a whole class of bugs of this type at one point: https://bugzilla.redhat.com/show_bug.cgi?id=1310879 https://bugzilla.redhat.com/show_bug.cgi?id=1298589 https://bugzilla.redhat.com/show_bug.cgi?id=1303084 However, all of those ones are fixed in openstack-heat-5.0.1-3.el7ost. If you're already running that build then please raise a new bugzilla against openstack-heat, and include any relevant tracebacks from /var/log/heat/heat-engine.log on the undercloud.
Comment #3 was from Zane Bitter
Confirming openstack-heat-5.0.1-3.elost: [root@director ~]# yum list openstack-heat* Loaded plugins: product-id, search-disabled-repos, subscription-manager Installed Packages openstack-heat-api.noarch 1:5.0.1-3.el7ost @rhelosp-8.0-puddle openstack-heat-api-cfn.noarch 1:5.0.1-3.el7ost @rhelosp-8.0-puddle openstack-heat-api-cloudwatch.noarch 1:5.0.1-3.el7ost @rhelosp-8.0-puddle openstack-heat-common.noarch 1:5.0.1-3.el7ost @rhelosp-8.0-puddle openstack-heat-engine.noarch 1:5.0.1-3.el7ost @rhelosp-8.0-puddle openstack-heat-templates.noarch 0-0.8.20150605git.el7ost @rhel-7-server-openstack-7.0-rpms
Created attachment 1137268 [details] Snippet from heat-engine.log during the 'openstack overcloud deploy' command
It looks like anothe manifestation of https://bugzilla.redhat.com/show_bug.cgi?id=1310879. We still store some mismatch of templates in case of failure, and we only workaround it when looking at properties. It looks like we have another issue with parameters this time.
OK, that's was not related to that previous issue, this is a relatively simple problem in the interaction between Tripleo and Heat when a parameter is removed. I believe the attached patch will fix it.
I reproduced the bug and can confirm that the attached patch fixed it in my testing. Thanks Thomas!
Tested openstack-heat-5.0.1-4.el7ost on my environment. The error does not appear anymore. Thanks Thomas, Jiri and Zane!
Following the internal mail thread, upgrades are working better now => this is verified.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHEA-2016-0603.html