Bug 1725865

Summary:	openstack overcloud upgrade prepare fails to update heat templates
Product:	Red Hat OpenStack	Reporter:	Sergii Golovatiuk <sgolovat>
Component:	openstack-heat	Assignee:	Rabi Mishra <ramishra>
Status:	CLOSED ERRATA	QA Contact:	Sasha Smolyak <ssmolyak>
Severity:	high	Docs Contact:
Priority:	urgent
Version:	15.0 (Stein)	CC:	apevec, cjeanner, emacchi, jstransk, lhh, mburns, ramishra, rrasouli, sbaker, shardy, ssmolyak
Target Milestone:	rc	Keywords:	Triaged
Target Release:	15.0 (Stein)
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:	openstack-heat-12.0.1-0.20190704050403.bf16acc.el8ost	Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-09-21 11:23:46 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Sergii Golovatiuk 2019-07-01 15:22:48 UTC

Description of problem:


How reproducible:

Always.

Steps to Reproduce:
1. Deploy OSP14
2. Upgrade Undercloud to 15.
3. Upgrade Upgrade of Overcloud

Actual results:

"openstack overcloud upgrade prepare" fails with the following output


2019-07-01 14:17:48Z [overcloud]: UPDATE_FAILED  The Referenced Attribute (Controller deployed_server_deployment_swift_data_map) is incorrect.
2019-07-01 14:17:48Z [overcloud.ControllerAllNodesValidationConfig]: UPDATE_IN_PROGRESS  state changed
2019-07-01 14:17:48Z [overcloud-hostsConfig-7jgatdsigqcv]: UPDATE_IN_PROGRESS  Stack UPDATE started
2019-07-01 14:17:48Z [overcloud-SshKnownHostsConfig-ntvcmllmgqmv]: UPDATE_COMPLETE  Stack UPDATE completed successfully
2019-07-01 14:17:48Z [overcloud-hostsConfig-7jgatdsigqcv]: UPDATE_COMPLETE  Stack UPDATE completed successfully
2019-07-01 14:17:48Z [overcloud-ControllerIpListMap-tdm2fhrcy5e5]: UPDATE_IN_PROGRESS  Stack UPDATE started
2019-07-01 14:17:49Z [overcloud-ControllerAllNodesValidationConfig-vxqw5fxzlbvc]: UPDATE_IN_PROGRESS  Stack UPDATE started
2019-07-01 14:17:49Z [overcloud-ControllerAllNodesValidationConfig-vxqw5fxzlbvc]: UPDATE_COMPLETE  Stack UPDATE completed successfully
2019-07-01 14:17:49Z [overcloud.SshKnownHostsConfig]: UPDATE_COMPLETE  state changed
2019-07-01 14:17:49Z [overcloud.hostsConfig]: UPDATE_COMPLETE  state changed
2019-07-01 14:17:49Z [overcloud-ControllerIpListMap-tdm2fhrcy5e5]: UPDATE_COMPLETE  Stack UPDATE completed successfully
2019-07-01 14:17:50Z [overcloud.ControllerAllNodesValidationConfig]: UPDATE_COMPLETE  state changed
2019-07-01 14:17:50Z [overcloud.ControllerIpListMap]: UPDATE_COMPLETE  state changed

 Stack overcloud/7aaf24c0-5bad-4630-a963-5a4f21f9c290 UPDATE_FAILED


Expected results:


 Stack overcloud/7aaf24c0-5bad-4630-a963-5a4f21f9c290 UPDATE_COMPLETE

Success

Additional info:

Revert of https://review.opendev.org/#/c/664790/ helped.

Comment 1 Jiri Stransky 2019-07-02 08:42:10 UTC

This is probably due to removal of the attribute from templates, but not from the stack. We have already hit similar issues several times in Heat, though previously mainly w/r/t resource registry, e.g. i was just able to dig up this:

https://bugs.launchpad.net/tripleo/+bug/1758014

We probably need a revert or at least a partial revert (put back the referenced attribute). Or we need to fix Heat if it's possible.

Comment 2 Jiri Stransky 2019-07-02 09:13:59 UTC

This could be removed in a 2 stage process. First remove the reference from overcloud.j2 and set the attribute in role.role.j2 to empty value, and in next release remove the attribute from role.role.j2 completely. That's very similar to how we "fixed" the resource registry issues earlier.

Comment 14 Ronnie Rasouli 2019-07-24 13:25:30 UTC

The process is working based on process:
https://gitlab.cee.redhat.com/osp15/osp-upgrade-el8

Comment 16 Sasha Smolyak 2019-07-31 11:19:35 UTC

The upgrade job passed.
Verified

Comment 20 errata-xmlrpc 2019-09-21 11:23:46 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2019:2811