Bug 1303112 - Stack failed with status: expected string or buffer
Stack failed with status: expected string or buffer
Status: CLOSED ERRATA
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-heat (Show other bugs)
8.0 (Liberty)
Unspecified Unspecified
unspecified Severity low
: ga
: 8.0 (Liberty)
Assigned To: Thomas Hervé
Omri Hochman
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2016-01-29 10:13 EST by Jiri Stransky
Modified: 2016-04-26 12:07 EDT (History)
12 users (show)

See Also:
Fixed In Version: openstack-heat-5.0.1-2.el7ost
Doc Type: Bug Fix
Doc Text:
Previously, heat changed the name of properties on several neutron resources; while it used a mechanism to support the old names when creating them, it failed to validate resources created with a previous version. Consequently, using Red Hat OpenStack Platform 8 to update a stack created in version 7 (or previous) using with a neutron port resource would fail by trying to lookup a 'None' object. With this fix, when heat updates the resource, it now uses the translation mechanism on old properties too. As a result, supporting deprecated properties now works as expected with resources created from a previous version.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-04-07 17:26:54 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
heat-engine-snippet.log (11.24 KB, text/plain)
2016-01-29 10:20 EST, Jiri Stransky
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
Launchpad 1541516 None None None 2016-02-03 12:14 EST
OpenStack gerrit 276266 None None None 2016-02-18 09:37 EST

  None (edit)
Description Jiri Stransky 2016-01-29 10:13:51 EST
Description of problem:

(Might be an intermittent error or a one-time issue, will update the BZ if i'm able to reproduce it.)

Trying an upgrade from 7.2 to stable/liberty or 8 poodle (first i tried with stable/liberty with custom upgrade patches, then with downstream RPM with just resource registry passed in, failing on the same issue):

[stack@instack ~]$ openstack overcloud deploy     --libvirt-type qemu     --ntp-server clock.redhat.com     --control-scale 3 --templates -e /usr/share/openstack-tripleo-heat-templates/overcloud-resource-registry-puppet.yaml
Deploying templates in the directory /usr/share/openstack-tripleo-heat-templates
Stack failed with status: expected string or buffer
Heat Stack update failed.

Versions:

heat used for deploy: 2015.1.2-4.el7ost
heat used for upgrade: 5.0.1-1.el7ost

t-h-t used for deploy: 0.8.6-94.el7ost
t-h-t used for upgrade: 0.8.7-7.el7ost
Comment 1 Jiri Stransky 2016-01-29 10:20 EST
Created attachment 1119451 [details]
heat-engine-snippet.log

Full heat-engine log is too big, attaching an excerpt which seems to be the important part
Comment 2 Jiri Stransky 2016-01-29 10:20:59 EST
Heat-engine log contains warnings that instances can't be found, however i can list them and ssh into them:

[stack@instack ~]$ nova list
+--------------------------------------+------------------------+--------+------------+-------------+---------------------+
| ID                                   | Name                   | Status | Task State | Power State | Networks            |
+--------------------------------------+------------------------+--------+------------+-------------+---------------------+
| 9b2935b3-0dbf-467e-a7b0-991f7da4ae4c | overcloud-compute-0    | ACTIVE | -          | Running     | ctlplane=192.0.2.8  |
| 1bfdeba9-1b1e-4668-8df5-659b4a2f6771 | overcloud-controller-0 | ACTIVE | -          | Running     | ctlplane=192.0.2.10 |
| fe4fa6cc-669e-49bc-aee7-323cca7c235c | overcloud-controller-1 | ACTIVE | -          | Running     | ctlplane=192.0.2.9  |
| 0cad78bd-bb65-4b5b-85a2-347464582512 | overcloud-controller-2 | ACTIVE | -          | Running     | ctlplane=192.0.2.11 |
+--------------------------------------+------------------------+--------+------------+-------------+---------------------+
Comment 4 Crag Wolfe 2016-02-02 16:04:05 EST
Any chance you have before/after-upgrade copies of /etc/heat/heat.conf? The not-finding nova instances feels like it might be an auth issue (likewise for the neutron port).
Comment 5 Crag Wolfe 2016-02-02 19:48:48 EST
Adding
            LOG.debug('_needs_update() looking up before_id %s, %s, %s' %
                        (before_props and before_props.items(), self.NETWORK, 'network'))

to line 428 of /usr/lib/python2.7/site-packages/heat/engine/resources/openstack/neutron/port.py might also provide a little insight.
Comment 6 Thomas Hervé 2016-02-03 11:57:38 EST
Reproduced. We're getting None in the network property, because the resource has been created in kilo, and we don't translate it.

http://paste.openstack.org/show/485868/ is a tentative fix.
Comment 7 Thomas Hervé 2016-02-03 12:20:40 EST
Slightly updated version at https://review.openstack.org/275832
Comment 10 errata-xmlrpc 2016-04-07 17:26:54 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-0603.html

Note You need to log in before you can comment on or make changes to this bug.