This service will be undergoing maintenance at 00:00 UTC, 2017-10-23 It is expected to last about 30 minutes
Bug 1304854 - Metadata polling unnecessarily calls Nova
Metadata polling unnecessarily calls Nova
Status: CLOSED ERRATA
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-heat (Show other bugs)
7.0 (Kilo)
Unspecified Unspecified
high Severity high
: z4
: 7.0 (Kilo)
Assigned To: Zane Bitter
Amit Ugol
: ZStream
Depends On:
Blocks: 1304856
  Show dependency treegraph
 
Reported: 2016-02-04 14:43 EST by Zane Bitter
Modified: 2016-04-26 09:23 EDT (History)
6 users (show)

See Also:
Fixed In Version: openstack-heat-2015.1.2-8.el7ost
Doc Type: Bug Fix
Doc Text:
Previously, when the metadata for a resource was requested from Heat, Heat fetched the value of every attribute of the resource, even though this data was not returned by the API. This meant at least one and possibly multiple pointless ReST API calls to the OpenStack service underlying the resource. The metadata in Heat gets polled regularly by in-guest agents like os-collect-config, because it is the mechanism by which software deployments are triggered. Because polling from a server generally uses a Heat "stack user" account from a different keystone domain rather than the stack owner's account, the resource cannot actually be found and "404 Not Found" messages were accumulating in the logs of both heat-engine and nova-api. Heat no longer calculates attribute values when only the metadata for a resource is requested.
Story Points: ---
Clone Of:
: 1304856 (view as bug list)
Environment:
Last Closed: 2016-02-18 11:43:29 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Launchpad 1507568 None None None 2016-02-04 14:43 EST
OpenStack gerrit 272353 None None None 2016-02-04 14:48 EST

  None (edit)
Description Zane Bitter 2016-02-04 14:43:55 EST
When the metadata for a resource is requested from Heat, Heat fetches the value of every attribute of the resource, even though this data is not returned by the API. This generally means at least one and possibly multiple pointless ReST API calls to the OpenStack service underlying the resource.

The metadata in Heat gets polled regularly by in-guest agents like os-collect-config, because it is the mechanism by which software deployments are triggered. In a large deployment (like a substantial TripleO cloud), there is a lot of polling... enough to put a substantial load on Heat. This is probably one of the reasons for the load.

In addition, because polling from a server generally uses a Heat "stack user" account from a different keystone domain rather than the stack owner's account, the resource cannot actually be found and the log is polluted with a lot of useless messages of the form:

WARNING heat.engine.clients.os.nova [req-2c4da8b3-f441-43c3-bb59-bbe1a6665a6f None 2240eae5cccc439298daa101fc8557bd-96361e11-11f8-45d1-a89b-732401f] Instance (099bb7bb-f22d-4739-a077-4776a09bbda9) not found: Instance 099bb7bb-f22d-4739-a077-4776a09bbda9 could not be found. (HTTP 404)

And of course nova-api also logs the 404s.
Comment 3 Amit Ugol 2016-02-16 07:17:36 EST
on latest version as of now, none of those messages show in the logs.
Comment 5 errata-xmlrpc 2016-02-18 11:43:29 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2016-0266.html

Note You need to log in before you can comment on or make changes to this bug.