Bug 1031034 (heat-bp-support-retry-with-idempotency)

Cloned from launchpad blueprint https://blueprints.launchpad.net/heat/+spec/support-retry-with-idempotency.

Description:

Currently Heat doesn't have a retry function in creating/updating a stack.
In case of API request failure, Heat would change stack status to "XXX_FAILURE" .
(Or start rollback process if the rollback flag was true.)

However, I think there are some circumstances/scenarios where retry is appropriate.
(i.e. 503 response or timeout due to server failover)

On the other hand, however, a retry function may result in creating duplicate resources when Heat couldn't get response from nova and other APIs.
In order to avoid this, I believe leveraging idempotency that is discussed in nova could be a solution.

https://blueprints.launchpad.net/nova/+spec/idempotentcy-client-token

Our idea for implementing a retry function to Heat is:
Heat to retry POST requests with a ClientToken header to avoid creating duplicate resources, until it reaches a retry limit.
Heat to retry PUT/GET requests until it gets 2xx response or it reaches a retry limit. ClientTokens are not used in this case.
Heat to retry DELETE requests until it gets 2xx response or 404 response or it reaches a retry limit. ClientTokens are not used in this case.

Note:
Some of pythonclients have retry functions, but they cannot be utilized because Heat has to judge the necessity of retry by itself.


Specification URL (additional information):

None