Red Hat Bugzilla – Bug 1438015
Heat doesn't renew token leading to authorization failure in deployments going past default 240 min timeout
Last modified: 2018-02-05 14:07:22 EST
Description of problem: When doing large scale deployments where timeout needs to be set higher than 240 mins, we see authorization failures in deployment command around the time default timeout 240 mins reached due to keystone token expiration set to 14400s (240mins). Heat doesn't seem to renew token from keystone, so the workaround is to also bump keystone token expiration time to the timeout value used in overcloud deploy command. We see this 2017-03-18 19:54:22Z [overcloud.Compute]: CREATE_FAILED Resource CREATE failed: Unauthorized: resources[81].resources.NovaCompute: The request you have made requires authentication. (HTTP 401) (Request-ID: req-f3373924-3da4-4349-8b46-b2430ad3dc3f) 2017-03-18 19:54:22Z [overcloud.Compute]: CREATE_FAILED Unauthorized: resources.Compute.resources[81].resources.NovaCompute: The request you have made requires authentication. (HTTP 401) (Request-ID: req- f3373924-3da4-4349-8b46-b2430ad3dc3f) 2017-03-18 19:54:23Z [overcloud]: CREATE_FAILED Resource CREATE failed: Unauthorized: resources.Compute.resources[81].resources.NovaCompute: The request you have made requires authentication. (HTTP 401) (Request-ID: req-f3373924-3da4-4349-8b46-b2430ad3dc3f) Version-Release number of selected component (if applicable): RHOP 10 How reproducible: 100% Steps to Reproduce: 1. Do large scale deployments 2. Bump timeout in deploy command to > 240 mins. 3. Actual results: Although timeout was set to 360minutes, stack create failed because of authorization errors. Expected results: Deployment should continue until timeout passed to the overcloud deploy command Additional info:
To allow re-authentication on token expiry, such that long-running tasks may complete, heat has a flag 'reauthentication_auth_method', which can be set to 'trusts' in heat.conf. This would allow for trust to be used in place of user token.
Do we need to change something in TripleO to make that the default?
Fixed upstream, but backports are not feasible due to reliance on new features as well as bug fixes in other projects. Retargeting for OSP12.
fixed landed on downstream
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:3462