Bug 1438015 - Heat doesn't renew token leading to authorization failure in deployments going past default 240 min timeout
Summary: Heat doesn't renew token leading to authorization failure in deployments goin...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: instack-undercloud
Version: 10.0 (Newton)
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: rc
: 12.0 (Pike)
Assignee: Rabi Mishra
QA Contact: Ronnie Rasouli
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-03-31 15:55 UTC by Sai Sindhur Malleni
Modified: 2018-02-05 19:07 UTC (History)
8 users (show)

Fixed In Version: instack-undercloud-7.1.1-0.20170616135935.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-12-13 21:22:29 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1683983 0 None None None 2017-04-19 13:18:51 UTC
OpenStack gerrit 457837 0 None MERGED Make 'reauthentication_auth_method=trusts' the default 2020-09-17 19:53:16 UTC
Red Hat Product Errata RHEA-2017:3462 0 normal SHIPPED_LIVE Red Hat OpenStack Platform 12.0 Enhancement Advisory 2018-02-16 01:43:25 UTC

Description Sai Sindhur Malleni 2017-03-31 15:55:35 UTC
Description of problem:
When doing large scale deployments where timeout needs to be set higher than 240 mins, we see authorization failures in deployment command around the time default timeout 240 mins reached due to keystone token expiration set to 14400s (240mins). Heat doesn't seem to renew token from keystone, so the workaround is to also bump keystone token expiration time to the timeout value used in overcloud deploy command. We see this

2017-03-18 19:54:22Z [overcloud.Compute]: CREATE_FAILED Resource CREATE failed: Unauthorized: resources[81].resources.NovaCompute: The request you have made requires authentication. (HTTP 401) (Request-ID: req-f3373924-3da4-4349-8b46-b2430ad3dc3f)
2017-03-18 19:54:22Z [overcloud.Compute]: CREATE_FAILED Unauthorized: resources.Compute.resources[81].resources.NovaCompute: The request you have made requires authentication. (HTTP 401) (Request-ID: req- f3373924-3da4-4349-8b46-b2430ad3dc3f)
2017-03-18 19:54:23Z [overcloud]: CREATE_FAILED Resource CREATE failed: Unauthorized: resources.Compute.resources[81].resources.NovaCompute: The request you have made requires authentication. (HTTP 401) (Request-ID: req-f3373924-3da4-4349-8b46-b2430ad3dc3f)
Version-Release number of selected component (if applicable):
RHOP 10

How reproducible:
100%

Steps to Reproduce:
1. Do large scale deployments
2. Bump timeout in deploy command to > 240 mins.
3.

Actual results:
Although timeout was set to 360minutes, stack create failed because of authorization errors.

Expected results:
Deployment should continue until timeout passed to the overcloud deploy command

Additional info:

Comment 1 Rabi Mishra 2017-04-17 06:00:52 UTC
To allow re-authentication on token expiry, such that long-running tasks may complete, heat has a flag 'reauthentication_auth_method', which can be set to 'trusts' in heat.conf. This would allow for trust to be used in place of user token.

Comment 2 Zane Bitter 2017-04-18 15:27:52 UTC
Do we need to change something in TripleO to make that the default?

Comment 3 Zane Bitter 2017-05-18 14:12:27 UTC
Fixed upstream, but backports are not feasible due to reliance on new features as well as bug fixes in other projects. Retargeting for OSP12.

Comment 8 Ronnie Rasouli 2017-12-05 08:49:11 UTC
fixed landed on downstream

Comment 11 errata-xmlrpc 2017-12-13 21:22:29 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:3462


Note You need to log in before you can comment on or make changes to this bug.