Description of problem: 'tempest cleanup' reports authorization error when running cleanup after test runs What problem/issue/behavior are you having trouble with? What do you expect to see? 'tempest cleanup' reports authorization error when running cleanup after test runs. Only noticed to occur when running tests using the heat plugin. (engcloud) [stack@director tempest]$ tempest cleanup Begin cleanup Process 18 projects Cleaning project: d5da6cf0b2d8442586875880559d4481-fc5be417-89c4-471e-bff4-edddd33 Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/tempest/cmd/cleanup.py", line 112, in take_action self._cleanup() File "/usr/lib/python2.7/site-packages/tempest/cmd/cleanup.py", line 175, in _cleanup self._clean_project(project) File "/usr/lib/python2.7/site-packages/tempest/cmd/cleanup.py", line 217, in _clean_project **kwargs)) File "/usr/lib/python2.7/site-packages/tempest/common/credentials_factory.py", line 298, in get_credentials **params) File "/usr/lib/python2.7/site-packages/tempest/lib/auth.py", line 648, in get_credentials creds = auth_provider.fill_credentials() File "/usr/lib/python2.7/site-packages/tempest/lib/auth.py", line 124, in fill_credentials auth_data = self.get_auth() File "/usr/lib/python2.7/site-packages/tempest/lib/auth.py", line 150, in get_auth self.set_auth() File "/usr/lib/python2.7/site-packages/tempest/lib/auth.py", line 159, in set_auth self.cache = self._get_auth() File "/usr/lib/python2.7/site-packages/tempest/lib/auth.py", line 314, in _get_auth token, auth_data = auth_func(**auth_params) File "/usr/lib/python2.7/site-packages/tempest/lib/services/identity/v3/token_client.py", line 181, in get_token body = self.auth(**kwargs) File "/usr/lib/python2.7/site-packages/tempest/lib/services/identity/v3/token_client.py", line 132, in auth resp, body = self.post(self.auth_url, body=body) File "/usr/lib/python2.7/site-packages/tempest/lib/common/rest_client.py", line 279, in post return self.request('POST', url, extra_headers, headers, body, chunked) File "/usr/lib/python2.7/site-packages/tempest/lib/services/identity/v3/token_client.py", line 161, in request raise exceptions.Unauthorized(resp_body['error']['message']) Unauthorized: Unauthorized Details: The request you have made requires authentication. (engcloud) [stack@director tempest]$ openstack project list --long +----------------------------------+------------------------------------------------------------------+----------------------------------+-----------------------------------+---------+ | ID | Name | Domain ID | Description | Enabled | +----------------------------------+------------------------------------------------------------------+----------------------------------+-----------------------------------+---------+ | 01de83ce02004f4da8ef29a227189ad9 | d5da6cf0b2d8442586875880559d4481-fc5be417-89c4-471e-bff4-edddd33 | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project | True | | 19bc9ab03bd74df0aae536a6b2299207 | d5da6cf0b2d8442586875880559d4481-5041044f-5420-4815-8733-1ddbcf8 | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project | True | | 25edc75d1146485981f9072ef27a6c48 | d5da6cf0b2d8442586875880559d4481-ec74e469-c36c-45fb-9760-48cc970 | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project | True | | 29845792d8654202afe11cc02525ab75 | admin | default | admin tenant | True | | 2fa4c6d547004df1871efe83e86b9617 | d5da6cf0b2d8442586875880559d4481-93285bd5-080c-46a9-8255-d48ed1a | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project | True | | 3dfbd8e3141149c3bdbb902e658422b4 | d5da6cf0b2d8442586875880559d4481-6da0b1b1-7629-4c88-a44d-f09eab8 | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project | True | | 3ec266e0162d447c805ee7a3ac2f4bc2 | d5da6cf0b2d8442586875880559d4481-189c0eb5-281d-4d24-bf1b-8491de9 | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project | True | | 65415adbcaac42d1b67620717e6401bd | d5da6cf0b2d8442586875880559d4481-60ba1f8b-5465-4e1b-88e9-886d985 | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project | True | | 672529ea19d14f7582a703c163646803 | service | default | Tenant for the openstack services | True | | 77d733d1e0774173b691fb628545466f | project2 | bde7454715144b3d875f6e2b8b9af6ce | | True | | 864645bbf2e94b679ec319601297d652 | project1 | bde7454715144b3d875f6e2b8b9af6ce | | True | | 8aa289cd6c024708a45a6b10a01e1fff | d5da6cf0b2d8442586875880559d4481-4ae0652d-929b-4d59-8600-da047b3 | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project | True | | 8d502b85a0e843bf94c638f994a2b903 | alt_demo | default | Project for Tempest alt_demo user | True | | 979d0c6b4e9a468e9074f1e24edb91a1 | d5da6cf0b2d8442586875880559d4481-5516a755-30b3-4c81-bcfd-02a3d51 | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project | True | | a127a899896746c3a82c5dfdad20f157 | d5da6cf0b2d8442586875880559d4481-2c2f0cd5-329b-4fbc-8609-b6238b1 | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project | True | | abb89fdb5a084defafae9910737440af | d5da6cf0b2d8442586875880559d4481-d98d99af-d6ea-4aa0-b4b8-65da437 | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project | True | | ad17e1ca2d444da7b9368acf93ec1ead | d5da6cf0b2d8442586875880559d4481-55960d7b-d5af-4dcd-8592-2b864ba | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project | True | | be0fad4925b048949586652ffebb510c | d5da6cf0b2d8442586875880559d4481-f9957699-ef1d-45ab-a75d-ede04f3 | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project | True | | bfd70eec072f41cebc460dbdb1982ee1 | d5da6cf0b2d8442586875880559d4481-8a7ee36d-9522-4cc1-8bb2-a28bf0c | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project | True | | c3416b1b9c2d42a9a2ce48a5cd060188 | d5da6cf0b2d8442586875880559d4481-b15903c3-b293-4996-9b24-85a86a6 | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project | True | | c57483997a8743f2a98d54e99846c24e | d5da6cf0b2d8442586875880559d4481-ac9fca08-816a-4ff5-9673-bb3086d | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project | True | | cc80d87c98af4f8bb8e0777c1fa58ca7 | d5da6cf0b2d8442586875880559d4481-c50fa6f6-ceec-46df-8393-18acac6 | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project | True | | d5da6cf0b2d8442586875880559d4481 | demo | default | Project for Tempest demo user | True | | eb8770dc05d044859d21b5f6cca43066 | d5da6cf0b2d8442586875880559d4481-64b07b14-ffdd-4219-b2aa-f49f563 | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project | True | +----------------------------------+------------------------------------------------------------------+----------------------------------+-----------------------------------+---------+ Where are you experiencing the behavior? What environment? OSP 13.0.8. Tempest installed on the director and the tests are executed on the overcloud. (engcloud) [stack@director-eng eng-tempest]$ rpm -qa | grep tempest python2-neutron-tests-tempest-0.0.1-0.20180425142847.02a5e2b.el7ost.noarch python2-keystone-tests-tempest-0.0.1-1.0.564670bgit.el7ost.noarch python2-tempest-18.0.0-10.el7ost.noarch python2-heat-tests-tempest-0.3.0-1.el7ost.noarch python2-novajoin-tests-tempest-0.0.1-0.20180219103837.59d7ba6.el7ost.noarch python2-cinder-tests-tempest-0.0.1-1.0.aafbdf8git.el7ost.noarch python2-horizon-tests-tempest-0.0.1-0.20180219094157.a23f407.el7ost.noarch openstack-tempest-18.0.0-10.el7ost.noarch python2-tempestconf-2.2.0-1.el7ost.noarch When does the behavior occur? Frequently? Repeatedly? At certain times? Repeatedly
*** Bug 1747513 has been marked as a duplicate of this bug. ***
Info how to reproduce the issue is described in the upstream bug: https://bugs.launchpad.net/tempest/+bug/1766582/comments/1 The exact reason why and a way how to fix this are still being investigated.
Created attachment 1662407 [details] verification output The 'Fixed in version' contains the fix for the issue, verified on a 13 env using 2020-02-06.2 puddle. Verification output attached.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:0769