Bug 1755132 - 'tempest cleanup' reports authorization error when running cleanup after test runs
Summary: 'tempest cleanup' reports authorization error when running cleanup after test...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tempest
Version: 13.0 (Queens)
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: z11
: 13.0 (Queens)
Assignee: Chandan Kumar
QA Contact: Martin Kopec
URL:
Whiteboard:
: 1747513 (view as bug list)
Depends On: 1746091
Blocks: epmosp13bugs 1588541
TreeView+ depends on / blocked
 
Reported: 2019-09-24 20:31 UTC by David Hill
Modified: 2023-09-07 20:39 UTC (History)
10 users (show)

Fixed In Version: openstack-tempest-18.0.0-12.el7ost
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-03-10 11:52:51 UTC
Target Upstream Version:
Embargoed:
mkopec: needinfo-
mkopec: needinfo-


Attachments (Terms of Use)
verification output (42.81 KB, text/plain)
2020-02-11 09:04 UTC, Martin Kopec
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1766582 0 None None None 2019-09-30 21:07:39 UTC
OpenStack gerrit 686738 0 'None' MERGED tempest cleanup - use admin_mgr only 2020-06-29 09:52:34 UTC
Red Hat Issue Tracker OSP-3808 0 None None None 2022-08-18 17:19:58 UTC
Red Hat Product Errata RHBA-2020:0769 0 None None None 2020-03-10 11:53:13 UTC

Description David Hill 2019-09-24 20:31:16 UTC
Description of problem:
'tempest cleanup' reports authorization error when running cleanup after test runs

What problem/issue/behavior are you having trouble with?  What do you expect to see?

'tempest cleanup' reports authorization error when running cleanup after test runs. Only noticed to occur when running tests using the heat plugin.

(engcloud) [stack@director tempest]$ tempest cleanup
Begin cleanup
Process 18 projects
Cleaning project:  d5da6cf0b2d8442586875880559d4481-fc5be417-89c4-471e-bff4-edddd33
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/tempest/cmd/cleanup.py", line 112, in take_action
    self._cleanup()
  File "/usr/lib/python2.7/site-packages/tempest/cmd/cleanup.py", line 175, in _cleanup
    self._clean_project(project)
  File "/usr/lib/python2.7/site-packages/tempest/cmd/cleanup.py", line 217, in _clean_project
    **kwargs))
  File "/usr/lib/python2.7/site-packages/tempest/common/credentials_factory.py", line 298, in get_credentials
    **params)
  File "/usr/lib/python2.7/site-packages/tempest/lib/auth.py", line 648, in get_credentials
    creds = auth_provider.fill_credentials()
  File "/usr/lib/python2.7/site-packages/tempest/lib/auth.py", line 124, in fill_credentials
    auth_data = self.get_auth()
  File "/usr/lib/python2.7/site-packages/tempest/lib/auth.py", line 150, in get_auth
    self.set_auth()
  File "/usr/lib/python2.7/site-packages/tempest/lib/auth.py", line 159, in set_auth
    self.cache = self._get_auth()
  File "/usr/lib/python2.7/site-packages/tempest/lib/auth.py", line 314, in _get_auth
    token, auth_data = auth_func(**auth_params)
  File "/usr/lib/python2.7/site-packages/tempest/lib/services/identity/v3/token_client.py", line 181, in get_token
    body = self.auth(**kwargs)
  File "/usr/lib/python2.7/site-packages/tempest/lib/services/identity/v3/token_client.py", line 132, in auth
    resp, body = self.post(self.auth_url, body=body)
  File "/usr/lib/python2.7/site-packages/tempest/lib/common/rest_client.py", line 279, in post
    return self.request('POST', url, extra_headers, headers, body, chunked)
  File "/usr/lib/python2.7/site-packages/tempest/lib/services/identity/v3/token_client.py", line 161, in request
    raise exceptions.Unauthorized(resp_body['error']['message'])
Unauthorized: Unauthorized
Details: The request you have made requires authentication.

(engcloud) [stack@director tempest]$ openstack project list --long
+----------------------------------+------------------------------------------------------------------+----------------------------------+-----------------------------------+---------+
| ID                               | Name                                                             | Domain ID                        | Description                       | Enabled |
+----------------------------------+------------------------------------------------------------------+----------------------------------+-----------------------------------+---------+
| 01de83ce02004f4da8ef29a227189ad9 | d5da6cf0b2d8442586875880559d4481-fc5be417-89c4-471e-bff4-edddd33 | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project           | True    |
| 19bc9ab03bd74df0aae536a6b2299207 | d5da6cf0b2d8442586875880559d4481-5041044f-5420-4815-8733-1ddbcf8 | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project           | True    |
| 25edc75d1146485981f9072ef27a6c48 | d5da6cf0b2d8442586875880559d4481-ec74e469-c36c-45fb-9760-48cc970 | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project           | True    |
| 29845792d8654202afe11cc02525ab75 | admin                                                            | default                          | admin tenant                      | True    |
| 2fa4c6d547004df1871efe83e86b9617 | d5da6cf0b2d8442586875880559d4481-93285bd5-080c-46a9-8255-d48ed1a | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project           | True    |
| 3dfbd8e3141149c3bdbb902e658422b4 | d5da6cf0b2d8442586875880559d4481-6da0b1b1-7629-4c88-a44d-f09eab8 | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project           | True    |
| 3ec266e0162d447c805ee7a3ac2f4bc2 | d5da6cf0b2d8442586875880559d4481-189c0eb5-281d-4d24-bf1b-8491de9 | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project           | True    |
| 65415adbcaac42d1b67620717e6401bd | d5da6cf0b2d8442586875880559d4481-60ba1f8b-5465-4e1b-88e9-886d985 | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project           | True    |
| 672529ea19d14f7582a703c163646803 | service                                                          | default                          | Tenant for the openstack services | True    |
| 77d733d1e0774173b691fb628545466f | project2                                                         | bde7454715144b3d875f6e2b8b9af6ce |                                   | True    |
| 864645bbf2e94b679ec319601297d652 | project1                                                         | bde7454715144b3d875f6e2b8b9af6ce |                                   | True    |
| 8aa289cd6c024708a45a6b10a01e1fff | d5da6cf0b2d8442586875880559d4481-4ae0652d-929b-4d59-8600-da047b3 | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project           | True    |
| 8d502b85a0e843bf94c638f994a2b903 | alt_demo                                                         | default                          | Project for Tempest alt_demo user | True    |
| 979d0c6b4e9a468e9074f1e24edb91a1 | d5da6cf0b2d8442586875880559d4481-5516a755-30b3-4c81-bcfd-02a3d51 | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project           | True    |
| a127a899896746c3a82c5dfdad20f157 | d5da6cf0b2d8442586875880559d4481-2c2f0cd5-329b-4fbc-8609-b6238b1 | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project           | True    |
| abb89fdb5a084defafae9910737440af | d5da6cf0b2d8442586875880559d4481-d98d99af-d6ea-4aa0-b4b8-65da437 | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project           | True    |
| ad17e1ca2d444da7b9368acf93ec1ead | d5da6cf0b2d8442586875880559d4481-55960d7b-d5af-4dcd-8592-2b864ba | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project           | True    |
| be0fad4925b048949586652ffebb510c | d5da6cf0b2d8442586875880559d4481-f9957699-ef1d-45ab-a75d-ede04f3 | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project           | True    |
| bfd70eec072f41cebc460dbdb1982ee1 | d5da6cf0b2d8442586875880559d4481-8a7ee36d-9522-4cc1-8bb2-a28bf0c | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project           | True    |
| c3416b1b9c2d42a9a2ce48a5cd060188 | d5da6cf0b2d8442586875880559d4481-b15903c3-b293-4996-9b24-85a86a6 | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project           | True    |
| c57483997a8743f2a98d54e99846c24e | d5da6cf0b2d8442586875880559d4481-ac9fca08-816a-4ff5-9673-bb3086d | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project           | True    |
| cc80d87c98af4f8bb8e0777c1fa58ca7 | d5da6cf0b2d8442586875880559d4481-c50fa6f6-ceec-46df-8393-18acac6 | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project           | True    |
| d5da6cf0b2d8442586875880559d4481 | demo                                                             | default                          | Project for Tempest demo user     | True    |
| eb8770dc05d044859d21b5f6cca43066 | d5da6cf0b2d8442586875880559d4481-64b07b14-ffdd-4219-b2aa-f49f563 | 22cbc8fbc95842f389af5f665d4c9898 | Heat stack user project           | True    |
+----------------------------------+------------------------------------------------------------------+----------------------------------+-----------------------------------+---------+

Where are you experiencing the behavior?  What environment?

OSP 13.0.8.
Tempest installed on the director and the tests are executed on the overcloud.

(engcloud) [stack@director-eng eng-tempest]$ rpm -qa | grep tempest
python2-neutron-tests-tempest-0.0.1-0.20180425142847.02a5e2b.el7ost.noarch
python2-keystone-tests-tempest-0.0.1-1.0.564670bgit.el7ost.noarch
python2-tempest-18.0.0-10.el7ost.noarch
python2-heat-tests-tempest-0.3.0-1.el7ost.noarch
python2-novajoin-tests-tempest-0.0.1-0.20180219103837.59d7ba6.el7ost.noarch
python2-cinder-tests-tempest-0.0.1-1.0.aafbdf8git.el7ost.noarch
python2-horizon-tests-tempest-0.0.1-0.20180219094157.a23f407.el7ost.noarch
openstack-tempest-18.0.0-10.el7ost.noarch
python2-tempestconf-2.2.0-1.el7ost.noarch

When does the behavior occur? Frequently?  Repeatedly?   At certain times?
Repeatedly

Comment 5 Martin Kopec 2019-09-30 21:10:13 UTC
*** Bug 1747513 has been marked as a duplicate of this bug. ***

Comment 7 Martin Kopec 2019-10-01 17:29:29 UTC
Info how to reproduce the issue is described in the upstream bug: 
https://bugs.launchpad.net/tempest/+bug/1766582/comments/1

The exact reason why and a way how to fix this are still being investigated.

Comment 15 Martin Kopec 2020-02-11 09:04:51 UTC
Created attachment 1662407 [details]
verification output

The 'Fixed in version' contains the fix for the issue, verified on a 13 env using 2020-02-06.2 puddle. Verification output attached.

Comment 17 errata-xmlrpc 2020-03-10 11:52:51 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0769


Note You need to log in before you can comment on or make changes to this bug.