Bug 1728151

Summary: tempest cleanup --init-saved-state still requests to Keystone v2 API
Product: Red Hat OpenStack Reporter: Takashi Kajinami <tkajinam>
Component: openstack-tempestAssignee: Martin Kopec <mkopec>
Status: CLOSED ERRATA QA Contact: Lukas Piwowarski <lpiwowar>
Severity: high Docs Contact:
Priority: high    
Version: 13.0 (Queens)CC: apevec, lhh, mkopec, sknauss, slinaber, udesale
Target Milestone: z8Keywords: Triaged, ZStream
Target Release: 13.0 (Queens)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-tempest-18.0.0-10.el7ost Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of:
: 1731828 (view as bug list) Environment:
Last Closed: 2019-09-03 16:58:11 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1731828, 1731829    
Attachments:
Description Flags
tempest.log where I see the problem is reproduced
none
bug-1728151.log none

Description Takashi Kajinami 2019-07-09 07:36:32 UTC
Description of problem:

When we run
 $ tempest cleanup --init-saved-state
according to the product documentation[1], we see that it is requesting
Keystone v2 api to list role, which is ignored inside command.

[1] https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/13/html/openstack_integration_test_suite_guide/chap-clean-tempest-resources

We see the following ERROR in tempest.log

~~~
2019-07-09 02:38:29.955 32177 INFO tempest.lib.common.rest_client [req-e88b41a5-7811-4fb3-85e7-9e41ac6412b5 ] Request (main): 404 GET http://192.168.24.7:35357/v2.0/OS-KSADM/roles 0.138s
2019-07-09 02:38:29.955 32177 DEBUG tempest.lib.common.rest_client [req-e88b41a5-7811-4fb3-85e7-9e41ac6412b5 ] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token'
: '<omitted>'}
        Body: None
    Response - Headers: {'status': '404', u'content-length': '142', 'content-location': 'http://192.168.24.7:35357/v2.0/OS-KSADM/roles', u'vary': 'X-Auth-Token', u'server': 'Apache', u'connection': 'close', u'da
te': 'Tue, 09 Jul 2019 06:38:29 GMT', u'content-type': 'application/json', u'x-openstack-request-id': 'req-e88b41a5-7811-4fb3-85e7-9e41ac6412b5'}
        Body: {"error": {"message": "(http://192.168.24.7:35357/v2.0/OS-KSADM/roles): The resource could not be found.", "code": 404, "title": "Not Found"}} _log_request_full /usr/lib/python2.7/site-packages/tem
pest/lib/common/rest_client.py:434
2019-07-09 02:38:29.956 32177 ERROR tempest.cmd.cleanup_service [-] Cannot retrieve Roles.: NotFound: Object not found
Details: {u'message': u'(http://192.168.24.7:35357/v2.0/OS-KSADM/roles): The resource could not be found.', u'code': 404, u'title': u'Not Found'}
2019-07-09 02:38:29.956 32177 ERROR tempest.cmd.cleanup_service Traceback (most recent call last):
2019-07-09 02:38:29.956 32177 ERROR tempest.cmd.cleanup_service   File "/usr/lib/python2.7/site-packages/tempest/cmd/cleanup_service.py", line 839, in list
2019-07-09 02:38:29.956 32177 ERROR tempest.cmd.cleanup_service     roles = self.client.list_roles()['roles']
2019-07-09 02:38:29.956 32177 ERROR tempest.cmd.cleanup_service   File "/usr/lib/python2.7/site-packages/tempest/lib/services/identity/v2/roles_client.py", line 59, in list_roles
2019-07-09 02:38:29.956 32177 ERROR tempest.cmd.cleanup_service     resp, body = self.get(url)
2019-07-09 02:38:29.956 32177 ERROR tempest.cmd.cleanup_service   File "/usr/lib/python2.7/site-packages/tempest/lib/common/rest_client.py", line 294, in get
2019-07-09 02:38:29.956 32177 ERROR tempest.cmd.cleanup_service     return self.request('GET', url, extra_headers, headers)
2019-07-09 02:38:29.956 32177 ERROR tempest.cmd.cleanup_service   File "/usr/lib/python2.7/site-packages/tempest/lib/common/rest_client.py", line 668, in request
2019-07-09 02:38:29.956 32177 ERROR tempest.cmd.cleanup_service     self._error_checker(resp, resp_body)
2019-07-09 02:38:29.956 32177 ERROR tempest.cmd.cleanup_service   File "/usr/lib/python2.7/site-packages/tempest/lib/common/rest_client.py", line 774, in _error_checker
2019-07-09 02:38:29.956 32177 ERROR tempest.cmd.cleanup_service     raise exceptions.NotFound(resp_body, resp=resp)
2019-07-09 02:38:29.956 32177 ERROR tempest.cmd.cleanup_service NotFound: Object not found
2019-07-09 02:38:29.956 32177 ERROR tempest.cmd.cleanup_service Details: {u'message': u'(http://192.168.24.7:35357/v2.0/OS-KSADM/roles): The resource could not be found.', u'code': 404, u'title': u'Not Found'}
2019-07-09 02:38:29.956 32177 ERROR tempest.cmd.cleanup_service
~~~

Because of this I'm afraid currently tempest cleanup does not deal with Keystone at all.

Version-Release number of selected component (if applicable):

$ rpm -qa | grep tempest
python2-tempestconf-2.0.0-1.el7ost.noarch
openstack-tempest-18.0.0-7.el7ost.noarch
python2-tempest-18.0.0-7.el7ost.noarch

How reproducible:
Always

Steps to Reproduce:
1. setup tempest according to product doc
2. run "tempest cleanup --init-saved-state "
3. see tempest.log

Actual results:
We see ERROR caused by access to Keystone v2 API

Expected results:
We do not see the ERROR and tempest cleanup command succeeds to list roles

Additional info:

Comment 2 Takashi Kajinami 2019-07-09 07:40:17 UTC
Created attachment 1588635 [details]
tempest.log where I see the problem is reproduced

Comment 11 Lukas Piwowarski 2019-08-20 14:48:18 UTC
Created attachment 1606110 [details]
bug-1728151.log

Comment 12 Lukas Piwowarski 2019-08-20 14:54:20 UTC
Tested with:
  - package with the bug (openstack-tempest-18.0.0-9.el7ost)
  - package with the fix (openstack-tempest-18.0.0-10.el7ost)

The bug was not reproduced when tested with the package with the fix as shown in the attached log file (bug-1731829.log).

Comment 13 Lukas Piwowarski 2019-08-20 14:55:55 UTC
Tested with:
  - package with the bug (openstack-tempest-18.0.0-9.el7ost)
  - package with the fix (openstack-tempest-18.0.0-10.el7ost)

The bug was not reproduced when tested with the package with the fix as shown in the attached log file (bug-1728151.log).

Comment 15 errata-xmlrpc 2019-09-03 16:58:11 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2623