Bug 1488095

Summary: [RFE] Containerize Tempest
Product: Red Hat OpenStack Reporter: Chandan Kumar <chkumar>
Component: openstack-containersAssignee: Chandan Kumar <chkumar>
Status: CLOSED ERRATA QA Contact: Martin Kopec <mkopec>
Severity: high Docs Contact:
Priority: high    
Version: 13.0 (Queens)CC: apevec, chkumar, jamsmith, jschluet, lhh, m.andre, matyoung, mburns, mmagr, rhallise, sclewis, slinaber, whayutin
Target Milestone: Upstream M3Keywords: FutureFeature, Triaged
Target Release: 13.0 (Queens)Flags: mkopec: needinfo-
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-tempest-container-13.0-18 Doc Type: Technology Preview
Doc Text:
From RHOS-12 onwards, the OpenStack services are becoming containerized. In this release, we containerize OpenStack Tempest as well. The containerized OpenStack Tempest is available as a Technology Preview.
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-06-28 08:00:49 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1574931    
Bug Blocks:    

Description Chandan Kumar 2017-09-04 10:19:45 UTC
From RHOS-12 onwards, all the OpenStack services are getting containerized. In order to move to containerized OpenStack distribution, we need to containerize OpenStack tempest also.

Tempest is a set of integration tests to be run against a live OpenStack cluster. Tempest has batteries of tests for OpenStack API validation, Scenarios, and other specific tests useful in validating an OpenStack deployment.

Tempest is an application which run from undercloud to validate overcloud cloud. Apart from OpenStack core services, other services are validated by using their respective tempest plugins.

Expected Outcome:
* Containerized tempest is available on Undercloud.
* Enduser can run tempest from containerized against overcloud from undercloud in order to validate the overcloud.
* Containerized tempest should expose the complete tempest cli on undercloud.

Challenges:
* How to containerize tempest plugins with respect to containerized Tempest?

Comment 9 wes hayutin 2018-01-08 15:36:56 UTC
Chandan,
Are there any additional rpm deps for tempest when you move to a container?

Comment 10 Chandan Kumar 2018-01-09 12:14:54 UTC
Hello Wes,

New dependency python-stestr will be included as a Requires for Tempest as a part of Queens Release.
Here is the tracker bug: https://bugzilla.redhat.com/show_bug.cgi?id=1507001

Comment 14 Chandan Kumar 2018-05-04 11:26:54 UTC
We are keeping tempest container in tech preview as it is dependent on containerized undercloud which is coming in RHOS-14.
Here is the doc instructions on how to use it: https://blogs.rdoproject.org/2018/05/consuming-kolla-tempest-container-image-for-running-tempest-tests/

And moving the bz to Modified

Comment 17 Martin Kopec 2018-06-07 16:40:29 UTC
Expected Outcome:
* Containerized tempest is available on Undercloud.
It's not available on Undercloud, it's in tech preview, but it can be pulled, for example I was able to pull it from:

registry.access.redhat.com/rhosp13-beta/openstack-tempest

So we could consider this point as done.


* Enduser can run tempest from containerized against overcloud from undercloud in order to validate the overcloud.
* Containerized tempest should expose the complete tempest cli on undercloud.
For the two expected outcomes above, there is no documentation for that for RHOS 13 [1], for example in comparison in RHOS12 [2]. How should a user know how to run it? How should a user know that the tempest cli is exposed completely?
There is only a link to a blog [3], which explains that, but in my opinion it's not enough.

[1] https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/13/html-single/openstack_integration_test_suite_guide
[2] https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/12/html-single/openstack_integration_test_suite_guide/
[3] https://blogs.rdoproject.org/2018/05/consuming-kolla-tempest-container-image-for-running-tempest-tests/

Chandan, in our discussion on IRC, you said, there is a bug for the missing documentation, can you add the link for that here?
I'm not sure if we can mark this as verified without having an official documentation.

Comment 18 wes hayutin 2018-06-18 18:44:04 UTC
Martin, 
There is a pending review for updated tempest doc [1].  Please assist in getting this through to merge.

[1] https://review.openstack.org/#/c/565161/

Comment 21 wes hayutin 2018-06-25 16:12:11 UTC
The steps should not be specific to Tripleo-Quickstart at all.
Steps should be outlined and documented for executing tempest manually which in fact is generated by TripleO-Quickstart but the tempest doc should not reference TripleO-Quickstart at all.

Comment 22 wes hayutin 2018-06-25 16:45:20 UTC
The proper documentation for containerized tempest is not correct in the review however the doc is avalable here [1]


[1] http://logs.openstack.org/32/576632/2/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/d7345a3/logs/docs/build/tempest-setup.html

Expecting to see updates to the review, to have QE confirm the steps and the documentation is of quality for tech preview.

Comment 23 Martin Kopec 2018-06-26 19:47:47 UTC
(In reply to Chandan Kumar from comment #0)
> Expected Outcome:
> * Containerized tempest is available on Undercloud.
As I mentioned before, the containerized tempest is not available on undercloud,
but it can be pulled:
   $ docker pull registry.access.redhat.com/rhosp13-beta/openstack-tempest

> * Enduser can run tempest from containerized against overcloud from
> undercloud in order to validate the overcloud.
This is verified, after the image is pulled a user can follow the documentation and run tempest from the container against overcloud or undercloud. I'm in touch with documentation team, so the documentation regarding to this will be available shortly after the release.

> * Containerized tempest should expose the complete tempest cli on undercloud.
This one is verified too. Tempest is installed inside the container, therefor a user can use all CLI arguments as if the tempest was not in a container.

This was tested on a latest RHOS13 with core puddle = 2018-06-21.2.

Just a note about challenges mentioned in the description:
> * How to containerize tempest plugins with respect to containerized Tempest?
The container available on registry.access.redhat.com/rhosp13-beta/openstack-tempest doesn't contain any Tempest plugins.

Comment 25 errata-xmlrpc 2018-06-28 08:00:49 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:2085