Bug 1239110 - The overcloud _validate script can't take a skip list as option
Summary: The overcloud _validate script can't take a skip list as option
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: python-rdomanager-oscplugin
Version: Director
Hardware: Unspecified
OS: Unspecified
high
urgent
Target Milestone: ga
: Director
Assignee: RHOS Maint
QA Contact: Udi
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-07-03 14:01 UTC by mathieu bultel
Modified: 2015-08-05 13:58 UTC (History)
9 users (show)

Fixed In Version: python-rdomanager-oscplugin-0.0.8-24.el7ost
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-08-05 13:58:25 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Gerrithub.io 238442 None None None Never
Red Hat Product Errata RHEA-2015:1549 normal SHIPPED_LIVE Red Hat Enterprise Linux OpenStack Platform director Release 2015-08-05 17:49:10 UTC

Description mathieu bultel 2015-07-03 14:01:03 UTC
Description of problem:

overcloud_validate script can only take as option tempest_args. It should also take a skip list to be able to run only specific tests and blacklist some bugging tests.

See patch here : https://review.gerrithub.io/#/c/238442/

Comment 5 Udi 2015-07-27 06:58:36 UTC
Please provide step-by-step instructions on how to create the skipfile and use it in a tempest run. 

I tried to run the validation like this:
openstack overcloud validate --skipfile ~/skipfile --overcloud-auth-url $OS_AUTH_URL --overcloud-admin-password $OS_PASSWORD

I got an exception and "The test run didn't actually run any tests":

2015-07-27 02:55:19.593 1704 INFO __main__ [-] Downloading image 48e1c4f2-593c-41f2-84cb-2b52a6b3fc1e to /home/stack/tempest/etc/cirros-0.3.1-x86_64-disk.img
2015-07-27 02:55:19.750 1704 CRITICAL tempest [-] InvalidHttpSuccessCode: The success code is different than the expected one
Details: Unexpected http success status code 204,
                         The expected status code is 200
2015-07-27 02:55:19.750 1704 TRACE tempest Traceback (most recent call last):
2015-07-27 02:55:19.750 1704 TRACE tempest   File "./tools/config_tempest.py", line 726, in <module>
2015-07-27 02:55:19.750 1704 TRACE tempest     main()
2015-07-27 02:55:19.750 1704 TRACE tempest   File "./tools/config_tempest.py", line 145, in main
2015-07-27 02:55:19.750 1704 TRACE tempest     args.image_disk_format)
2015-07-27 02:55:19.750 1704 TRACE tempest   File "./tools/config_tempest.py", line 521, in create_tempest_images
2015-07-27 02:55:19.750 1704 TRACE tempest     disk_format=disk_format)
2015-07-27 02:55:19.750 1704 TRACE tempest   File "./tools/config_tempest.py", line 547, in find_or_upload_image
2015-07-27 02:55:19.750 1704 TRACE tempest     _download_image(client, image['id'], path)
2015-07-27 02:55:19.750 1704 TRACE tempest   File "./tools/config_tempest.py", line 683, in _download_image
2015-07-27 02:55:19.750 1704 TRACE tempest     body = client.get_image_file(id)
2015-07-27 02:55:19.750 1704 TRACE tempest   File "/home/stack/tempest/tempest/services/image/v2/json/image_client.py", line 147, in get_image_file
2015-07-27 02:55:19.750 1704 TRACE tempest     self.expected_success(200, resp.status)
2015-07-27 02:55:19.750 1704 TRACE tempest   File "/usr/lib/python2.7/site-packages/tempest_lib/common/rest_client.py", line 236, in expected_success
2015-07-27 02:55:19.750 1704 TRACE tempest     raise exceptions.InvalidHttpSuccessCode(details)
2015-07-27 02:55:19.750 1704 TRACE tempest InvalidHttpSuccessCode: The success code is different than the expected one
2015-07-27 02:55:19.750 1704 TRACE tempest Details: Unexpected http success status code 204,
2015-07-27 02:55:19.750 1704 TRACE tempest                          The expected status code is 200
2015-07-27 02:55:19.750 1704 TRACE tempest 
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \
${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./tempest/test_discover} --list 
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \
${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./tempest/test_discover} --list 
The test run didn't actually run any tests

I also tried the same command without any --skipfile option and all the tests threw exceptions and 0 passed... What am I doing wrong?

Comment 6 mathieu bultel 2015-07-27 07:18:42 UTC
You must provide a file with this syntax :

+tempest.api.identity
-tempest.api.network.admin.test_external_network_extension.ExternalNetworksTestJSON.test_delete_external_networks_with_floating_ip
-tempest.api.image.v1.test_images.CreateRegisterImagesTest.test_register_remote_image


+ for test to white list
- for test to black list

You could provide a command like this :

--tempest-args tempest.api.network --skipefile /home/my/skipfile

with the line :
-tempest.api.network.admin.test_external_network_extension.ExternalNetworksTestJSON.test_delete_external_networks_with_floating_ip

It will play all tempest.api.network without the test in the skip file.

Comment 7 Alexander Chuzhoy 2015-07-30 21:01:07 UTC
Verified:

Environment:
instack-undercloud-2.1.2-22.el7ost.noarch
python-rdomanager-oscplugin-0.0.8-43.el7ost.noarch

Create a skipfile in /home/stack/skipfile.

Started validation with:

openstack overcloud validate --skipfile /home/stack/skipfile --overcloud-auth-url $OS_AUTH_URL --overcloud-admin-password $OS_PASSWORD


The skip list in the file was skipped.

Comment 9 errata-xmlrpc 2015-08-05 13:58:25 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2015:1549


Note You need to log in before you can comment on or make changes to this bug.