Bug 1239110

Please provide step-by-step instructions on how to create the skipfile and use it in a tempest run. 

I tried to run the validation like this:
openstack overcloud validate --skipfile ~/skipfile --overcloud-auth-url $OS_AUTH_URL --overcloud-admin-password $OS_PASSWORD

I got an exception and "The test run didn't actually run any tests":

2015-07-27 02:55:19.593 1704 INFO __main__ [-] Downloading image 48e1c4f2-593c-41f2-84cb-2b52a6b3fc1e to /home/stack/tempest/etc/cirros-0.3.1-x86_64-disk.img
2015-07-27 02:55:19.750 1704 CRITICAL tempest [-] InvalidHttpSuccessCode: The success code is different than the expected one
Details: Unexpected http success status code 204,
                         The expected status code is 200
2015-07-27 02:55:19.750 1704 TRACE tempest Traceback (most recent call last):
2015-07-27 02:55:19.750 1704 TRACE tempest   File "./tools/config_tempest.py", line 726, in <module>
2015-07-27 02:55:19.750 1704 TRACE tempest     main()
2015-07-27 02:55:19.750 1704 TRACE tempest   File "./tools/config_tempest.py", line 145, in main
2015-07-27 02:55:19.750 1704 TRACE tempest     args.image_disk_format)
2015-07-27 02:55:19.750 1704 TRACE tempest   File "./tools/config_tempest.py", line 521, in create_tempest_images
2015-07-27 02:55:19.750 1704 TRACE tempest     disk_format=disk_format)
2015-07-27 02:55:19.750 1704 TRACE tempest   File "./tools/config_tempest.py", line 547, in find_or_upload_image
2015-07-27 02:55:19.750 1704 TRACE tempest     _download_image(client, image['id'], path)
2015-07-27 02:55:19.750 1704 TRACE tempest   File "./tools/config_tempest.py", line 683, in _download_image
2015-07-27 02:55:19.750 1704 TRACE tempest     body = client.get_image_file(id)
2015-07-27 02:55:19.750 1704 TRACE tempest   File "/home/stack/tempest/tempest/services/image/v2/json/image_client.py", line 147, in get_image_file
2015-07-27 02:55:19.750 1704 TRACE tempest     self.expected_success(200, resp.status)
2015-07-27 02:55:19.750 1704 TRACE tempest   File "/usr/lib/python2.7/site-packages/tempest_lib/common/rest_client.py", line 236, in expected_success
2015-07-27 02:55:19.750 1704 TRACE tempest     raise exceptions.InvalidHttpSuccessCode(details)
2015-07-27 02:55:19.750 1704 TRACE tempest InvalidHttpSuccessCode: The success code is different than the expected one
2015-07-27 02:55:19.750 1704 TRACE tempest Details: Unexpected http success status code 204,
2015-07-27 02:55:19.750 1704 TRACE tempest                          The expected status code is 200
2015-07-27 02:55:19.750 1704 TRACE tempest 
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \
${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./tempest/test_discover} --list 
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \
${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./tempest/test_discover} --list 
The test run didn't actually run any tests

I also tried the same command without any --skipfile option and all the tests threw exceptions and 0 passed... What am I doing wrong?

You must provide a file with this syntax :

+tempest.api.identity
-tempest.api.network.admin.test_external_network_extension.ExternalNetworksTestJSON.test_delete_external_networks_with_floating_ip
-tempest.api.image.v1.test_images.CreateRegisterImagesTest.test_register_remote_image


+ for test to white list
- for test to black list

You could provide a command like this :

--tempest-args tempest.api.network --skipefile /home/my/skipfile

with the line :
-tempest.api.network.admin.test_external_network_extension.ExternalNetworksTestJSON.test_delete_external_networks_with_floating_ip

It will play all tempest.api.network without the test in the skip file.

Verified:

Environment:
instack-undercloud-2.1.2-22.el7ost.noarch
python-rdomanager-oscplugin-0.0.8-43.el7ost.noarch

Create a skipfile in /home/stack/skipfile.

Started validation with:

openstack overcloud validate --skipfile /home/stack/skipfile --overcloud-auth-url $OS_AUTH_URL --overcloud-admin-password $OS_PASSWORD


The skip list in the file was skipped.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2015:1549