Bug 1275107 - Overcloud Tempest Swift tests fail
Summary: Overcloud Tempest Swift tests fail
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: rhosp-director
Version: 7.0 (Kilo)
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: ---
Assignee: Hugh Brock
QA Contact: Shai Revivo
URL:
Whiteboard:
Depends On:
Blocks: 1339488
TreeView+ depends on / blocked
 
Reported: 2015-10-25 22:56 UTC by Pablo Caruana
Modified: 2019-11-14 07:05 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-10-14 00:58:46 UTC
Target Upstream Version:
tshefi: automate_bug+


Attachments (Terms of Use)

Description Pablo Caruana 2015-10-25 22:56:58 UTC
Created attachment 1086309 [details]
tempest.conf

After having used RDOManager (Director) to deploy a testing Overcloud (1 controller node and 1 compute node on KVM), and following advice from Red Hat documentation, when running the Tempest suite of tests to validate the Overcloud by running the following command:

$ source ~/overcloudrc
$ keystone role-create --name heat_stack_owner
$ openstack overcloud validate --overcloud-auth-url $OS_AUTH_URL --overcloud-admin-password $OS_PASSWORD

However, some of the Tempest tests fail. After tracking down some of them to the S3 tests:

$ cd /home/stack/tempest
./tools/run-tests.sh '.*s3.*'

As far as It  can  be read from the source code, the S3 tests rely on the "boto" Python library. The "boto" logging is limited  and after quick hacking the code to figure out why these S3 tests are failing. Turns out the S3 tests are failing because they are trying to hit the S3 API on "localhost:8080" which clearly won't work: the Tempest tests are ignoring Keystone endpoints completely:

$ keystone endpoint-get --service object-store
+------------------------+----------------------------------------------------------------------+
|        Property        |                                Value                                 |
+------------------------+----------------------------------------------------------------------+
| object-store.publicURL | http://192.168.246.100:8080/v1/AUTH_4b2cfc10af4546bb88e078435912d6c2 |
+------------------------+----------------------------------------------------------------------+

From there  Swift S3 API listens on 192.168.246.100:8080 (not localhost).

Potential rca is etc/tempest.conf is not properly configured. This is an extract from etc/tempest.conf.sample in particular for the "boto" configuration:

[boto]

#
# From tempest.config
#

# EC2 URL (string value)
#ec2_url = http://localhost:8773/services/Cloud

# S3 URL (string value)
#s3_url = http://localhost:8080

# AWS Secret Key (string value)
#aws_secret = <None>

It doesn't look a coincidence that S3 tests are using http://localhost:8080 (the assumed default value for s3_url) instead of http://192.168.246.100:8080.

The "overcloud validate" plugin runs the following shell commands to configure and run Tempest:

$ mkdir /home/stack/tempest
$ cd /home/stack/tempest
$ /usr/share/openstack-tempest-kilo/tools/configure-tempest-directory
$ ./tools/config_tempest.py --out etc/tempest.conf --deployer-input /dev/null --debug --create identity.uri http://192.168.246.100:5000/v2.0/ compute.allow_tenant_isolation true object-storage.operator_role SwiftOperator identity.admin_password xxxxxx compute.build_timeout 500 compute.image_ssh_user cirros compute.ssh_user cirros network.build_timeout 500 volume.build_timeout 500 scenario.ssh_user cirros

attached to the BZ the resulting etc/tempest.conf file.

Expected the Swift S3 tests to pass, and have them talk to the correct Swift service (not localhost).

When does the behavior occur? Frequently?  Repeatedly?   At certain times?

Always.

Comment 3 Mike Burns 2016-04-07 20:54:03 UTC
This bug did not make the OSP 8.0 release.  It is being deferred to OSP 10.

Comment 5 Paul Grist 2016-10-14 00:58:46 UTC
This was OSP7 and looks stale. Tempest tests are part of our automation. I'm going to close this out, but adding several folks and please re-open this if it is still a problem.

Comment 6 Tzach Shefi 2018-04-22 12:26:33 UTC
Also as Paul mentioned we now have automation in place testing Swift, both upstream tempest as well as our own extended downstream tempest. 

I'd feel more at easy if I knew which test(s) failed on this bug. 
But as we test newer releases without tests failing
I'm sure pretty confident this has gone away, is tested. 

Setting as QE_automated.


Note You need to log in before you can comment on or make changes to this bug.