Bug 1230840 - Tempest needs additional configuration to run successfully
Summary: Tempest needs additional configuration to run successfully
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: rhosp-director
Version: 7.0 (Kilo)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ga
: Director
Assignee: Imre Farkas
QA Contact: yfried
URL:
Whiteboard:
Depends On:
Blocks: 1249640
TreeView+ depends on / blocked
 
Reported: 2015-06-11 15:28 UTC by David Kranz
Modified: 2016-09-20 05:03 UTC (History)
10 users (show)

Fixed In Version: python-rdomanager-oscplugin-0.0.8-31.el7ost
Doc Type: Bug Fix
Doc Text:
Previously, deployment specific values were not provided in OpenStack Integration Test Suite (tempest), resulting in the failure of some tempest tests. With this update, a '--deployer-input' flag is added for the 'openstack overcloud validate' command so the administrator can provide a file (tempest.conf) containing the deployment specific values. As a result of using the '--deployer-input filename' flag, fewer tests result in failure.
Clone Of:
: 1249640 (view as bug list)
Environment:
Last Closed: 2015-08-05 13:53:19 UTC
Target Upstream Version:


Attachments (Terms of Use)
output of heat stack-show overcloud (31.92 KB, text/plain)
2015-06-23 11:31 UTC, Imre Farkas
no flags Details


Links
System ID Priority Status Summary Last Updated
Gerrithub.io 239395 None None None Never
Red Hat Product Errata RHEA-2015:1549 normal SHIPPED_LIVE Red Hat Enterprise Linux OpenStack Platform director Release 2015-08-05 17:49:10 UTC

Description David Kranz 2015-06-11 15:28:17 UTC
These comments apply to any installer we have or will have. Tempest has a number of configuration options (listed below) for features whose presence can not be discovered through apis. When the installer configures tempest, it must specify these values in tempest.conf if they are different from the defaults. The installer should write the required values to a file in tempest.conf format and then pass it to the config_tempest script using the --deployer-input command line argument. This file then needs to be "exported" so that users can do the same to run tempest against the cloud after installation. 

The specific values needed will change from release to release as new features are added to OpenStack. Also, in the future when a cloud is upgraded to a new major rhel osp version, new values will have to be supplied and a new partial tempest.conf exported.

Most, but not all, of these values appear in tempest.conf sections with the format [<service-name>-feature-enabled]

Current values and defaults as of kilo, from etc/tempest.conf.sample in the tempest repo:

[compute-feature-enabled]
# Does the test environment support changing the admin password?
# (boolean value)
#change_password = false

# Does the test environment support obtaining instance serial console
# output? (boolean value)
#console_output = true

# Does the test environment support resizing? (boolean value)
#resize = false

# Does the test environment support pausing? (boolean value)
#pause = true

# Does the test environment support shelving/unshelving? (boolean
# value)
#shelve = true

# Does the test environment support suspend/resume? (boolean value)
#suspend = true

# Does the test environment support live migration available? (boolean
# value)
#live_migration = true

# Does the test environment use block devices for live migration
# (boolean value)
#block_migration_for_live_migration = false

# Does the test environment block migration support cinder iSCSI
# volumes. Note, libvirt doesn't support this, see
# https://bugs.launchpad.net/nova/+bug/1398999 (boolean value)
#block_migrate_cinder_iscsi = false

# Does the test system allow live-migration of paused instances? Note,
# this is more than just the ANDing of paused and live_migrate, but
# all 3 should be set to True to run those tests (boolean value)
#live_migrate_paused_instances = false

# Enable VNC console. This configuration value should be same as
# [nova.vnc]->vnc_enabled in nova.conf (boolean value)
#vnc_console = false

# Enable Spice console. This configuration value should be same as
# [nova.spice]->enabled in nova.conf (boolean value)
#spice_console = false

# Enable RDP console. This configuration value should be same as
# [nova.rdp]->enabled in nova.conf (boolean value)
#rdp_console = false

# Does the test environment support instance rescue mode? (boolean
# value)
#rescue = true

# Enables returning of the instance password by the relevant server
# API calls such as create, rebuild or rescue. (boolean value)
#enable_instance_password = true

# Does the test environment support dynamic network interface
# attachment? (boolean value)
#interface_attach = true

# Does the test environment support creating snapshot images of
# running instances? (boolean value)
#snapshot = true

# Does the test environment have the ec2 api running? (boolean value)
#ec2_api = true

# Does Nova preserve preexisting ports from Neutron when deleting an
# instance? This should be set to True if testing Kilo+ Nova. (boolean
# value)
#preserve_ports = false

[identity-feature-enabled]

# Does the identity service have delegation and impersonation enabled
# (boolean value)
#trust = true

# Is the v2 identity API enabled (boolean value)
#api_v2 = true

# Is the v3 identity API enabled (boolean value)
#api_v3 = true

[image-feature-enabled]

# Is the v2 image API enabled (boolean value)
#api_v2 = true

# Is the v1 image API enabled (boolean value)
#api_v1 = true

[network-feature-enabled]


# Allow the execution of IPv6 tests (boolean value)
#ipv6 = true

# A list of enabled network extensions with a special entry all which
# indicates every extension is enabled. Empty list indicates all
# extensions are disabled (list value)
#api_extensions = all

# Allow the execution of IPv6 subnet tests that use the extended IPv6
# attributes ipv6_ra_mode and ipv6_address_mode (boolean value)
#ipv6_subnet_attributes = false

# Does the test environment support changing port admin state (boolean
# value)
#port_admin_state_change = true

[volume]

# Name of the backend1 (must be declared in cinder.conf) (string
# value)
#backend1_name = BACKEND_1

# Name of the backend2 (must be declared in cinder.conf) (string
# value)
#backend2_name = BACKEND_2

# Backend protocol to target when creating volume types (string value)
#storage_protocol = iSCSI

# Backend vendor to target when creating volume types (string value)
#vendor_name = Open Source


[volume-feature-enabled]

# Runs Cinder multi-backend test (requires 2 backends) (boolean value)
#multi_backend = false

# Runs Cinder volumes backup test (boolean value)
#backup = true

# Runs Cinder volume snapshot test (boolean value)
#snapshot = true

# Is the v1 volume API enabled (boolean value)
#api_v1 = true

# Is the v2 volume API enabled (boolean value)
#api_v2 = true

# Update bootable status of a volume Not implemented on icehouse
# (boolean value)
#bootable = false

Comment 3 David Kranz 2015-06-11 15:58:02 UTC
[object-storage]


# Role to add to users created for swift tests to enable creating
# containers (string value)
#operator_role = Member

# User role that has reseller admin (string value)
#reseller_admin_role = ResellerAdmin

# Name of sync realm. A sync realm is a set of clusters that have
# agreed to allow container syncing with each other. Set the same
# realm name as Swift's container-sync-realms.conf (string value)
#realm_name = realm1

# One name of cluster which is set in the realm whose name is set in
# 'realm_name' item in this file. Set the same cluster name as Swift's
# container-sync-realms.conf (string value)
#cluster_name = name1

Comment 5 Omri Hochman 2015-06-22 18:36:42 UTC
re-targeting to GA :

We're suspecting this bug related to many of the FAILED (failures=245) that we're getting when running tempest as - post-deployment-validation against the overcluod - using the command "openstack overcloud validate " 

Results: 

Traceback (most recent call last):
  File "/home/stack/tempest/tempest/api/network/admin/test_external_network_extension.py", line 106, in test_delete_external_networks_with_floating_ip
    enable_dhcp=False)
  File "/home/stack/tempest/tempest/api/network/base.py", line 240, in create_subnet
    raise exceptions.BuildErrorException(message)
BuildErrorException: Server %(server_id)s failed to build and is in ERROR status
Details: Available CIDR for subnet creation could not be found


Ran 807 tests in 7092.710s

FAILED (failures=245)

Comment 6 David Kranz 2015-06-22 20:46:38 UTC
Sorry, looks like I forgot these. They may not all be necessary.

[network]

# The cidr block to allocate tenant ipv4 subnets from (string value)
#tenant_network_cidr = 10.100.0.0/16

# The mask bits for tenant ipv4 subnets (integer value)
#tenant_network_mask_bits = 28

# The cidr block to allocate tenant ipv6 subnets from (string value)
#tenant_network_v6_cidr = 2003::/48

# The mask bits for tenant ipv6 subnets (integer value)
#tenant_network_v6_mask_bits = 64


# Default floating network name. Used to allocate floating IPs when
# neutron is enabled. (string value)
#floating_network_name = <None>


# List of dns servers which should be used for subnet creation (list
# value)
#dns_servers = 8.8.8.8,8.8.4.4

# vnic_type to use when Launching instances with pre-configured ports.
# Supported ports are: ['normal','direct','macvtap'] (string value)
# Allowed values: <None>, normal, direct, macvtap
#port_vnic_type = <None>

Comment 7 Imre Farkas 2015-06-23 11:29:55 UTC
I tried to figure out the values for each of the config options. None of these are included amongst the parameter we are passing to heat (I added the output of a heat stack-show command, so you can check it). 
I also tried to check different config files on the deployed overcloud with the same result. Eg. even is 'change_password' is listed in the compute-feature-enabled group, it's not included nova.conf.

Could you please comment on each of those options on how/where to check the values on a deployment?

Comment 8 Imre Farkas 2015-06-23 11:31:05 UTC
Created attachment 1042257 [details]
output of heat stack-show overcloud

Comment 9 David Kranz 2015-06-23 17:25:28 UTC
There is not necessarily a name-match between a feature-enabled item in tempest.conf and some option in nova.conf, cinder.conf, etc. For many of these values the defaults will be ok. If the comments in tempest.conf.sample are not sufficient to determine whether a cloud has the feature, the definitive explanation is which tests are skipped based on its absence. In some cases some one who has details of how the service in question is deployed by Director will need to be consulted.

Comment 12 Imre Farkas 2015-07-09 13:11:13 UTC
WIP patch already proposed: https://review.gerrithub.io/#/c/238706/
It generates a partial tempest config and passes it to config_tempest.py which processes it properly.

I still have problems with figuring out what the values should be as mentioned in Comment 6. I sent an email about the network part to the mailing list: http://post-office.corp.redhat.com/archives/openstack-management-team-list/2015-July/msg00292.html

Comment 13 Imre Farkas 2015-07-09 15:26:50 UTC
Considering the implications of #1241583, it might not even make sense to generate the partial tempest configuration as it will depend on how the deployer configures the network. 
As an alternative, we can provide a flag to overcloud validate so a partial tempest config written by the administrator could be passed.

Comment 15 Imre Farkas 2015-07-10 09:02:55 UTC
Patch merged in rdo-management/python-rdomanager-oscplugin: https://github.com/rdo-management/python-rdomanager-oscplugin/commit/b728836ae7fe0766800582728f1afd9b00b92158

Downstream patch proposed: https://code.engineering.redhat.com/gerrit/#/c/52764/

Comment 17 Alexander Stafeyev 2015-07-22 08:37:23 UTC
What is the validation steps ? (David) 

The validation is using the additional flag with the reproduction steps? (Imre)

Comment 18 Alexander Stafeyev 2015-08-02 07:39:20 UTC
Hi, 
I have limited tools for the bug verification so it would be great if you could check the fix. 

tnx

Comment 20 yfried 2015-08-03 07:41:56 UTC
According to David, this bug isn't solved.

Comment 21 Imre Farkas 2015-08-03 08:45:06 UTC
Please consider the scope change for 7.0 as described in Comment 13 and Comment 14

Comment 30 errata-xmlrpc 2015-08-05 13:53:19 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2015:1549


Note You need to log in before you can comment on or make changes to this bug.