Bug 1480273

Summary: neutron.tests.tempest.api.test_subnetpools fail
Product: Red Hat OpenStack Reporter: Arie Bregman <abregman>
Component: openstack-neutronAssignee: Jakub Libosvar <jlibosva>
Status: CLOSED ERRATA QA Contact: Toni Freger <tfreger>
Severity: high Docs Contact:
Priority: high    
Version: 12.0 (Pike)CC: amuller, chrisw, ihrachys, jlibosva, nyechiel, srevivo
Target Milestone: betaKeywords: AutomationBlocker, Triaged
Target Release: 12.0 (Pike)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-neutron-11.0.1-0.20170831212231.d6f8c44.el7ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-12-13 21:51:30 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Arie Bregman 2017-08-10 14:49:58 UTC
Description of problem:

The following two tests fail on OSP 12 HA:

neutron.tests.tempest.api.test_subnetpools.DefaultSubnetPoolsTest.test_convert_default_subnetpool_to_non_default
neutron.tests.tempest.api.test_subnetpools.DefaultSubnetPoolsTest.test_convert_subnetpool_to_default_subnetpool

Version-Release number of selected component (if applicable):

openstack-neutron-11.0.0-0.20170804190460.el7ost.noarch

How reproducible: 100%

Steps to Reproduce:
1. Deploy OSP 12 HA
2. Run neutron API tests

Actual results:

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/tempest/test.py", line 122, in wrapper
    return func(*func_args, **func_kwargs)
  File "/usr/lib/python2.7/site-packages/neutron/tests/tempest/api/test_subnetpools.py", line 386, in test_convert_default_subnetpool_to_non_default
    is_default=True)
  File "/usr/lib/python2.7/site-packages/neutron/tests/tempest/api/test_subnetpools.py", line 49, in _create_subnetpool
    return cls.create_subnetpool(name=name, is_admin=is_admin, **kwargs)
  File "/usr/lib/python2.7/site-packages/neutron/tests/tempest/api/base.py", line 417, in create_subnetpool
    body = cls.admin_client.create_subnetpool(name, **kwargs)
  File "/usr/lib/python2.7/site-packages/neutron/tests/tempest/services/network/json/network_client.py", line 190, in create_subnetpool
    resp, body = self.post(uri, body)
  File "/usr/lib/python2.7/site-packages/tempest/lib/common/rest_client.py", line 270, in post
    return self.request('POST', url, extra_headers, headers, body, chunked)
  File "/usr/lib/python2.7/site-packages/tempest/lib/common/rest_client.py", line 659, in request
    self._error_checker(resp, resp_body)
  File "/usr/lib/python2.7/site-packages/tempest/lib/common/rest_client.py", line 770, in _error_checker
    raise exceptions.BadRequest(resp_body, resp=resp)
tempest.lib.exceptions.BadRequest: Bad request
Details: {u'message': u'Invalid input for operation: A default subnetpool for this IP family has already been set. Only one default may exist per IP family.', u'type': u'InvalidInput', u'detail': u''}



Expected results: All tests passed successfully

Comment 2 Jakub Libosvar 2017-08-10 16:26:53 UTC
This sounds like there already exists a default subnet. The test has a protection against this and in case the subnet is present, test is skipped (like it's done in upstream gate).

Comment 3 Jakub Libosvar 2017-08-10 16:43:50 UTC
The test is racy, as default subnet can exist only once in cloud and there are two tests that create default and one that updates to default, it happens that tests are running in parallel while the check for existing subnet is at the class level. So it happens that:

 1) class checks for default subnet, it's not there
 3) test1 creates default subnet -> it's fine, we now have our unique resource
 4) test2 creates default subnet -> the error we see cause test1 has the default

From the tempest logs:

Step1:
2017-08-10 07:03:12.341 3008 INFO tempest.lib.common.rest_client [req-07271c2f-6725-4f77-b676-a55a95adbf7b ] Request (DefaultSubnetPoolsTest:setUpClass): 200 GET http://10.0.0.103:9696/v2.0/subnetpools 0.418s
ion/json', 'X-Auth-Token': '<omitted>'}
        Body: None
    Response - Headers: {'status': '200', u'content-length': '18', 'content-location': 'http://10.0.0.103:9696/v2.0/subnetpools', u'date': 'Thu, 10 Aug 2017 11:03:12 GMT', u'content-type': 'application/json', u'connection': 'close', u'x-openstack-request-id': 'req-07271c2f-6725-4f77-b676-a55a95adbf7b'}
        Body: {"subnetpools":[]} _log_request_full /usr/lib/python2.7/site-packages/tempest/lib/common/rest_client.py:425

Step2:
2017-08-10 07:03:12.998 3008 INFO tempest.lib.common.rest_client [req-6322524f-d1d8-4c7c-abd4-f08862bcec60 ] Request (DefaultSubnetPoolsTest:test_admin_create_default_subnetpool): 201 POST http://10.0.0.103:9696/v2.0/subnetpools 0.655s
2017-08-10 07:03:12.998 3008 DEBUG tempest.lib.common.rest_client [req-6322524f-d1d8-4c7c-abd4-f08862bcec60 ] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
        Body: {"subnetpool": {"is_default": true, "prefixes": ["10.11.12.0/24"], "name": "tempest-smoke-subnetpool-2026337716", "min_prefixlen": "29"}}
    Response - Headers: {'status': '201', u'content-length': '508', 'content-location': 'http://10.0.0.103:9696/v2.0/subnetpools', u'date': 'Thu, 10 Aug 2017 11:03:12 GMT', u'content-type': 'application/json', u'connection': 'close', u'x-openstack-request-id': 'req-6322524f-d1d8-4c7c-abd4-f08862bcec60'}
        Body: {"subnetpool":{"is_default":true,"description":"","default_quota":null,"tenant_id":"542c5acbca3f49a0bc89d0903eb5c7e5","created_at":"2017-08-10T11:03:12Z","tags":[],"updated_at":"2017-08-10T11:03:12Z","prefixes":["10.11.12.0/24"],"min_prefixlen":"29","max_prefixlen":"32","address_scope_id":null,"revision_number":0,"ip_version":4,"shared":false,"default_prefixlen":"29","project_id":"542c5acbca3f49a0bc89d0903eb5c7e5","id":"dd1b15f4-0dc1-4582-9435-394a5b2bdea9","name":"tempest-smoke-subnetpool-2026337716"}} _log_request_full /usr/lib/python2.7/site-packages/tempest/lib/common/rest_client.py:425

Step3:
2017-08-10 07:03:15.667 3008 INFO tempest.lib.common.rest_client [req-24e48bfe-473f-44e0-aaf4-8f2debf81a0e ] Request (DefaultSubnetPoolsTest:test_convert_subnetpool_to_default_subnetpool): 400 PUT http://10.0.0.103:9696/v2.0/subnetpools/fb199e24-a9e2-443f-81cc-3c07c3bd7a20 0.842s
2017-08-10 07:03:15.668 3008 DEBUG tempest.lib.common.rest_client [req-24e48bfe-473f-44e0-aaf4-8f2debf81a0e ] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
        Body: {"subnetpool": {"is_default": true}}
    Response - Headers: {'status': '400', u'content-length': '203', 'content-location': 'http://10.0.0.103:9696/v2.0/subnetpools/fb199e24-a9e2-443f-81cc-3c07c3bd7a20', u'date': 'Thu, 10 Aug 2017 11:03:15 GMT', u'content-type': 'application/json', u'connection': 'close', u'x-openstack-request-id': 'req-24e48bfe-473f-44e0-aaf4-8f2debf81a0e'}
        Body: {"NeutronError": {"message": "Invalid input for operation: A default subnetpool for this IP family has already been set. Only one default may exist per IP family.", "type": "InvalidInput", "detail": ""}} _log_request_full /usr/lib/python2.7/site-packages/tempest/lib/common/rest_client.py:425

Comment 9 errata-xmlrpc 2017-12-13 21:51:30 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:3462