Bug 1440897
| Summary: | Frequent failure at Subnet deletion with 3 controller | |||
|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Attila Fazekas <afazekas> | |
| Component: | openstack-neutron | Assignee: | Ihar Hrachyshka <ihrachys> | |
| Status: | CLOSED ERRATA | QA Contact: | Alexander Stafeyev <astafeye> | |
| Severity: | high | Docs Contact: | ||
| Priority: | high | |||
| Version: | 11.0 (Ocata) | CC: | afazekas, amuller, astafeye, chrisw, ihrachys, jlibosva, nyechiel, srevivo | |
| Target Milestone: | rc | Flags: | afazekas:
needinfo-
|
|
| Target Release: | 11.0 (Ocata) | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | openstack-neutron-10.0.0-16.el7ost | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 1440901 1440908 (view as bug list) | Environment: | ||
| Last Closed: | 2017-05-17 20:19:46 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 1440901, 1440908 | |||
This is a known regression in Ocata and earlier tracked in upstream as: https://bugs.launchpad.net/neutron/+bug/1680912 Backport is posted d/s, should be in any day now. Setting as GA blocker. What is that means ?> Which test exaclty? I need reproduction steps please. Attila mentions "25% chance with ostestr -r PortsIpV6TestJSON" in the original bug report, if that passes reliably in a loop then I'd suggest that we're good. openstack-neutron-10.0.1-1.el7ost.noarch Ran the following 4 times - 100% success ostestr -r PortsIpV6TestJSON ============== Worker Balance ============== - Worker 0 (14 tests) => 0:02:00.718520 Slowest Tests: Test id Runtime (s) -------------------------------------------------------------------------------------------------------------------------------------------------------- ----------- tempest.api.network.test_ports.PortsIpV6TestJSON.test_port_list_filter_by_router_id[id-5ad01ed0-0e6e-4c5d-8194-232801b15c72] 27.051 tempest.api.network.test_ports.PortsIpV6TestJSON.test_create_update_port_with_second_ip[id-63aeadd4-3b49-427f-a3b1-19ca81f06270] 20.893 tempest.api.network.test_ports.PortsIpV6TestJSON.test_update_port_with_two_security_groups_and_extra_attributes[id-edf6766d-3d40-4621-bc6e-2521a44c257d] 14.518 tempest.api.network.test_ports.PortsIpV6TestJSON.test_update_port_with_security_group_and_extra_attributes[id-58091b66-4ff4-4cc1-a549-05d60c7acd1a] 12.271 tempest.api.network.test_ports.PortsIpV6TestJSON.test_port_list_filter_by_ip[id-e7fe260b-1e79-4dd3-86d9-bec6a7959fc5] 12.247 tempest.api.network.test_ports.PortsIpV6TestJSON.test_create_port_with_no_securitygroups[id-4179dcb9-1382-4ced-84fe-1b91c54f5735,smoke] 11.316 tempest.api.network.test_ports.PortsIpV6TestJSON.test_create_port_in_allowed_allocation_pools[id-0435f278-40ae-48cb-a404-b8a087bc09b1,smoke] 7.614 tempest.api.network.test_ports.PortsIpV6TestJSON.test_create_bulk_port[id-67f1b811-f8db-43e2-86bd-72c074d4a42c] 7.372 tempest.api.network.test_ports.PortsIpV6TestJSON.test_create_show_delete_port_user_defined_mac[id-13e95171-6cbd-489c-9d7c-3f9c58215c18] 4.071 tempest.api.network.test_ports.PortsIpV6TestJSON.test_create_update_delete_port[id-c72c1c0c-2193-4aca-aaa4-b1442640f51c,smoke] 2.601 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:1245 |
Description of problem: The 4 thread tempest runs frequently fails with subnet deletion. The tempest log contains multiple connection retry to the neutron server: 2017-04-07 02:01:40,246 24529 WARNING [urllib3.connectionpool] Retrying (Retry(total=9, connect=None, read=None, redirect=5)) after connection broken by 'ReadTimeoutError("HTTPConnectionPool(host='10.0.0.110', port=9696): Read timed out. (read timeout=60)",)': /v2.0/subnets/9197a378-4a60-4e60-a0e0-1df71e26a35e The exception at the cleanup phase: Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/tempest/lib/services/network/subnets_client.py", line 52, in delete_subnet return self.delete_resource(uri) File "/usr/lib/python2.7/site-packages/tempest/lib/services/network/base.py", line 41, in delete_resource resp, body = self.delete(req_uri) File "/usr/lib/python2.7/site-packages/tempest/lib/common/rest_client.py", line 307, in delete return self.request('DELETE', url, extra_headers, headers, body) File "/usr/lib/python2.7/site-packages/tempest/lib/common/rest_client.py", line 665, in request self._error_checker(resp, resp_body) File "/usr/lib/python2.7/site-packages/tempest/lib/common/rest_client.py", line 762, in _error_checker raise exceptions.NotFound(resp_body, resp=resp) tempest.lib.exceptions.NotFound: Object not found Details: {u'message': u'Subnet 9197a378-4a60-4e60-a0e0-1df71e26a35e could not be found.', u'type': u'SubnetNotFound', u'detail': u''} Version-Release number of selected component (if applicable): python-neutron-lbaas-10.0.0-5.el7ost.noarch python-neutronclient-6.1.0-1.el7ost.noarch puppet-neutron-10.3.0-2.el7ost.noarch openstack-neutron-common-10.0.0-13.el7ost.noarch openstack-neutron-lbaas-10.0.0-5.el7ost.noarch openstack-neutron-openvswitch-10.0.0-13.el7ost.noarch python-neutron-10.0.0-13.el7ost.noarch openstack-neutron-10.0.0-13.el7ost.noarch openstack-neutron-metering-agent-10.0.0-13.el7ost.noarch python-neutron-lib-1.1.0-1.el7ost.noarch openstack-neutron-sriov-nic-agent-10.0.0-13.el7ost.noarch openstack-neutron-ml2-10.0.0-13.el7ost.noarch How reproducible: It is reproducible even without a full tempest run. 25% chance with ostestr -r PortsIpV6TestJSON