Bug 1390863

Summary: Tempest network scenario tests regressions
Product: Red Hat OpenStack Reporter: Arie Bregman <abregman>
Component: openstack-neutronAssignee: Ihar Hrachyshka <ihrachys>
Status: CLOSED ERRATA QA Contact: Eran Kuris <ekuris>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 10.0 (Newton)CC: amuller, chrisw, ihrachys, jjoyce, nyechiel, oblaut, samccann, srevivo
Target Milestone: z4Keywords: Automation, AutomationBlocker, Triaged, ZStream
Target Release: 10.0 (Newton)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-neutron-9.3.1-3.el7ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1438346 1452884 (view as bug list) Environment:
Last Closed: 2017-09-06 17:17:18 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1438346    

Description Arie Bregman 2016-11-02 06:54:57 UTC 
Description of problem:

Running tempest, the following tests are failing

tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops
 tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_update_instance_port_admin_state
 tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_update_router_admin_state
tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_slaac_from_os
 tempest.scenario.test_network_v6.TestGettingAddress.test_multi_prefix_dhcpv6_stateless
 tempest.scenario.test_network_v6.TestGettingAddress.test_multi_prefix_slaac
tempest.scenario.test_snapshot_pattern.TestSnapshotPattern.test_snapshot_pattern

Version-Release number of selected component (if applicable):
openstack-neutron-9.0.0-1.5.el7ost.noarch
openstack-neutron-bigswitch-agent-7.0.5-0.20161015221623.f83c499.el7ost.noarch
openstack-neutron-bigswitch-lldp-7.0.5-0.20161015221623.f83c499.el7ost.noarch
openstack-neutron-common-9.0.0-1.5.el7ost.noarch
openstack-neutron-lbaas-9.0.0-2.el7ost.noarch
openstack-neutron-metering-agent-9.0.0-1.5.el7ost.noarch
openstack-neutron-ml2-9.0.0-1.5.el7ost.noarch
openstack-neutron-openvswitch-9.0.0-1.5.el7ost.noarch
openstack-neutron-sriov-nic-agent-9.0.0-1.5.el7ost.noarch

How reproducible: 100%


Steps to Reproduce:
1. Deploy RHOSP 10 HA (3 controllers, 2 compute)
2. Run tempest network scenario tests

Actual results: The above tests are failing with this traceback :

Traceback (most recent call last):
  File "/home/stack/tempest/tempest/test.py", line 100, in wrapper
    return f(self, *func_args, **func_kwargs)
  File "/home/stack/tempest/tempest/scenario/test_snapshot_pattern.py", line 71, in test_snapshot_pattern
    private_key=keypair['private_key'])
  File "/home/stack/tempest/tempest/scenario/manager.py", line 604, in get_timestamp
    private_key=private_key)
  File "/home/stack/tempest/tempest/scenario/manager.py", line 323, in get_remote_client
    linux_client.validate_authentication()
  File "/home/stack/tempest/tempest/common/utils/linux/remote_client.py", line 54, in wrapper
    six.reraise(*original_exception)
  File "/home/stack/tempest/tempest/common/utils/linux/remote_client.py", line 35, in wrapper
    return function(self, *args, **kwargs)
  File "/home/stack/tempest/tempest/common/utils/linux/remote_client.py", line 99, in validate_authentication
    self.ssh_client.test_connection_auth()
  File "/home/stack/tempest/tempest/lib/common/ssh.py", line 176, in test_connection_auth
    connection = self._get_ssh_connection()
  File "/home/stack/tempest/tempest/lib/common/ssh.py", line 90, in _get_ssh_connection
    password=self.password)
tempest.lib.exceptions.SSHTimeout: Connection to the 192.0.2.61 via SSH timed out.
User: cirros, Password: None


Expected results: Tests passed 100%


Additional info: Link to the CI job will be provided in next comment
Comment 3 Ihar Hrachyshka 2017-05-18 17:18:10 UTC 
Neutron side workaround applied that should help CI to succeed.
Comment 6 Ihar Hrachyshka 2017-05-24 01:04:10 UTC 
We still see failures in gate. We may need to backport https://review.openstack.org/#/c/467427/ and also enable arp_accept to mitigate the issue for now.
Comment 8 Ihar Hrachyshka 2017-05-24 21:55:20 UTC 
Sorry, but why have you moved it to MODIFIED/ON_QA? I reopened it for a reason.
Comment 9 Assaf Muller 2017-05-25 13:25:38 UTC 
(In reply to Ihar Hrachyshka from comment #8)
> Sorry, but why have you moved it to MODIFIED/ON_QA? I reopened it for a
> reason.

I'm assuming it's a script that is working off the 'fixed-in-version' field.
Comment 12 Ihar Hrachyshka 2017-05-31 21:02:37 UTC 
Note: this fix should be tested with https://bugzilla.redhat.com/show_bug.cgi?id=1457506 fixed on tripleo side.
Comment 16 Eran Kuris 2017-08-20 11:45:22 UTC 
fixed verified 
openstack-neutron-9.4.0-2.el7ost.noarch
Comment 18 errata-xmlrpc 2017-09-06 17:17:18 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2663