Bug 1452887

Summary: Neutron L3 agent sends gratuitous ARP packets with 1 sec delay, which may conflict with default Linux kernel locktime value (1 sec)
Product: Red Hat OpenStack Reporter: Ihar Hrachyshka <ihrachys>
Component: openstack-neutronAssignee: Ihar Hrachyshka <ihrachys>
Status: CLOSED ERRATA QA Contact: GenadiC <gcheresh>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 10.0 (Newton)CC: abregman, amuller, ccollett, chrisw, ihrachys, nyechiel, oblaut, srevivo, tfreger
Target Milestone: zstreamKeywords: Triaged, ZStream
Target Release: 7.0 (Kilo)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-neutron-2015.1.4-16.el7ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1452884 Environment:
Last Closed: 2017-09-12 17:30:15 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Ihar Hrachyshka 2017-05-19 21:14:12 UTC
+++ This bug was initially created as a clone of Bug #1452884 +++

+++ This bug was initially created as a clone of Bug #1390863 +++

Description of problem:

Running tempest, the following tests are failing

tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops
 tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_update_instance_port_admin_state
 tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_update_router_admin_state
tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_slaac_from_os
 tempest.scenario.test_network_v6.TestGettingAddress.test_multi_prefix_dhcpv6_stateless
 tempest.scenario.test_network_v6.TestGettingAddress.test_multi_prefix_slaac
tempest.scenario.test_snapshot_pattern.TestSnapshotPattern.test_snapshot_pattern

Version-Release number of selected component (if applicable):
openstack-neutron-9.0.0-1.5.el7ost.noarch
openstack-neutron-bigswitch-agent-7.0.5-0.20161015221623.f83c499.el7ost.noarch
openstack-neutron-bigswitch-lldp-7.0.5-0.20161015221623.f83c499.el7ost.noarch
openstack-neutron-common-9.0.0-1.5.el7ost.noarch
openstack-neutron-lbaas-9.0.0-2.el7ost.noarch
openstack-neutron-metering-agent-9.0.0-1.5.el7ost.noarch
openstack-neutron-ml2-9.0.0-1.5.el7ost.noarch
openstack-neutron-openvswitch-9.0.0-1.5.el7ost.noarch
openstack-neutron-sriov-nic-agent-9.0.0-1.5.el7ost.noarch

How reproducible: 100%


Steps to Reproduce:
1. Deploy RHOSP 10 HA (3 controllers, 2 compute)
2. Run tempest network scenario tests

Actual results: The above tests are failing with this traceback :

Traceback (most recent call last):
  File "/home/stack/tempest/tempest/test.py", line 100, in wrapper
    return f(self, *func_args, **func_kwargs)
  File "/home/stack/tempest/tempest/scenario/test_snapshot_pattern.py", line 71, in test_snapshot_pattern
    private_key=keypair['private_key'])
  File "/home/stack/tempest/tempest/scenario/manager.py", line 604, in get_timestamp
    private_key=private_key)
  File "/home/stack/tempest/tempest/scenario/manager.py", line 323, in get_remote_client
    linux_client.validate_authentication()
  File "/home/stack/tempest/tempest/common/utils/linux/remote_client.py", line 54, in wrapper
    six.reraise(*original_exception)
  File "/home/stack/tempest/tempest/common/utils/linux/remote_client.py", line 35, in wrapper
    return function(self, *args, **kwargs)
  File "/home/stack/tempest/tempest/common/utils/linux/remote_client.py", line 99, in validate_authentication
    self.ssh_client.test_connection_auth()
  File "/home/stack/tempest/tempest/lib/common/ssh.py", line 176, in test_connection_auth
    connection = self._get_ssh_connection()
  File "/home/stack/tempest/tempest/lib/common/ssh.py", line 90, in _get_ssh_connection
    password=self.password)
tempest.lib.exceptions.SSHTimeout: Connection to the 192.0.2.61 via SSH timed out.
User: cirros, Password: None


Expected results: Tests passed 100%


Additional info: Link to the CI job will be provided in next comment

--- Additional comment from Arie Bregman on 2016-11-02 02:56:31 EDT ---

Link to the job (includes test results + artifacts from the nodes):

https://rhos-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/neutron-tempest-rhos-coreci/1032/testReport

--- Additional comment from Jakub Libosvar on 2016-12-19 06:02:29 EST ---

Link to newer failures: https://rhos-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/neutron-tempest-rhos-coreci/1433/testReport/

--- Additional comment from Ihar Hrachyshka on 2017-05-18 13:18:10 EDT ---

Neutron side workaround applied that should help CI to succeed.

Comment 5 GenadiC 2017-09-06 07:32:09 UTC
Code verification in python-neutron-2015.1.4-21.el7ost.noarch

Comment 8 errata-xmlrpc 2017-09-12 17:30:15 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2694