Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1784807

Summary: Tempest test tempest.scenario.test_network_basic_ops.TestNetworkBasicOps fails.
Product: Red Hat OpenStack Reporter: Sofer Athlan-Guyot <sathlang>
Component: openstack-neutronAssignee: Jakub Libosvar <jlibosva>
Status: CLOSED DUPLICATE QA Contact: Eran Kuris <ekuris>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 16.0 (Train)CC: amuller, chrisw, dalvarez, jlibosva, pgrist, scohen
Target Milestone: z2Keywords: Triaged
Target Release: 16.0 (Train on RHEL 8.1)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-03-04 10:30:14 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Sofer Athlan-Guyot 2019-12-18 11:35:10 UTC
Description of problem: Upon installation of OSP16 RHOS_TRUNK-16.0-RHEL-8-20191217.n.1 we have an tempest test failure:

        Body: b''
2019-12-18 04:46:45,643 321251 INFO     [tempest.lib.common.rest_client] Request (TestNetworkBasicOps:_run_cleanups): 204 DELETE https://10.0.0.101:13696/v2.0/networks/401c738e-553c-4f8c-b24c-4307a6c2dfc1 1.963s
2019-12-18 04:46:45,644 321251 DEBUG    [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
        Body: None
    Response - Headers: {'content-length': '0', 'x-openstack-request-id': 'req-b788f044-96a1-418f-b79c-66e337c646ed', 'date': 'Wed, 18 Dec 2019 04:46:45 GMT', 'connection': 'close', 'status': '204', 'content-location': 'https://10.0.0.101:13696/v2.0/networks/401c738e-553c-4f8c-b24c-4307a6c2dfc1'}
        Body: b''
}}}

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/tempest/common/utils/__init__.py", line 108, in wrapper
    return func(*func_args, **func_kwargs)
  File "/usr/lib/python3.6/site-packages/tempest/common/utils/__init__.py", line 89, in wrapper
    return f(*func_args, **func_kwargs)
  File "/usr/lib/python3.6/site-packages/tempest/scenario/test_network_basic_ops.py", line 884, in test_port_security_macspoofing_port
    nic=spoof_nic, should_succeed=True)
  File "/usr/lib/python3.6/site-packages/tempest/scenario/manager.py", line 1078, in check_remote_connectivity
    self.fail(msg)
  File "/usr/lib/python3.6/site-packages/unittest2/case.py", line 693, in fail
    raise self.failureException(msg)
AssertionError: Timed out waiting for 10.100.0.28 to become reachable from 10.0.0.214


Version-Release number of selected component (if applicable): RHOS_TRUNK-16.0-RHEL-8-20191217.n.1


How reproducible: always and since a lot of puddle.


Steps to Reproduce:
1. Deploy OSP16 RHOS_TRUNK-16.0-RHEL-8-20191217.n.1
2. run the mentioned tempest test

Additional info:

Error can be seen there https://rhos-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/DFG-upgrades-updates-16-from-passed_phase1-HA-ipv4/28//artifact/tempest-results/

it happen in the tempest run after installation (RHOS_TRUNK-16.0-RHEL-8-20191217.n.1), after the "update" (to RHOS_TRUNK-16.0-RHEL-8-20191217.n.1), and after the reboot.  It's pretty consistent.

Comment 1 Daniel Alvarez Sanchez 2019-12-18 13:39:32 UTC
Hi, the test `test_port_security_macspoofing_port` is skipped upstream [0] due to [1] so it should be skipped as well in the upgrades/updates jobs when using OVN (default).
We can use this BZ to modify the test in such way that we only test traffic from the spoofed MAC (via netcat as the LP bug suggests) instead of using ping.


[0] https://opendev.org/openstack/networking-ovn/src/branch/master/devstack/devstackgaterc#L81
[1] https://bugs.launchpad.net/tempest/+bug/1728886

Comment 3 Daniel Alvarez Sanchez 2019-12-19 15:42:41 UTC
I proposed this patch to fix the failing test and works locally :)

https://review.opendev.org/#/c/700005/

Once we get passing CI upstream we'll handle the backports

Comment 5 Sofer Athlan-Guyot 2019-12-20 13:56:37 UTC
In the meantime, I've added:

     ir_tempest_run_override_options: |-
      --blackre "test_port_security_macspoofing_port|DHCPAgentSchedulers|test_agent_management.AgentManagementTestJSON|test_router_interface_status|test_qos|InternalDNSTest.test_dns_domain_and_name"

for ovn based network in the update jobs and we don't have the issue anymore (obviously ...)

When you feel confident we can remove this please let us know.

Thanks again for the pointers,

Comment 8 Jakub Libosvar 2020-03-04 10:30:14 UTC

*** This bug has been marked as a duplicate of bug 1805709 ***