Bug 2024898

Description of problem:

OSP17 jobs failing on network related tempest test, Instance not reachable. 

tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_connectivity_between_vms_on_different_networks[compute,id-1546850e-fbaa-42f5-8b5f-03d8a6a95f15,network,slow]


~~~
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/tempest/common/utils/__init__.py", line 70, in wrapper
    return f(*func_args, **func_kwargs)
  File "/usr/lib/python3.6/site-packages/tempest/scenario/test_network_basic_ops.py", line 512, in test_connectivity_between_vms_on_different_networks
    new_server_ips, should_connect=True)
  File "/usr/lib/python3.6/site-packages/tempest/scenario/test_network_basic_ops.py", line 386, in _check_server_connectivity
    should_connect)
  File "/usr/lib/python3.6/site-packages/tempest/scenario/manager.py", line 1453, in check_remote_connectivity
    self.fail(msg)
  File "/usr/lib/python3.6/site-packages/unittest2/case.py", line 690, in fail
    raise self.failureException(msg)
AssertionError: Timed out waiting for <IP> to become reachable from <another IP>
~~~

Neutron related error in logs(containers-multinode job):-


~~~
2021-11-19 03:27:58.642 ERROR /var/log/containers/neutron/dhcp-agent.log: 136540 ERROR neutron_lib.rpc [req-76d48ac0-74d3-46a9-bec3-6e1a4e73780f - - - - -] Timeout in RPC method get_active_networks_info. Waiting for 43 seconds before next attempt. If the server is not down, consider increasing the rpc_response_timeout option as Neutron server(s) may be overloaded and unable to respond quickly enough.: oslo_messaging.exceptions.MessagingTimeout: Timed out waiting for a reply to message ID 53ad17c5a27648a4a677ca8dc235c8cf
~~~

Neutron related error in logs(ovb job):-

~~~
021-11-19 09:59:38.620 ERROR /var/log/containers/neutron/server.log.1: 27 ERROR neutron_lib.callbacks.manager [req-29005ad3-e904-41ac-a266-28eff6c4c9c4 1a53295027e247f9bef97c1e2728c0d8 5089ae5963ec4669863fa9995c9085f1 - default default] Error during notification for neutron.services.logapi.logging_plugin.LoggingPlugin._clean_logs_by_target_id-16795 port, after_delete: TypeError: _clean_logs_by_target_id() got an unexpected keyword argument 'context'
2021-11-19 09:59:38.620 ERROR /var/log/containers/neutron/server.log.1: 27 ERROR neutron_lib.callbacks.manager Traceback (most recent call last):
2021-11-19 09:59:38.620 ERROR /var/log/containers/neutron/server.log.1: 27 ERROR neutron_lib.callbacks.manager   File "/usr/lib/python3.6/site-packages/neutron_lib/callbacks/manager.py", line 197, in _notify_loop
2021-11-19 09:59:38.620 ERROR /var/log/containers/neutron/server.log.1: 27 ERROR neutron_lib.callbacks.manager     callback(resource, event, trigger, **kwargs)
2021-11-19 09:59:38.620 ERROR /var/log/containers/neutron/server.log.1: 27 ERROR neutron_lib.callbacks.manager TypeError: _clean_logs_by_target_id() got an unexpected keyword argument 'context'
~~~


Version-Release number of selected component (if applicable):

RHOSP 17, Integration line


How reproducible:
Everytime


Steps to Reproduce:
1. Deploy OSP17 multinode environment
2. Run tempest test - tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_connectivity_between_vms_on_different_networks


Actual results:

Instance not reachable causing failure of tempest tests.


Expected results:

Tempest tests should pass


Additional info:

Last green run of job noticed on 12th Nov