16.2 Integration pipeline, tempest test: neutron_tempest_plugin.scenario.test_internal_dns.InternalDNSTest is failing in CI. Job name: periodic-tripleo-ci-rhel-8-standalone-full-tempest-scenario-rhos-16.2 Logs: https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-periodic-integration-rhos-16.2/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-rhel-8-standalone-full-tempest-scenario-rhos-16.2/96d184b/logs/undercloud/var/log/tempest/stestr_results.html ~~~ Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/tempest/common/utils/__init__.py", line 108, in wrapper return func(*func_args, **func_kwargs) File "/usr/lib/python3.6/site-packages/neutron_tempest_plugin/scenario/test_internal_dns.py", line 76, in test_dns_domain_and_name self.assertIn('openstackgate.local', resolv_conf) File "/usr/lib/python3.6/site-packages/testtools/testcase.py", line 417, in assertIn self.assertThat(haystack, Contains(needle), message) File "/usr/lib/python3.6/site-packages/testtools/testcase.py", line 498, in assertThat raise mismatch_error testtools.matchers._impl.MismatchError: 'openstackgate.local' not in 'nameserver 10.5.30.160\n' ~~~ Following error observed in neutronserver logs:- https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-periodic-integration-rhos-16.2/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-rhel-8-standalone-full-tempest-scenario-rhos-16.2/96d184b/logs/undercloud/var/log/containers/neutron/server.log.1 ~~~ 2021-05-03 01:47:23.285 18 ERROR ovsdbapp.backend.ovs_idl.idlutils [-] Unable to open stream to tcp:192.168.24.1:6641 to retrieve schema: Connection refused . 2021-05-03 01:47:23.364 16 ERROR ovsdbapp.backend.ovs_idl.idlutils [-] Unable to open stream to tcp:192.168.24.1:6641 to retrieve schema: Connection refused . . 2021-05-03 01:53:56.212 15 ERROR ovsdbapp.backend.ovs_idl.transaction [req-e8bb48e2-6490-4680-b551-d235ad5196e4 a75c96127fe9433390969eeefceebd71 b0d6aefc397d4a1e801c23b6df857468 - default default] Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 128, in run txn.results.put(txn.do_commit()) File "/usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 86, in do_commit command.run_idl(txn) File "/usr/lib/python3.6/site-packages/networking_ovn/ovsdb/commands.py", line 937, in run_idl resource_id=self.name, resource_type=self.resource_type) networking_ovn.common.exceptions.RevisionConflict: OVN revision number for a1d1fb0c-4044-41b8-bd00-142f811d9c9a (type: ports) is equal or higher than the given resource. Skipping update ~~~ Another example: https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-periodic-integration-rhos-16.2/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-rhel-8-standalone-full-tempest-scenario-rhos-16.2/06615e7/logs/undercloud/var/log/tempest/stestr_results.html Version-Release number of selected component (if applicable): 16.2 How reproducible: Everytime in Integration pipeline https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/builds?job_name=periodic-tripleo-ci-rhel-8-standalone-full-tempest-scenario-rhos-16.2
The failed test was skipped earlier, failed test is also skipped in upstream for all the branches. https://opendev.org/openstack/openstack-tempest-skiplist/src/branch/master/roles/validate-tempest/vars/tempest_skip.yml#L71-L97 Ex: neutron_tempest_plugin.scenario.test_internal_dns.InternalDNSTest was not running earlier Now: https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-periodic-integration-rhos-16.2/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-rhel-8-standalone-full-tempest-scenario-rhos-16.2/96d184b/logs/undercloud/var/log/tempest/stestr_results.html Earlier: https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-periodic-integration-rhos-16.2/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-rhel-8-standalone-full-tempest-scenario-rhos-16.2/dda768e/logs/undercloud/var/log/tempest/stestr_results.html This is caused by tripleo-quickstart-extras patch caused some tests which were earlier skipped to run, we are reverting that patch https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/789450 In skiplist https://bugs.launchpad.net/tripleo/+bug/1832166 is mentioned as reason to add these test in skiplist, but it seems this bug was never fully solved, Would be great if we can investigate the failure so that we can unskip these tests in downstream and upstream if they add value.