Bug 1956801 - 16.2 tempest test: neutron_tempest_plugin.scenario.test_internal_dns.InternalDNSTest failed with MismatchError: 'openstackgate.local' not in 'nameserver 10.5.30.160
Summary: 16.2 tempest test: neutron_tempest_plugin.scenario.test_internal_dns.Internal...
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-neutron
Version: 16.2 (Train)
Hardware: x86_64
OS: Linux
urgent
urgent
Target Milestone: ---
: ---
Assignee: Candido Campos
QA Contact: Eran Kuris
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-05-04 12:42 UTC by Sandeep Yadav
Modified: 2021-05-17 14:00 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-05-17 14:00:05 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description Sandeep Yadav 2021-05-04 12:42:13 UTC
16.2 Integration pipeline, tempest test: neutron_tempest_plugin.scenario.test_internal_dns.InternalDNSTest is failing in CI.

Job name:  periodic-tripleo-ci-rhel-8-standalone-full-tempest-scenario-rhos-16.2


Logs:
https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-periodic-integration-rhos-16.2/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-rhel-8-standalone-full-tempest-scenario-rhos-16.2/96d184b/logs/undercloud/var/log/tempest/stestr_results.html

~~~
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/tempest/common/utils/__init__.py", line 108, in wrapper
    return func(*func_args, **func_kwargs)
  File "/usr/lib/python3.6/site-packages/neutron_tempest_plugin/scenario/test_internal_dns.py", line 76, in test_dns_domain_and_name
    self.assertIn('openstackgate.local', resolv_conf)
  File "/usr/lib/python3.6/site-packages/testtools/testcase.py", line 417, in assertIn
    self.assertThat(haystack, Contains(needle), message)
  File "/usr/lib/python3.6/site-packages/testtools/testcase.py", line 498, in assertThat
    raise mismatch_error
testtools.matchers._impl.MismatchError: 'openstackgate.local' not in 'nameserver 10.5.30.160\n'
~~~

Following error observed in neutronserver logs:-

https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-periodic-integration-rhos-16.2/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-rhel-8-standalone-full-tempest-scenario-rhos-16.2/96d184b/logs/undercloud/var/log/containers/neutron/server.log.1
~~~
2021-05-03 01:47:23.285 18 ERROR ovsdbapp.backend.ovs_idl.idlutils [-] Unable to open stream to tcp:192.168.24.1:6641 to retrieve schema: Connection refused
.
2021-05-03 01:47:23.364 16 ERROR ovsdbapp.backend.ovs_idl.idlutils [-] Unable to open stream to tcp:192.168.24.1:6641 to retrieve schema: Connection refused
.
.
2021-05-03 01:53:56.212 15 ERROR ovsdbapp.backend.ovs_idl.transaction [req-e8bb48e2-6490-4680-b551-d235ad5196e4 a75c96127fe9433390969eeefceebd71 b0d6aefc397d4a1e801c23b6df857468 - default default] Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 128, in run
    txn.results.put(txn.do_commit())
  File "/usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 86, in do_commit
    command.run_idl(txn)
  File "/usr/lib/python3.6/site-packages/networking_ovn/ovsdb/commands.py", line 937, in run_idl
    resource_id=self.name, resource_type=self.resource_type)
networking_ovn.common.exceptions.RevisionConflict: OVN revision number for a1d1fb0c-4044-41b8-bd00-142f811d9c9a (type: ports) is equal or higher than the given resource. Skipping update

~~~

Another example:
https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-periodic-integration-rhos-16.2/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-rhel-8-standalone-full-tempest-scenario-rhos-16.2/06615e7/logs/undercloud/var/log/tempest/stestr_results.html


Version-Release number of selected component (if applicable):
16.2

How reproducible:
Everytime in Integration pipeline

https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/builds?job_name=periodic-tripleo-ci-rhel-8-standalone-full-tempest-scenario-rhos-16.2

Comment 1 Sandeep Yadav 2021-05-05 10:04:09 UTC
The failed test was skipped earlier, failed test is also skipped in upstream for all the branches. 

https://opendev.org/openstack/openstack-tempest-skiplist/src/branch/master/roles/validate-tempest/vars/tempest_skip.yml#L71-L97

Ex: neutron_tempest_plugin.scenario.test_internal_dns.InternalDNSTest was not running earlier

Now:
https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-periodic-integration-rhos-16.2/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-rhel-8-standalone-full-tempest-scenario-rhos-16.2/96d184b/logs/undercloud/var/log/tempest/stestr_results.html

Earlier:
https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-periodic-integration-rhos-16.2/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-rhel-8-standalone-full-tempest-scenario-rhos-16.2/dda768e/logs/undercloud/var/log/tempest/stestr_results.html


This is caused by tripleo-quickstart-extras patch caused some tests which were earlier skipped to run, we are reverting that patch https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/789450 

In skiplist https://bugs.launchpad.net/tripleo/+bug/1832166 is mentioned as reason to add these test in skiplist, but it seems this bug was never fully solved, Would be great if we can investigate the failure so that we can unskip these tests in downstream and upstream if they add value.


Note You need to log in before you can comment on or make changes to this bug.