Bug 2089304

Summary: fs020 and full-tempest-scenario job failing on tempest test neutron_tempest_plugin.scenario.test_trunk.TrunkTest.test_trunk_subport_lifecycle
Product: Red Hat OpenStack Reporter: Sandeep Yadav <sandyada>
Component: openstack-neutronAssignee: Fernando Royo <froyo>
Status: CLOSED ERRATA QA Contact: Fiorella Yanac <fyanac>
Severity: urgent Docs Contact:
Priority: high    
Version: 17.0 (Wallaby)CC: bcafarel, chrisw, froyo, jlibosva, mandreou, ralonsoh, scohen, ykarel
Target Milestone: betaKeywords: Triaged
Target Release: 17.0   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: openstack-neutron-18.4.1-0.20220601131831.dedcbbd.el8ost Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of:
: 2090604 (view as bug list) Environment:
Last Closed: 2022-09-21 12:21:38 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2090604    

Description Sandeep Yadav 2022-05-23 11:50:47 UTC
Description of problem:

Since 20th may, tempest test "neutron_tempest_plugin.scenario.test_trunk.TrunkTest.test_trunk_subport_lifecycle" is failing on both 16.2 and 17 integration line in fs020 and full-tempest-scenario job.

Version-Release number of selected component (if applicable):

* 16.2 on rhel-8
* 17 on rhel-8
* 17 on rhel-9


How reproducible:

Everytime


Steps to Reproduce:
1. Deploy a standalone environment
2. Run tempest tests


Actual results:

neutron_tempest_plugin.scenario.test_trunk.TrunkTest.test_trunk_subport_lifecycle fails:-

Error Snippet:

neutron_tempest_plugin.scenario.test_trunk.TrunkTest.test_trunk_subport_lifecycle[id-bb13fe28-f152-4000-8131-37890a40c79e]
--------------------------------------------------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):
      File "/usr/lib/python3.9/site-packages/neutron_tempest_plugin/common/utils.py", line 83, in wait_until_true
        eventlet.sleep(sleep)
      File "/usr/lib/python3.9/site-packages/eventlet/greenthread.py", line 36, in sleep
        hub.switch()
      File "/usr/lib/python3.9/site-packages/eventlet/hubs/hub.py", line 313, in switch
        return self.greenlet.switch()
    eventlet.timeout.Timeout: 60 seconds
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/usr/lib/python3.9/site-packages/neutron_tempest_plugin/scenario/test_trunk.py", line 263, in test_trunk_subport_lifecycle
        self._wait_for_port(port, status=constants.DOWN)
      File "/usr/lib/python3.9/site-packages/neutron_tempest_plugin/scenario/test_trunk.py", line 164, in _wait_for_port
        utils.wait_until_true(
      File "/usr/lib/python3.9/site-packages/neutron_tempest_plugin/common/utils.py", line 87, in wait_until_true
        raise exception
    RuntimeError: Timed out waiting for port '463791ef-a113-4e61-87d4-01cfb7a5ebb6' to transition to get status 'DOWN'.
~~~


Expected results:

Tempest test should pass


Additional info:

This may not be a neutron related bug but we need help from the neutron experts in isolating the issue.

Comment 7 Marios Andreou 2022-05-27 07:33:12 UTC
also seeing this on upstream master fs20 @ https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-1ctlr_2comp-featureset020-master/56ce076/logs/undercloud/var/log/tempest/stestr_results.html.gz  

test_trunk_subport_lifecycle[id-bb13fe28-f152-4000-8131-37890a40c79e]

        Body: b'{"itemNotFound": {"code": 404, "message": "Instance ccc2061d-83dc-4bd0-aeb7-045340f5102a could not be found."}}'

Comment 8 Yatin Karel 2022-05-27 08:18:53 UTC
(In reply to Marios Andreou from comment #7)
> also seeing this on upstream master fs20 @
> https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.
> org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-1ctlr_2comp-
> featureset020-master/56ce076/logs/undercloud/var/log/tempest/stestr_results.
> html.gz  
> 
> test_trunk_subport_lifecycle[id-bb13fe28-f152-4000-8131-37890a40c79e]
> 
>         Body: b'{"itemNotFound": {"code": 404, "message": "Instance
> ccc2061d-83dc-4bd0-aeb7-045340f5102a could not be found."}}'

@Marios That is different issue as it's failed during SSH, also is that consistently failing?

Comment 9 Marios Andreou 2022-05-27 10:14:25 UTC
(In reply to Yatin Karel from comment #8)
> (In reply to Marios Andreou from comment #7)
> > also seeing this on upstream master fs20 @
> > https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.
> > org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-1ctlr_2comp-
> > featureset020-master/56ce076/logs/undercloud/var/log/tempest/stestr_results.
> > html.gz  
> > 
> > test_trunk_subport_lifecycle[id-bb13fe28-f152-4000-8131-37890a40c79e]
> > 
> >         Body: b'{"itemNotFound": {"code": 404, "message": "Instance
> > ccc2061d-83dc-4bd0-aeb7-045340f5102a could not be found."}}'
> 
> @Marios That is different issue as it's failed during SSH, also is that
> consistently failing?

only seen it once so far

Comment 18 errata-xmlrpc 2022-09-21 12:21:38 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Release of components for Red Hat OpenStack Platform 17.0 (Wallaby)), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2022:6543