Bug 1810451
| Summary: | OSP 13, neutron ml2-ovs, DPDK - stop/wait/start of nova instance sometimes deletes tbr and yields Error executing command: RowNotFound: Cannot find Bridge with name= | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Ignacio <igarciam> |
| Component: | openstack-neutron | Assignee: | Nate Johnston <njohnston> |
| Status: | CLOSED ERRATA | QA Contact: | Yariv <yrachman> |
| Severity: | urgent | Docs Contact: | |
| Priority: | high | ||
| Version: | 13.0 (Queens) | CC: | amuller, bcafarel, chrisw, ekuris, njohnston, scohen, vkhitrin |
| Target Milestone: | z12 | Keywords: | Triaged, ZStream |
| Target Release: | 13.0 (Queens) | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | openstack-neutron-12.1.1-15.el7ost | Doc Type: | No Doc Update |
| Doc Text: | Story Points: | --- | |
| Clone Of: | 1655177 | Environment: | |
| Last Closed: | 2020-06-24 11:53:08 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Ignacio
2020-03-05 09:53:13 UTC
What is happening here is an artifact of the interplay between Neutron trunking and DPDK vhostuser mode. DPDK/vhu mode means that when an instance is powered off the port is deleted, and when an instance is powered on a port is created. This means a reboot is functionally a super fast delete-then-create. Neutron trunking mode in combination with DPDK/vhu implements a trunk bridge for each tenant, and the ports for the instances are created as subports of that bridge. The standard way a trunk bridge works is that when all the subports are deleted, a thread is spawned to delete the trunk bridge, because that is an expensive and time-consuming operation. That means that if the port in question is the only port on the trunk on that compute node, this happens: 1. The port is deleted (Mar 24 11:33:13) 2. A thread is spawned to delete the trunk 3. The port is recreated (Mar 24 11:33:15) If the trunk is deleted after #3 happens then the instance has no networking and is inaccessible; this is the scenario that was dealt with in the previous bug, BZ-1655177. What is happening here is that the trunk is being deleted in the middle of the execution of #3, so that it stops existing in the middle of the port creation logic but before the port is actually created. Since this is a timing issue between two different threads it's difficult to stamp out entirely, but I think the best way to do it is to add a slight delay in the trunk deletion thread, just a second or two. That will give the port time to come back online and avoid the trunk deletion entirely. Filed launchpad bug [1] per review feedback [2]. [1] https://bugs.launchpad.net/neutron/+bug/1869244 [2] https://review.opendev.org/#/c/714783/2/ Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:2724 |