1956074 – check_for_mcast_flood_reports periodic in maintenace task fails because of ovsdb lock

Bug 1956074 - check_for_mcast_flood_reports periodic in maintenace task fails because of ovsdb lock

Summary: check_for_mcast_flood_reports periodic in maintenace task fails because of ov...

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat OpenStack
Classification:	Red Hat
Component:	python-networking-ovn
Sub Component:
Version:	16.1 (Train)
Hardware:	Unspecified
OS:	Unspecified
Priority:	medium
Severity:	medium
Target Milestone:	z7
Target Release:	16.1 (Train on RHEL 8.2)
Assignee:	Lucas Alvares Gomes
QA Contact:	Eran Kuris
Docs Contact:
URL:
Whiteboard:
Duplicates (1):	1979795 (view as bug list)
Depends On:
Blocks:	1983979
TreeView+	depends on / blocked

Reported:	2021-05-02 14:32 UTC by Jakub Libosvar
Modified:	2024-12-20 19:59 UTC (History)
CC List:	9 users (show)
Fixed In Version:	python-networking-ovn-7.3.1-1.20210518143301.4e24f4c.el8ost
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Clones:	1983979 (view as bug list)
Environment:
Last Closed:	2021-12-09 20:19:04 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Priority	Status	Summary	Last Updated
OpenStack gerrit	789547	None	MERGED	[OVN] Check for lock in check_for_mcast_flood_reports	2023-08-28 12:33:13 UTC
Red Hat Issue Tracker	OSP-3534	None	None	None	2021-11-18 11:31:46 UTC
Red Hat Knowledge Base (Solution)	6191261	None	None	None	2022-03-18 19:13:10 UTC
Red Hat Product Errata	RHBA-2021:3762	None	None	None	2021-12-09 20:19:31 UTC

Description Jakub Libosvar 2021-05-02 14:32:02 UTC

Description of problem:
2021-05-02 14:23:29.193 599 ERROR futurist.periodics [-] Failed to call periodic 'networking_ovn.common.maintenance.DBInconsistenciesPeriodics.check_for_mcast_flood_reports' (it runs every 600.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
2021-05-02 14:23:29.193 599 ERROR futurist.periodics Traceback (most recent call last):
2021-05-02 14:23:29.193 599 ERROR futurist.periodics   File "/usr/lib/python3.6/site-packages/ovsdbapp/api.py", line 111, in transaction
2021-05-02 14:23:29.193 599 ERROR futurist.periodics     yield self._nested_txns_map[cur_thread_id]
2021-05-02 14:23:29.193 599 ERROR futurist.periodics KeyError: 139844066981736
2021-05-02 14:23:29.193 599 ERROR futurist.periodics
2021-05-02 14:23:29.193 599 ERROR futurist.periodics During handling of the above exception, another exception occurred:
2021-05-02 14:23:29.193 599 ERROR futurist.periodics
2021-05-02 14:23:29.193 599 ERROR futurist.periodics Traceback (most recent call last):
2021-05-02 14:23:29.193 599 ERROR futurist.periodics   File "/usr/lib/python3.6/site-packages/futurist/periodics.py", line 290, in run
2021-05-02 14:23:29.193 599 ERROR futurist.periodics     work()
2021-05-02 14:23:29.193 599 ERROR futurist.periodics   File "/usr/lib/python3.6/site-packages/futurist/periodics.py", line 64, in __call__
2021-05-02 14:23:29.193 599 ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
2021-05-02 14:23:29.193 599 ERROR futurist.periodics   File "/usr/lib/python3.6/site-packages/futurist/periodics.py", line 178, in decorator
2021-05-02 14:23:29.193 599 ERROR futurist.periodics     return f(*args, **kwargs)
2021-05-02 14:23:29.193 599 ERROR futurist.periodics   File "/usr/lib/python3.6/site-packages/networking_ovn/common/maintenance.py", line 667, in check_for_mcast_flood_reports
2021-05-02 14:23:29.193 599 ERROR futurist.periodics     for port in self._nb_idl.lsp_list().execute(check_error=True):
2021-05-02 14:23:29.193 599 ERROR futurist.periodics   File "/usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 42, in execute
2021-05-02 14:23:29.193 599 ERROR futurist.periodics     t.add(self)
2021-05-02 14:23:29.193 599 ERROR futurist.periodics   File "/usr/lib64/python3.6/contextlib.py", line 88, in __exit__
2021-05-02 14:23:29.193 599 ERROR futurist.periodics     next(self.gen)
2021-05-02 14:23:29.193 599 ERROR futurist.periodics   File "/usr/lib/python3.6/site-packages/networking_ovn/ovsdb/impl_idl_ovn.py", line 196, in transaction
2021-05-02 14:23:29.193 599 ERROR futurist.periodics     yield t
2021-05-02 14:23:29.193 599 ERROR futurist.periodics   File "/usr/lib64/python3.6/contextlib.py", line 88, in __exit__
2021-05-02 14:23:29.193 599 ERROR futurist.periodics     next(self.gen)
2021-05-02 14:23:29.193 599 ERROR futurist.periodics   File "/usr/lib/python3.6/site-packages/ovsdbapp/api.py", line 119, in transaction
2021-05-02 14:23:29.193 599 ERROR futurist.periodics     del self._nested_txns_map[cur_thread_id]
2021-05-02 14:23:29.193 599 ERROR futurist.periodics   File "/usr/lib/python3.6/site-packages/ovsdbapp/api.py", line 69, in __exit__
2021-05-02 14:23:29.193 599 ERROR futurist.periodics     self.result = self.commit()
2021-05-02 14:23:29.193 599 ERROR futurist.periodics   File "/usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 62, in commit
2021-05-02 14:23:29.193 599 ERROR futurist.periodics     raise result.ex
2021-05-02 14:23:29.193 599 ERROR futurist.periodics   File "/usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 128, in run
2021-05-02 14:23:29.193 599 ERROR futurist.periodics     txn.results.put(txn.do_commit())
2021-05-02 14:23:29.193 599 ERROR futurist.periodics   File "/usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 115, in do_commit
2021-05-02 14:23:29.193 599 ERROR futurist.periodics     raise RuntimeError(msg)
2021-05-02 14:23:29.193 599 ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
2021-05-02 14:23:29.193 599 ERROR futurist.periodics

It seems the check_for_mcast_flood_reports method doesn't use has_lock

Version-Release number of selected component (if applicable):
OSP 16.1.5

How reproducible:
Always

Steps to Reproduce:
1. Install openstack with at least 3 controllers
2.
3.

Actual results:
All controllers that don't hold the lock fail

Expected results:


Additional info:

Comment 4 Lucas Alvares Gomes 2021-07-12 12:30:12 UTC

*** Bug 1979795 has been marked as a duplicate of this bug. ***

Comment 16 Eran Kuris 2021-11-02 10:04:12 UTC

Fix verified: 
RHOS-16.1-RHEL-8-20211007.n.1

()[neutron@controller-1 /]$ rpm -qa | grep  ovn
puppet-ovn-15.4.1-1.20210528102649.192ac4e.el8ost.noarch
python3-networking-ovn-7.3.1-1.20210714143309.el8ost.noarch

Comment 24 errata-xmlrpc 2021-12-09 20:19:04 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform 16.1.7 (Train) bug fix and enhancement advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:3762

Note You need to log in before you can comment on or make changes to this bug.