Bug 2128923

Summary:	TRY_AGAIN messages can flood neutron logs
Product:	Red Hat OpenStack	Reporter:	Terry Wilson <twilson>
Component:	openstack-neutron	Assignee:	Terry Wilson <twilson>
Status:	CLOSED ERRATA	QA Contact:	Maor <mblue>
Severity:	high	Docs Contact:
Priority:	high
Version:	17.0 (Wallaby)	CC:	chrisw, scohen
Target Milestone:	z1	Keywords:	Triaged
Target Release:	17.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	openstack-neutron-18.4.1-0.20221123190754.5258354.el9ost	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:
Clones:	2129416 (view as bug list)		Environment:
Last Closed:	2023-01-25 12:30:26 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	2129416, 2129417

Description Terry Wilson 2022-09-22 01:26:47 UTC

Description of problem:
After the fix for bz2123014 merged, occasionally neutron could enter a loop continually retrying transactions and flooding logs with messages like:

OVSDB transaction returned TRY_AGAIN, retrying do_commit /usr/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:107 

Version-Release number of selected component (if applicable):

17.0.0

How reproducible: Variable in the real world, 100% in the lab

Steps to reproduce:
1) start neutron w/ debug=True and allow to complete loading/downloading ovn db data
2) iptables -I INPUT 1 -p tcp --dport 6641 -j DROP
3) openstack network create test1

Actual Results:
Lots of very fast TRY_AGAIN log messages

Expected results:
A single (or very few) TRY_AGAIN message and upon removing the blocking iptables rule the transaction should proceed (providing connectivity is restored before the ovsdb_connection_timeout)

The issue is that the wait_for_change() method that ovsdbapp calls is actually being monkeypatched by neutron code and the version in the neutron codebase has an indentation error which makes the method a NoOp.

Comment 13 errata-xmlrpc 2023-01-25 12:30:26 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Red Hat OpenStack Platform 17.0 (openstack-neutron) security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:0275