Bug 2128923 - TRY_AGAIN messages can flood neutron logs
Summary: TRY_AGAIN messages can flood neutron logs
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-neutron
Version: 17.0 (Wallaby)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: z1
: 17.0
Assignee: Terry Wilson
QA Contact: Maor
URL:
Whiteboard:
Depends On:
Blocks: 2129416 2129417
TreeView+ depends on / blocked
 
Reported: 2022-09-22 01:26 UTC by Terry Wilson
Modified: 2023-04-12 19:20 UTC (History)
2 users (show)

Fixed In Version: openstack-neutron-18.4.1-0.20221123190754.5258354.el9ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 2129416 (view as bug list)
Environment:
Last Closed: 2023-01-25 12:30:26 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 857775 0 None MERGED Fix indentation issue in wait_for_change override 2022-11-15 21:45:04 UTC
Red Hat Issue Tracker OSP-18863 0 None None None 2022-09-22 01:35:00 UTC
Red Hat Product Errata RHSA-2023:0275 0 None None None 2023-01-25 12:30:47 UTC

Description Terry Wilson 2022-09-22 01:26:47 UTC
Description of problem:
After the fix for bz2123014 merged, occasionally neutron could enter a loop continually retrying transactions and flooding logs with messages like:

OVSDB transaction returned TRY_AGAIN, retrying do_commit /usr/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:107 

Version-Release number of selected component (if applicable):

17.0.0

How reproducible: Variable in the real world, 100% in the lab

Steps to reproduce:
1) start neutron w/ debug=True and allow to complete loading/downloading ovn db data
2) iptables -I INPUT 1 -p tcp --dport 6641 -j DROP
3) openstack network create test1

Actual Results:
Lots of very fast TRY_AGAIN log messages

Expected results:
A single (or very few) TRY_AGAIN message and upon removing the blocking iptables rule the transaction should proceed (providing connectivity is restored before the ovsdb_connection_timeout)

The issue is that the wait_for_change() method that ovsdbapp calls is actually being monkeypatched by neutron code and the version in the neutron codebase has an indentation error which makes the method a NoOp.

Comment 13 errata-xmlrpc 2023-01-25 12:30:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Red Hat OpenStack Platform 17.0 (openstack-neutron) security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:0275


Note You need to log in before you can comment on or make changes to this bug.