Bug 2228186 - Neutron workers may not have OVN IDLs instantiated
Summary: Neutron workers may not have OVN IDLs instantiated
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-neutron
Version: 17.1 (Wallaby)
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: z1
: 17.1
Assignee: Jakub Libosvar
QA Contact: Vadim Khitrin
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-08-01 17:05 UTC by Jakub Libosvar
Modified: 2023-09-20 00:30 UTC (History)
9 users (show)

Fixed In Version: openstack-neutron-18.6.1-1.20230518200972.el9ost
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-09-20 00:29:52 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 890245 0 None MERGED hash-ring: Retry all DB operations if inactive 2023-08-21 15:34:09 UTC
OpenStack gerrit 891277 0 None MERGED hash-ring: Retry all DB operations if inactive 2023-08-21 15:34:08 UTC
Red Hat Issue Tracker OSP-27123 0 None None None 2023-08-01 17:06:02 UTC
Red Hat Product Errata RHBA-2023:5138 0 None None None 2023-09-20 00:30:28 UTC

Description Jakub Libosvar 2023-08-01 17:05:32 UTC 
Description of problem:
In a very rare occasion during the Neutron server startup, it may happen that connection to MySQL server is lost. If this happens after the parent process spawns workers but before OVN IDLs are instantiated, then setting up hash ring raises an exception when attempting to delete the hash ring node in post_fork_initialize() [1]. This happens before the OVN IDLs are instantiated [2] but after the post fork event is cleared [3].

If this happens, the API worker starts handling requests anyways without having connection to the OVN databases. Any request, e.g. "router list", will hang until HTTP timeout because we wait indefinitely for the post_fork_event to be set [4]. As Neutron is waiting, there is no sign of any ERROR or traceback in the logs and it's not intuitive at all what is happening, as haproxy successfully redirects traffic to Neutron server but there is no sign about the request in Neutron logs.

[1] https://github.com/openstack/neutron/blob/45cdd90640160a78329bc1677b587793c175db48/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py#L345
[2] https://github.com/openstack/neutron/blob/45cdd90640160a78329bc1677b587793c175db48/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py#L348
[3] https://github.com/openstack/neutron/blob/45cdd90640160a78329bc1677b587793c175db48/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py#L341
[4] https://github.com/openstack/neutron/blob/45cdd90640160a78329bc1677b587793c175db48/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py#L173

Version-Release number of selected component (if applicable):
Probably all versions

How reproducible:
Very rare

Steps to Reproduce:
1. Start Neutron server while stopping MySQL DB server or interrupt traffic to the server
2. The MySQL connection must be lost after api workers are spawned but before they clear hash ring nodes.

Actual results:
Neutron stops handling requests that require OVN DB data

Expected results:
Neutron waits on startup until mysql is established

Additional info:
Some requests that don't use OVN DB still work. Only the particular worker that suffered the problem during the startup won't be able to handle requests. For user it leads to very sporadic failures as it requires a request that needs OVN DB and lands on the malfunctioned worker.

Log snippet from the worker start:
2023-08-01 09:33:02.879 7 INFO oslo_service.service [req-b96f1dc7-0ace-4235-b7eb-5a1cafec2a2f - - - - -] Starting 2 workers
2023-08-01 09:33:02.887 7 DEBUG oslo_service.service [req-b96f1dc7-0ace-4235-b7eb-5a1cafec2a2f - - - - -] Started child 105 _start_child /usr/lib/python3.9/site-packages/oslo_service/service.py:576
2023-08-01 09:33:02.893 105 DEBUG neutron_lib.callbacks.manager [-] Notify callbacks ['neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver.post_fork_initialize-1809925'] for process, after_
init _notify_loop /usr/lib/python3.9/site-packages/neutron_lib/callbacks/manager.py:192
2023-08-01 09:33:02.896 105 DEBUG oslo_concurrency.lockutils [-] Lock "hash_ring_probe_lock" acquired by "neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver._setup_hash_ring" :: waited 0.0
01s inner /usr/lib/python3.9/site-packages/oslo_concurrency/lockutils.py:355
2023-08-01 09:33:02.914 105 DEBUG oslo_concurrency.lockutils [req-2d2a4a3d-4132-41c2-a6cb-ccba7c69cf61 - - - - -] Lock "hash_ring_probe_lock" released by "neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver._setup_hash_ring" :: held 0.018s inner /usr/lib/python3.9/site-packages/oslo_concurrency/lockutils.py:367
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager [req-2d2a4a3d-4132-41c2-a6cb-ccba7c69cf61 - - - - -] Error during notification for neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver.post_fork_initialize-1809925 process, after_init: oslo_db.exception.DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query')
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager Traceback (most recent call last):
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 2336, in _wrap_pool_connect
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     return fn()
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/pool/base.py", line 364, in connect
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     return _ConnectionFairy._checkout(self)
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/pool/base.py", line 778, in _checkout
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     fairy = _ConnectionRecord.checkout(pool)
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/pool/base.py", line 495, in checkout
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     rec = pool._do_get()
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/pool/impl.py", line 140, in _do_get
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     self._dec_overflow()
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     compat.raise_(
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     raise exception
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/pool/impl.py", line 137, in _do_get
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     return self._create_connection()
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/pool/base.py", line 309, in _create_connection
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     return _ConnectionRecord(self)
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/pool/base.py", line 440, in __init__
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     self.__connect(first_connect_check=True)
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/pool/base.py", line 661, in __connect
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     pool.logger.debug("Error on connect(): %s", e)
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     compat.raise_(
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     raise exception
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/pool/base.py", line 656, in __connect
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     connection = pool._invoke_creator(self)
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     return dialect.connect(*cargs, **cparams)
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/default.py", line 509, in connect
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     return self.dbapi.connect(*cargs, **cparams)
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib/python3.9/site-packages/pymysql/__init__.py", line 94, in Connect
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     return Connection(*args, **kwargs)
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 327, in __init__
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     self.connect()
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 587, in connect
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     self._get_server_information()
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 969, in _get_server_information
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     packet = self._read_packet()
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 646, in _read_packet
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     packet_header = self._read_bytes(4)
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 698, in _read_bytes
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     raise err.OperationalError(
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager pymysql.err.OperationalError: (2013, 'Lost connection to MySQL server during query')
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager The above exception was the direct cause of the following exception:
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager Traceback (most recent call last):
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib/python3.9/site-packages/neutron_lib/callbacks/manager.py", line 197, in _notify_loop
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     callback(resource, event, trigger, **kwargs)
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib/python3.9/site-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py", line 322, in post_fork_initialize
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     self._setup_hash_ring()
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib/python3.9/site-packages/oslo_concurrency/lockutils.py", line 360, in inner
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     return f(*args, **kwargs)
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib/python3.9/site-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py", line 298, in _setup_hash_ring
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     self._clean_hash_ring()
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib/python3.9/site-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py", line 272, in _clean_hash_ring
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     ovn_hash_ring_db.remove_nodes_from_host(admin_context,
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib/python3.9/site-packages/neutron/db/ovn_hash_ring_db.py", line 42, in remove_nodes_from_host
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     context.session.query(ovn_models.OVNHashRing).filter(
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/orm/query.py", line 3926, in delete
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     delete_op.exec_()
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/orm/persistence.py", line 1697, in exec_
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     self._do_exec()
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/orm/persistence.py", line 1930, in _do_exec
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     self._execute_stmt(delete_stmt)
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/orm/persistence.py", line 1702, in _execute_stmt
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     self.result = self.query._execute_crud(stmt, self.mapper)
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/orm/query.py", line 3564, in _execute_crud
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     conn = self._connection_from_session(
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/orm/query.py", line 3550, in _connection_from_session
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     conn = self.session.connection(**kw)
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/orm/session.py", line 1142, in connection
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     return self._connection_for_bind(
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/orm/session.py", line 1150, in _connection_for_bind
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     return self.transaction._connection_for_bind(
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/orm/session.py", line 433, in _connection_for_bind
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     conn = bind._contextual_connect()
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 2302, in _contextual_connect
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     self._wrap_pool_connect(self.pool.connect, None),
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 2339, in _wrap_pool_connect
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     Connection._handle_dbapi_exception_noconnection(
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 1581, in _handle_dbapi_exception_noconnection
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     util.raise_(newraise, with_traceback=exc_info[2], from_=e)
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     raise exception
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 2336, in _wrap_pool_connect
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     return fn()
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/pool/base.py", line 364, in connect
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     return _ConnectionFairy._checkout(self)
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/pool/base.py", line 778, in _checkout
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     fairy = _ConnectionRecord.checkout(pool)
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/pool/base.py", line 495, in checkout
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     rec = pool._do_get()
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/pool/impl.py", line 140, in _do_get
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     self._dec_overflow()
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     compat.raise_(
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     raise exception
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/pool/impl.py", line 137, in _do_get
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     return self._create_connection()
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/pool/base.py", line 309, in _create_connection
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     return _ConnectionRecord(self)
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/pool/base.py", line 440, in __init__
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     self.__connect(first_connect_check=True)
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/pool/base.py", line 661, in __connect
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     pool.logger.debug("Error on connect(): %s", e)
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     compat.raise_(
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     raise exception
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/pool/base.py", line 656, in __connect
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     connection = pool._invoke_creator(self)
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     return dialect.connect(*cargs, **cparams)
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/default.py", line 509, in connect
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     return self.dbapi.connect(*cargs, **cparams)
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib/python3.9/site-packages/pymysql/__init__.py", line 94, in Connect
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     return Connection(*args, **kwargs)
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 327, in __init__
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     self.connect()
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 587, in connect
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     self._get_server_information()
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 969, in _get_server_information
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     packet = self._read_packet()
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 646, in _read_packet
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     packet_header = self._read_bytes(4)
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager   File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 698, in _read_bytes
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager     raise err.OperationalError(
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager oslo_db.exception.DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query')
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager (Background on this error at: http://sqlalche.me/e/13/e3q8)
2023-08-01 09:33:02.914 105 ERROR neutron_lib.callbacks.manager
2023-08-01 09:33:02.941 105 INFO neutron.wsgi [-] (105) wsgi starting up on http://[::1]:9696
2023-08-01 09:33:03.789 105 DEBUG neutron.wsgi [-] (105) accepted ('::1', 39026, 0, 0) server /usr/lib/python3.9/site-packages/eventlet/wsgi.py:992
2023-08-01 09:33:03.797 105 INFO neutron.wsgi [-] 172.17.1.147,::1 "GET /healthcheck HTTP/1.1" status: 200  len: 142 time: 0.0064757
Comment 17 errata-xmlrpc 2023-09-20 00:29:52 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Release of components for Red Hat OpenStack Platform 17.1.1 (Wallaby)), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:5138
Note You need to log in before you can comment on or make changes to this bug.