Bug 2128253
| Summary: | [16.2] neutron-ovn-db-sync-util failed on one of the controller nodes during OVN migration | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Roman Safronov <rsafrono> |
| Component: | python-networking-ovn | Assignee: | OSP Team <rhos-maint> |
| Status: | CLOSED DUPLICATE | QA Contact: | Eran Kuris <ekuris> |
| Severity: | high | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 16.2 (Train) | CC: | apevec, averdagu, lhh, majopela, scohen |
| Target Milestone: | --- | Keywords: | Regression |
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2022-09-20 15:06:40 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
*** This bug has been marked as a duplicate of bug 2087721 *** |
Description of problem: neutron-ovn-db-sync-util failed on one of the controller nodes during OVN migration, This caused that OVN migration process stopped unexpectedly. From the OVN migration log TASK [migration : Sync neutron db with OVN db (container) - Run 1] ************* task path: /home/stack/ovn_migration/playbooks/roles/migration/tasks/sync-dbs.yml:7 Tuesday 20 September 2022 10:59:51 +0000 (0:00:01.209) 0:37:34.880 ***** META: noop META: noop fatal: [controller-0]: FAILED! => {"changed": true, "cmd": ["podman", "exec", "7bf2452aaa20", "neutron-ovn-db-sync-util", "--config-file", "/etc/neutron/neutron.conf", "--config-file", "/etc/neutron/plugins/ml2/ml2_conf.ini", "--ovn-neutron_sync_mode", "migrate"], "delta": "0:00:26.088154", "end": "2022-09-20 11:00:17.840082", "msg": "non-zero return code", "rc": 1, "start": "2022-09-20 10:59:51.751928", "stderr": "Deprecated: Option \"vif_type\" from group \"ovn\" is deprecated for removal (The port VIF type is now determined based on the OVN chassis information when the port is bound to a host.). Its value may be silently ignored in the future.\nDeprecated: Option \"ovn_l3_mode\" from group \"ovn\" is deprecated for removal (This option is no longer used. Native L3 support in OVN is always used.). Its value may be silently ignored in the future.", "stderr_lines": ["Deprecated: Option \"vif_type\" from group \"ovn\" is deprecated for removal (The port VIF type is now determined based on the OVN chassis information when the port is bound to a host.). Its value may be silently ignored in the future.", "Deprecated: Option \"ovn_l3_mode\" from group \"ovn\" is deprecated for removal (This option is no longer used. Native L3 support in OVN is always used.). Its value may be silently ignored in the future."], "stdout": "", "stdout_lines": []} from the neutron_ovn_db_sync_util log on controller-0 2022-09-20 11:00:16.545 37 INFO networking_ovn.cmd.neutron_ovn_db_sync_util [req-bb69d394-b6f3-46dc-8b84-28515df88278 - - - - -] Migrating Neutron database from OVS to OVN 2022-09-20 11:00:16.820 37 CRITICAL neutron_ovn_db_sync_util [req-bb69d394-b6f3-46dc-8b84-28515df88278 - - - - -] Unhandled error: sqlalchemy.exc.InvalidRequestError: This session is in 'ina ctive' state, due to the SQL transaction being rolled back; no further SQL can be emitted within this transaction. 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util Traceback (most recent call last): 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util File "/usr/bin/neutron-ovn-db-sync-util", line 10, in <module> 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util sys.exit(main()) 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util File "/usr/lib/python3.6/site-packages/networking_ovn/cmd/neutron_ovn_db_sync_util.py", line 235, in main 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util db_migration.migrate_neutron_database_to_ovn(core_plugin) 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util File "/usr/lib/python3.6/site-packages/networking_ovn/ml2/db_migration.py", line 98, in migrate_neutron_database_to_ovn 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util pb_missed.add(port_id) 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util File "/usr/lib64/python3.6/contextlib.py", line 88, in __exit__ 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util next(self.gen) 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util File "/usr/lib/python3.6/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 1065, in _transaction_scope 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util yield resource 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util File "/usr/lib64/python3.6/contextlib.py", line 88, in __exit__ 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util next(self.gen) 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util File "/usr/lib/python3.6/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util self.session.rollback() 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util self.force_reraise() 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util six.reraise(self.type_, self.value, self.tb) 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util raise value 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util File "/usr/lib/python3.6/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 664, in _session 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util self._end_session_transaction(self.session) 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util File "/usr/lib/python3.6/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 692, in _end_session_transaction 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util session.commit() 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/session.py", line 1026, in commit 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util self.transaction.commit() 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/session.py", line 491, in commit 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util self._assert_active(prepared_ok=True) 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/session.py", line 298, in _assert_active 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util "This session is in 'inactive' state, due to the " 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util sqlalchemy.exc.InvalidRequestError: This session is in 'inactive' state, due to the SQL transaction being rolled back; no further SQL can be emitted within this transaction. 2022-09-20 11:00:16.820 37 ERROR neutron_ovn_db_sync_util Version-Release number of selected component (if applicable): RHOS-16.2-RHEL-8-20220902.n.1 How reproducible: Failed once for now Steps to Reproduce: 1.Deploy HA environment with ml2/ovs backend 2.Create an internal network, router, connect the internal network to external via the router, create a vm connected to the internal network 3.Start ovn migration according to the official procedure https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.2/html-single/migrating_the_networking_service_to_the_ml2ovn_mechanism_driver/index Actual results: OVN migration failed on "Sync neutron db with OVN db" stage Expected results: OVN migration completed successfully Additional info: