Bug 1948459 - UpdateLRouterPortCommand sets networks to []
Summary: UpdateLRouterPortCommand sets networks to []
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: python-networking-ovn
Version: 16.1 (Train)
Hardware: Unspecified
OS: Unspecified
low
low
Target Milestone: z8
: 16.1 (Train on RHEL 8.2)
Assignee: Arnau Verdaguer
QA Contact: Eran Kuris
URL:
Whiteboard:
Depends On:
Blocks: 2036878
TreeView+ depends on / blocked
 
Reported: 2021-04-12 08:43 UTC by Jakub Libosvar
Modified: 2022-12-07 20:25 UTC (History)
6 users (show)

Fixed In Version: python-networking-ovn-7.3.1-1.20220127153446.4e24f4c.el8ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 2036878 (view as bug list)
Environment:
Last Closed: 2022-12-07 20:24:45 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1948457 0 None None None 2021-11-08 14:29:42 UTC
Red Hat Issue Tracker OSP-2297 0 None None None 2021-11-17 09:37:22 UTC
Red Hat Product Errata RHBA-2022:8795 0 None None None 2022-12-07 20:25:37 UTC

Description Jakub Libosvar 2021-04-12 08:43:16 UTC 
Description of problem:
This was spotted on production. There is a port update call on logical router port that causes following error:
2021-04-12 08:19:31.571 45 ERROR ovsdbapp.backend.ovs_idl.vlog [-] attempting to write bad value to column networks (ovsdb error: 0 values when type requires between 1 and 9223372036854775807): ovs.db.error.Error: ovsdb error: 0 values when type requires between 1 and 9223372036854775807

It is caused by 2021-04-12 08:19:31.561 45 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(name=lrp-f4e4f2c7-fdb2-402b-a3dd-c57f77a71c21, columns={'external_ids': {'neutron:revision_number': '3', 'neutron:subnet_ids': '', 'neutron:network_name': 'neutron-addc0895-6bac-44c0-ab8c-acad2c5136ed', 'neutron:router_name': '8ea71d05-dd8a-4659-a043-a7c5421f4b2a'}, 'options': {}, 'networks': [], 'ipv6_ra_configs': {}}, if_exists=True) do_commit /usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:84

command that sets 'networks': [] which is not allowed value

Version-Release number of selected component (if applicable):
python3-networking-ovn-7.3.1-1.20200902233413.el8ost.noarch

How reproducible:
Not sure

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

2021-04-12 08:19:30.433 45 DEBUG neutron.api.v2.base [req-16c0255c-6847-4c24-af88-00689e05039b 90a7eee68827d01de68e9928934aa42fbfc7a0831e88943f852da9a90ca46d2b 926599aed01742beac38883731aaa00d - 62cf1b5ec006489db
99e2b0ebfb55f57 62cf1b5ec006489db99e2b0ebfb55f57] Request body: {'port': {'fixed_ips': [], 'name': 'f4e4f2c7-fdb2-402b-a3dd-c57f77a71c21', 'admin_state_up': False}} prepare_request_body /usr/lib/python3.6/site-pa
ckages/neutron/api/v2/base.py:719
2021-04-12 08:19:30.619 45 DEBUG neutron_lib.callbacks.manager [req-16c0255c-6847-4c24-af88-00689e05039b 90a7eee68827d01de68e9928934aa42fbfc7a0831e88943f852da9a90ca46d2b 926599aed01742beac38883731aaa00d - 62cf1b5
ec006489db99e2b0ebfb55f57 62cf1b5ec006489db99e2b0ebfb55f57] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler--9223372036844294712'] for port, before_update
 _notify_loop /usr/lib/python3.6/site-packages/neutron_lib/callbacks/manager.py:193
2021-04-12 08:19:30.945 45 DEBUG neutron.db.db_base_plugin_common [req-16c0255c-6847-4c24-af88-00689e05039b 90a7eee68827d01de68e9928934aa42fbfc7a0831e88943f852da9a90ca46d2b 926599aed01742beac38883731aaa00d - 62cf
1b5ec006489db99e2b0ebfb55f57 62cf1b5ec006489db99e2b0ebfb55f57] Delete allocated IP 172.16.1.1 (addc0895-6bac-44c0-ab8c-acad2c5136ed/ee7e4d07-d2d5-46b3-bb25-f90c46b4d496) _delete_ip_allocation /usr/lib/python3.6/s
ite-packages/neutron/db/db_base_plugin_common.py:123
2021-04-12 08:19:31.084 45 DEBUG neutron_lib.callbacks.manager [req-16c0255c-6847-4c24-af88-00689e05039b 90a7eee68827d01de68e9928934aa42fbfc7a0831e88943f852da9a90ca46d2b 926599aed01742beac38883731aaa00d - 62cf1b5
ec006489db99e2b0ebfb55f57 62cf1b5ec006489db99e2b0ebfb55f57] Notify callbacks ['neutron.services.qos.qos_plugin.QoSPlugin._validate_update_port_callback-126327294'] for port, precommit_update _notify_loop /usr/lib
/python3.6/site-packages/neutron_lib/callbacks/manager.py:193
2021-04-12 08:19:31.084 45 DEBUG networking_ovn.ml2.mech_driver [req-16c0255c-6847-4c24-af88-00689e05039b 90a7eee68827d01de68e9928934aa42fbfc7a0831e88943f852da9a90ca46d2b 926599aed01742beac38883731aaa00d - 62cf1b
5ec006489db99e2b0ebfb55f57 62cf1b5ec006489db99e2b0ebfb55f57] No provisioning block for port f4e4f2c7-fdb2-402b-a3dd-c57f77a71c21 since it does not have a host _is_port_provisioning_required /usr/lib/python3.6/sit
e-packages/networking_ovn/ml2/mech_driver.py:515
2021-04-12 08:19:31.092 45 DEBUG neutron_lib.callbacks.manager [req-16c0255c-6847-4c24-af88-00689e05039b 90a7eee68827d01de68e9928934aa42fbfc7a0831e88943f852da9a90ca46d2b 926599aed01742beac38883731aaa00d - 62cf1b5
ec006489db99e2b0ebfb55f57 62cf1b5ec006489db99e2b0ebfb55f57] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change--9223372036844294745', 'neutron.plugins.ml2.e
xtensions.dns_integration._update_port_in_external_dns_service--9223363271428709338', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event--9223372036851573210', 'neutron.api.rpc.agentnotifiers.dhcp_rpc
_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-126673939', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed--9223363271428404732', 'networking_ovn.l3.l3_ovn.OVNL3RouterPlug
in._port_update-8765418206903', 'neutron.services.trunk.plugin.TrunkPlugin._trigger_trunk_status_change--9223372036854282501', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-129698338'
, 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-225567489'] for port, after_update _notify_loop /usr/lib/python3.6/site-packages/neutron_lib/callbacks/manager.py:193
2021-04-12 08:19:31.102 45 DEBUG oslo_concurrency.lockutils [req-1140afbb-d3f3-485d-a3e3-ce76c91311ae c875199b65e93d82f221fc4c4b6fb6907b8ed345c87f782936c20559b6979045 ebcbee8b5de946ff87b388031a57b15e - - -] Lock
"event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:327
2021-04-12 08:19:31.197 45 DEBUG neutron.api.rpc.handlers.resources_rpc [req-16c0255c-6847-4c24-af88-00689e05039b 90a7eee68827d01de68e9928934aa42fbfc7a0831e88943f852da9a90ca46d2b 926599aed01742beac38883731aaa00d
- - -] Pushing event updated for resources: {'Port': ['ID=f4e4f2c7-fdb2-402b-a3dd-c57f77a71c21,revision_number=3']} push /usr/lib/python3.6/site-packages/neutron/api/rpc/handlers/resources_rpc.py:243
2021-04-12 08:19:31.444 45 DEBUG oslo_concurrency.lockutils [req-16c0255c-6847-4c24-af88-00689e05039b 90a7eee68827d01de68e9928934aa42fbfc7a0831e88943f852da9a90ca46d2b 926599aed01742beac38883731aaa00d - - -] Lock
"event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.343s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:339
2021-04-12 08:19:31.455 45 DEBUG neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api [req-16c0255c-6847-4c24-af88-00689e05039b 90a7eee68827d01de68e9928934aa42fbfc7a0831e88943f852da9a90ca46d2b 926599aed01742beac38883731aaa00d - 62cf1b5ec006489db99e2b0ebfb55f57 62cf1b5ec006489db99e2b0ebfb55f57] Network addc0895-6bac-44c0-ab8c-acad2c5136ed is not hosted by any dhcp agent _notify_agents /usr/lib/python3.6/site-packages/neutron/api/rpc/agentnotifiers/dhcp_rpc_agent_api.py:218
2021-04-12 08:19:31.551 45 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(name=lrp-f4e4f2c7-fdb2-402b-a3dd-c57f77a71c21, resource={'id': 'f4e4f2c7-fdb2-402b-a3dd-c57f77a71c21', 'name': 'f4e4f2c7-fdb2-402b-a3dd-c57f77a71c21', 'network_id': 'addc0895-6bac-44c0-ab8c-acad2c5136ed', 'tenant_id': '926599aed01742beac38883731aaa00d', 'mac_address': 'fa:16:3e:66:c1:e1', 'admin_state_up': False, 'status': 'DOWN', 'device_id': '8ea71d05-dd8a-4659-a043-a7c5421f4b2a', 'device_owner': 'network:router_interface', 'fixed_ips': [], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'qos_policy_id': None, 'port_security_enabled': False, 'dns_name': '', 'dns_assignment': [], 'resource_request': None, 'ip_allocation': 'immediate', 'tags': [], 'created_at': '2021-04-12T08:08:32Z', 'updated_at': '2021-04-12T08:19:30Z', 'revision_number': 3, 'project_id': '926599aed01742beac38883731aaa00d'}, resource_type=router_ports, if_exists=True) do_commit /usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:84
2021-04-12 08:19:31.561 45 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(name=lrp-f4e4f2c7-fdb2-402b-a3dd-c57f77a71c21, columns={'external_ids': {'neutron:revision_number': '3', 'neutron:subnet_ids': '', 'neutron:network_name': 'neutron-addc0895-6bac-44c0-ab8c-acad2c5136ed', 'neutron:router_name': '8ea71d05-dd8a-4659-a043-a7c5421f4b2a'}, 'options': {}, 'networks': [], 'ipv6_ra_configs': {}}, if_exists=True) do_commit /usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:84
2021-04-12 08:19:31.571 45 ERROR ovsdbapp.backend.ovs_idl.vlog [-] attempting to write bad value to column networks (ovsdb error: 0 values when type requires between 1 and 9223372036854775807): ovs.db.error.Error: ovsdb error: 0 values when type requires between 1 and 9223372036854775807
2021-04-12 08:19:31.572 45 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(lswitch_port=f4e4f2c7-fdb2-402b-a3dd-c57f77a71c21, lrouter_port=lrp-f4e4f2c7-fdb2-402b-a3dd-c57f77a71c21, is_gw_port=False, if_exists=True, lsp_address=router) do_commit /usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:84
2021-04-12 08:19:31.742 45 INFO networking_ovn.db.revision [req-16c0255c-6847-4c24-af88-00689e05039b 90a7eee68827d01de68e9928934aa42fbfc7a0831e88943f852da9a90ca46d2b 926599aed01742beac38883731aaa00d - 62cf1b5ec00
6489db99e2b0ebfb55f57 62cf1b5ec006489db99e2b0ebfb55f57] Successfully bumped revision number for resource f4e4f2c7-fdb2-402b-a3dd-c57f77a71c21 (type: router_ports) to 3
2021-04-12 08:19:32.094 45 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(name=f4e4f2c7-fdb2-402b-a3dd-c57f77a71c21, resource={'id': 'f4e4f2c7-fdb2-402b-a3dd-c57f77a71c21', 'name': 'f4e4f2c7-fdb2-402b-a3dd-c57f77a71c21', 'network_id': 'addc0895-6bac-44c0-ab8c-acad2c5136ed', 'tenant_id': '926599aed01742beac38883731aaa00d', 'mac_address': 'fa:16:3e:66:c1:e1', 'admin_state_up': False, 'status': 'DOWN', 'device_id': '8ea71d05-dd8a-4659-a043-a7c5421f4b2a', 'device_owner': 'network:router_interface', 'fixed_ips': [], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'qos_policy_id': None, 'port_security_enabled': False, 'dns_name': '', 'dns_assignment': [], 'resource_request': None, 'ip_allocation': 'immediate', 'tags': [], 'created_at': '2021-04-12T08:08:32Z', 'updated_at': '2021-04-12T08:19:30Z', 'revision_number': 3, 'project_id': '926599aed01742beac38883731aaa00d', 'network': {'id': 'addc0895-6bac-44c0-ab8c-acad2c5136ed', 'name': 'swift-58-1618214844-unit-management', 'tenant_id': '926599aed01742beac38883731aaa00d', 'admin_state_up': True, 'mtu': 1442, 'status': 'ACTIVE', 'subnets': ['ee7e4d07-d2d5-46b3-bb25-f90c46b4d496'], 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'vlan_transparent': None, 'description': '', 'qos_policy_id': None, 'port_security_enabled': False, 'dns_domain': '', 'l2_adjacency': True, 'tags': [], 'created_at': '2021-04-12T08:07:52Z', 'updated_at': '2021-04-12T08:19:14Z', 'revision_number': 4, 'project_id': '926599aed01742beac38883731aaa00d', 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 184}}, resource_type=ports, if_exists=True) do_commit /usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:84
2021-04-12 08:19:32.212 45 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLSwitchPortCommand(lport=f4e4f2c7-fdb2-402b-a3dd-c57f77a71c21, columns={'external_ids': {'neutron:port_name': 'f4e4f2c7-fdb2-402b-a3dd-c57f77a71c21', 'neutron:device_id': '8ea71d05-dd8a-4659-a043-a7c5421f4b2a', 'neutron:project_id': '926599aed01742beac38883731aaa00d', 'neutron:cidrs': '', 'neutron:device_owner': 'network:router_interface', 'neutron:network_name': 'neutron-addc0895-6bac-44c0-ab8c-acad2c5136ed', 'neutron:security_group_ids': '', 'neutron:revision_number': '3'}, 'parent_name': [], 'tag': [], 'options': {'requested-chassis': '', 'router-port': 'lrp-f4e4f2c7-fdb2-402b-a3dd-c57f77a71c21'}, 'enabled': False, 'port_security': [], 'dhcpv4_options': [], 'dhcpv6_options': [], 'ha_chassis_group': []}, if_exists=False) do_commit /usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:84
2021-04-12 08:19:32.332 45 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): PgDelPortCommand(port_group=neutron_pg_drop, lsp=['f4e4f2c7-fdb2-402b-a3dd-c57f77a71c21'], if_exists=False) do_commit /usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:84
2021-04-12 08:19:32.544 45 INFO networking_ovn.db.revision [req-16c0255c-6847-4c24-af88-00689e05039b 90a7eee68827d01de68e9928934aa42fbfc7a0831e88943f852da9a90ca46d2b 926599aed01742beac38883731aaa00d - 62cf1b5ec006489db99e2b0ebfb55f57 62cf1b5ec006489db99e2b0ebfb55f57] Successfully bumped revision number for resource f4e4f2c7-fdb2-402b-a3dd-c57f77a71c21 (type: ports) to 3
2021-04-12 08:19:32.558 45 DEBUG neutron.plugins.ml2.plugin [req-16c0255c-6847-4c24-af88-00689e05039b 90a7eee68827d01de68e9928934aa42fbfc7a0831e88943f852da9a90ca46d2b 926599aed01742beac38883731aaa00d - 62cf1b5ec006489db99e2b0ebfb55f57 62cf1b5ec006489db99e2b0ebfb55f57] In _notify_port_updated(), no bound segment for port f4e4f2c7-fdb2-402b-a3dd-c57f77a71c21 on network addc0895-6bac-44c0-ab8c-acad2c5136ed _notify_port_updated /usr/lib/python3.6/site-packages/neutron/plugins/ml2/plugin.py:823
2021-04-12 08:19:32.562 45 DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-16c0255c-6847-4c24-af88-00689e05039b 90a7eee68827d01de68e9928934aa42fbfc7a0831e88943f852da9a90ca46d2b 926599aed01742beac38883731aaa00d - 62cf1b5ec006489db99e2b0ebfb55f57 62cf1b5ec006489db99e2b0ebfb55f57] Attributes excluded by policy engine: ['binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'resource_request'] _exclude_attributes_by_policy /usr/lib/python3.6/site-packages/neutron/pecan_wsgi/hooks/policy_enforcement.py:256
2021-04-12 08:19:32.562 45 DEBUG neutron_lib.callbacks.manager [req-16c0255c-6847-4c24-af88-00689e05039b 90a7eee68827d01de68e9928934aa42fbfc7a0831e88943f852da9a90ca46d2b 926599aed01742beac38883731aaa00d - 62cf1b5ec006489db99e2b0ebfb55f57 62cf1b5ec006489db99e2b0ebfb55f57] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification--9223372036852734677'] for port, before_response _notify_loop /usr/lib/python3.6/site-packages/neutron_lib/callbacks/manager.py:193
2021-04-12 08:19:32.564 45 INFO neutron.wsgi [req-16c0255c-6847-4c24-af88-00689e05039b 90a7eee68827d01de68e9928934aa42fbfc7a0831e88943f852da9a90ca46d2b 926599aed01742beac38883731aaa00d - 62cf1b5ec006489db99e2b0ebfb55f57 62cf1b5ec006489db99e2b0ebfb55f57] 172.16.21.60 "PUT /v2.0/ports/f4e4f2c7-fdb2-402b-a3dd-c57f77a71c21.json HTTP/1.1" status: 200  len: 933 time: 2.1345954
Comment 2 Arnau Verdaguer 2021-11-08 12:28:06 UTC 
How to reproduce it:

1. Create a network and a subnet
2. Create a router
3. Attach subnet to router
4. Remove IP from the router port

This will cause to update the port and remove the only IP that it has. Since it's a network:router_port owned this will cause to
call ovn_client.update_router_port, who will finally call update_lrouter_port with an empty network list. This will cause an error
since the network column has a constraint where the value must be greater than 0.
Comment 3 Rodolfo Alonso 2021-11-08 14:27:54 UTC 
Hi Arnau:

Thanks for the update, I didn't find it before. I'll update the U/S launchpad bug with this info.

Regards.
Comment 12 errata-xmlrpc 2022-12-07 20:24:45 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform 16.1.9 bug fix and enhancement advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:8795
Note You need to log in before you can comment on or make changes to this bug.