Bug 2142589 - KeyError when creating a router connected to a vlan provider network
Summary: KeyError when creating a router connected to a vlan provider network
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: ovn-bgp-agent
Version: 17.1 (Wallaby)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: beta
: 17.1
Assignee: Luis Tomas Bolivar
QA Contact: Eduardo Olivares
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-11-14 14:11 UTC by Eduardo Olivares
Modified: 2023-08-16 01:12 UTC (History)
3 users (show)

Fixed In Version: ovn-bgp-agent-0.3.1-1.20221116111131.4e87e63.el9ost
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-08-16 01:12:28 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 864271 0 None MERGED Fix KeyError at add_ip_route for provider vlan networks 2022-11-14 14:31:52 UTC
Red Hat Issue Tracker OSP-20152 0 None None None 2022-11-14 14:23:06 UTC
Red Hat Product Errata RHEA-2023:4577 0 None None None 2023-08-16 01:12:52 UTC

Description Eduardo Olivares 2022-11-14 14:11:42 UTC
Description of problem:
An Exception was raised when the following router, connected to a provider-vlan network, was created:
TASK [create router for provider-vlan] ***************************************
task path: /root/bgp-playbooks-templates/playbooks/frrempest.yml:191            
changed: [undercloudbgp-0] => {"changed": true, "router": {"admin_state_up": true, "availability_zone_hints": [], "availability_zones": [], "created_at": "2022-11-10T14:22:03Z", "description": "", "external_gateway_info": {"enable_snat":
true, "external_fixed_ips": [{"ip_address": "172.24.200.92", "subnet_id": "ce0620c8-8dae-42a8-b61b-3ba3c6b89eaa"}, {"ip_address": "2001:db9::f816:3eff:fe19:d969", "subnet_id": "b1b35c3f-879a-4ccf-b9f8-6691c861bb12"}], "network_id": "30b18c76-c4c3-4d60-a601-b97d840a527a"}, "id": "aa6e811b-5e93-4884-aa5b-8696bcd38b38", "name": "router-vlan", "project_id": "29cae679482745afbc9b1851d85ee6b6", "revision_number": 3, "routes": [], "status": "ACTIVE", "tags": [], "tenant_id": "29cae679482745afbc9b1851d85ee6b6", "updated_at": "2022-11-10T14:22:04Z"}}


As you can see, the task finishes successfully, but the ovn-bgp-agent throws and exception.
/var/log/containers/stdouts/ovn_bgp_agent.log.1@ctrl-1-0
----------------------------------------------------------
2022-11-10T14:22:05.668363568+00:00 stdout F 2022-11-10 14:22:05.668 99396 DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched UPDATE: TenantPortCreatedEvent(events=('update',), table='Port_Binding', conditions=None, old_conditions=None) to 
row=Port_Binding(mac=['fa:16:3e:19:d9:69 172.24.200.92/24 2001:db9::f816:3eff:fe19:d969/64'], nat_addresses=[], type=chassisredirect, virtual_parent=[], up=[True], options={'always-redirect': 'true', 'distributed-port': 'lrp-33f102ea-03ed
-41fd-b35a-c7695918ee19'}, parent_port=[], external_ids={'neutron:network_name': 'neutron-30b18c76-c4c3-4d60-a601-b97d840a527a', 'neutron:revision_number': '1', 'neutron:router_name': 'aa6e811b-5e93-4884-aa5b-8696bcd38b38', 'neutron:subne
t_ids': 'ce0620c8-8dae-42a8-b61b-3ba3c6b89eaa b1b35c3f-879a-4ccf-b9f8-6691c861bb12'}, tag=[], encap=[], chassis=[<ovs.db.idl.Row object at 0x7f5c515b4790>], datapath=0222d076-ed25-4f82-9678-cafbcc418186, gateway_chassis=[], tunnel_key=2, 
logical_port=cr-lrp-33f102ea-03ed-41fd-b35a-c7695918ee19, requested_chassis=[]) old=Port_Binding(up=[False], chassis=[]) matches /usr/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/event.py:43ESC[00m
2022-11-10T14:22:05.668897688+00:00 stdout F 2022-11-10 14:22:05.668 99396 DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched UPDATE: PortBindingChassisCreatedEvent(events=('update',), table='Port_Binding', conditions=None, old_conditions=N
one) to row=Port_Binding(mac=['fa:16:3e:19:d9:69 172.24.200.92/24 2001:db9::f816:3eff:fe19:d969/64'], nat_addresses=[], type=chassisredirect, virtual_parent=[], up=[True], options={'always-redirect': 'true', 'distributed-port': 'lrp-33f10
2ea-03ed-41fd-b35a-c7695918ee19'}, parent_port=[], external_ids={'neutron:network_name': 'neutron-30b18c76-c4c3-4d60-a601-b97d840a527a', 'neutron:revision_number': '1', 'neutron:router_name': 'aa6e811b-5e93-4884-aa5b-8696bcd38b38', 'neutr
on:subnet_ids': 'ce0620c8-8dae-42a8-b61b-3ba3c6b89eaa b1b35c3f-879a-4ccf-b9f8-6691c861bb12'}, tag=[], encap=[], chassis=[<ovs.db.idl.Row object at 0x7f5c515b4790>], datapath=0222d076-ed25-4f82-9678-cafbcc418186, gateway_chassis=[], tunnel
_key=2, logical_port=cr-lrp-33f102ea-03ed-41fd-b35a-c7695918ee19, requested_chassis=[]) old=Port_Binding(up=[False], chassis=[]) matches /usr/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/event.py:43ESC[00m
2022-11-10T14:22:05.669418224+00:00 stdout F 2022-11-10 14:22:05.669 99396 DEBUG oslo_concurrency.lockutils [-] Lock "bgp" acquired by "ovn_bgp_agent.drivers.openstack.ovn_bgp_driver.OVNBGPDriver.expose_ip" :: waited 0.000s inner /usr/lib
...
2022-11-10T14:22:06.107510357+00:00 stdout F 2022-11-10 14:22:06.107 99396 DEBUG oslo_concurrency.lockutils [-] Lock "bgp" released by "ovn_bgp_agent.drivers.openstack.ovn_bgp_driver.OVNBGPDriver.expose_ip" :: held 0.438s inner /usr/lib/python3.9/site-packages/oslo_concurrency/lockutils.py:367ESC[00m
2022-11-10T14:22:06.108992207+00:00 stdout F 2022-11-10 14:22:06.107 99396 ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: KeyError: 'object does not exists'                                                                  
2022-11-10T14:22:06.108992207+00:00 stdout F 2022-11-10 14:22:06.107 99396 ERROR ovsdbapp.event Traceback (most recent call last):                                                                                                           
2022-11-10T14:22:06.108992207+00:00 stdout F 2022-11-10 14:22:06.107 99396 ERROR ovsdbapp.event   File "/usr/lib/python3.9/site-packages/ovsdbapp/event.py", line 159, in notify_loop                                                        
2022-11-10T14:22:06.108992207+00:00 stdout F 2022-11-10 14:22:06.107 99396 ERROR ovsdbapp.event     match.run(event, row, updates)                                                                                                           
2022-11-10T14:22:06.108992207+00:00 stdout F 2022-11-10 14:22:06.107 99396 ERROR ovsdbapp.event   File "/usr/lib/python3.9/site-packages/ovn_bgp_agent/drivers/openstack/watchers/bgp_watcher.py", line 52, in run                           
2022-11-10T14:22:06.108992207+00:00 stdout F 2022-11-10 14:22:06.107 99396 ERROR ovsdbapp.event     self.agent.expose_ip(ips, row)                                                                                                           
2022-11-10T14:22:06.108992207+00:00 stdout F 2022-11-10 14:22:06.107 99396 ERROR ovsdbapp.event   File "/usr/lib/python3.9/site-packages/oslo_concurrency/lockutils.py", line 360, in inner                                                  
2022-11-10T14:22:06.108992207+00:00 stdout F 2022-11-10 14:22:06.107 99396 ERROR ovsdbapp.event     return f(*args, **kwargs)                                                                                                                
2022-11-10T14:22:06.108992207+00:00 stdout F 2022-11-10 14:22:06.107 99396 ERROR ovsdbapp.event   File "/usr/lib/python3.9/site-packages/ovn_bgp_agent/drivers/openstack/ovn_bgp_driver.py", line 480, in expose_ip                          
2022-11-10T14:22:06.108992207+00:00 stdout F 2022-11-10 14:22:06.107 99396 ERROR ovsdbapp.event     self._expose_ip(ips, row, associated_port)                                                                                               
2022-11-10T14:22:06.108992207+00:00 stdout F 2022-11-10 14:22:06.107 99396 ERROR ovsdbapp.event   File "/usr/lib/python3.9/site-packages/ovn_bgp_agent/drivers/openstack/ovn_bgp_driver.py", line 597, in _expose_ip                         
2022-11-10T14:22:06.108992207+00:00 stdout F 2022-11-10 14:22:06.107 99396 ERROR ovsdbapp.event     linux_net.add_ip_route(                                                                                                                  
2022-11-10T14:22:06.108992207+00:00 stdout F 2022-11-10 14:22:06.107 99396 ERROR ovsdbapp.event   File "/usr/lib/python3.9/site-packages/ovn_bgp_agent/utils/linux_net.py", line 492, in add_ip_route                                        
2022-11-10T14:22:06.108992207+00:00 stdout F 2022-11-10 14:22:06.107 99396 ERROR ovsdbapp.event     oif = ndb.interfaces[oif_name]['index']                                                                                                  
2022-11-10T14:22:06.108992207+00:00 stdout F 2022-11-10 14:22:06.107 99396 ERROR ovsdbapp.event   File "/usr/lib/python3.9/site-packages/pr2modules/ndb/auth_manager.py", line 69, in guard                                                  
2022-11-10T14:22:06.108992207+00:00 stdout F 2022-11-10 14:22:06.107 99396 ERROR ovsdbapp.event     return f(obj, *argv, **kwarg)                                                                                                            
2022-11-10T14:22:06.108992207+00:00 stdout F 2022-11-10 14:22:06.107 99396 ERROR ovsdbapp.event   File "/usr/lib/python3.9/site-packages/pr2modules/ndb/main.py", line 557, in __getitem__                                                   
2022-11-10T14:22:06.108992207+00:00 stdout F 2022-11-10 14:22:06.107 99396 ERROR ovsdbapp.event     ret = self.template(key, table)                                                                                                          
2022-11-10T14:22:06.108992207+00:00 stdout F 2022-11-10 14:22:06.107 99396 ERROR ovsdbapp.event   File "/usr/lib/python3.9/site-packages/pr2modules/ndb/main.py", line 394, in template                                                      
2022-11-10T14:22:06.108992207+00:00 stdout F 2022-11-10 14:22:06.107 99396 ERROR ovsdbapp.event     return iclass(self,
2022-11-10T14:22:06.108992207+00:00 stdout F 2022-11-10 14:22:06.107 99396 ERROR ovsdbapp.event   File "/usr/lib/python3.9/site-packages/pr2modules/ndb/objects/interface.py", line 431, in __init__                                         
2022-11-10T14:22:06.108992207+00:00 stdout F 2022-11-10 14:22:06.107 99396 ERROR ovsdbapp.event     super(Interface, self).__init__(*argv, **kwarg)
2022-11-10T14:22:06.108992207+00:00 stdout F 2022-11-10 14:22:06.107 99396 ERROR ovsdbapp.event   File "/usr/lib/python3.9/site-packages/pr2modules/ndb/objects/__init__.py", line 327, in __init__
2022-11-10T14:22:06.108992207+00:00 stdout F 2022-11-10 14:22:06.107 99396 ERROR ovsdbapp.event     raise KeyError('object does not exists')
2022-11-10T14:22:06.108992207+00:00 stdout F 2022-11-10 14:22:06.107 99396 ERROR ovsdbapp.event KeyError: 'object does not exists'
2022-11-10T14:22:06.108992207+00:00 stdout F 2022-11-10 14:22:06.107 99396 ERROR ovsdbapp.event ESC[00m





This is the task that reproduces this bug:
  - name: create router for provider-vlan
    openstack.cloud.router:
      cloud: overcloud
      name: router-vlan
      state: present
      network: provider-vlan
      interfaces:
        - private-vlan-subnet
        - private-vlan-subnet-ipv6

Comment 1 Luis Tomas Bolivar 2022-11-14 14:24:19 UTC
Fix: https://review.opendev.org/c/x/ovn-bgp-agent/+/864271

Comment 16 errata-xmlrpc 2023-08-16 01:12:28 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Release of components for Red Hat OpenStack Platform 17.1 (Wallaby)), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2023:4577


Note You need to log in before you can comment on or make changes to this bug.