Bug 2188326 - Route to FIP exposed/withdrawn by the sync process
Summary: Route to FIP exposed/withdrawn by the sync process
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: ovn-bgp-agent
Version: 17.1 (Wallaby)
Hardware: Unspecified
OS: Unspecified
urgent
high
Target Milestone: ga
: 17.1
Assignee: Luis Tomas Bolivar
QA Contact: Eduardo Olivares
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-04-20 13:35 UTC by Eduardo Olivares
Modified: 2023-08-16 01:15 UTC (History)
3 users (show)

Fixed In Version: ovn-bgp-agent-0.3.1-1.20230324161202.ddb740e.el9ost
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-08-16 01:14:48 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 880528 0 None MERGED Ensure FIPs events are properly handled 2023-04-21 10:24:05 UTC
Red Hat Issue Tracker OSP-24421 0 None None None 2023-04-20 13:43:10 UTC
Red Hat Product Errata RHEA-2023:4577 0 None None None 2023-08-16 01:15:18 UTC

Description Eduardo Olivares 2023-04-20 13:35:54 UTC
Description of problem:
When a FIP is attached to a VM's port, the ovn bgp agent should expose the route to its IP immediately. However, this was not working like that, but the sync process was exposing the FIP some time later (depending on the sync interval configured, 2 minutes by default).

The same happens when the FIP is deleted (or dettached from the VM's port).

Apparently, this issue was introduced with the following patch:
https://opendev.org/openstack/ovn-bgp-agent/commit/d9c7de64650a8a0c31172a1fd50200ae41b6281f


Version-Release number of selected component (if applicable):
RHOS-17.1-RHEL-9-20230419.n.1

How reproducible:
100%

Steps to Reproduce:
1. create a VM connected to a tenant network and add a FIP to it
2. check the routes on the extvm or undercloud or simply check the IPs added to the dummy interface bgp-nic on the compute where the VM is running


Actual results:
The IP is exposed/withdrawn some time later, it may take up to 2 mins (default sync interval).


Expected results:
The IP should be exposed/withdrawn immediately when a FIP is attached/detached to/from a VM's port.

Comment 7 Eduardo Olivares 2023-04-28 10:56:07 UTC
Verified on RHOS-17.1-RHEL-9-20230426.n.1.

1) Add a FIP to a VM:
$ openstack server add floating ip kyrhrvei 172.24.100.211
As soon as the FIP is added, it is exposed.

The ovn-bgp-agent logs show that as soon as the FIPSetEvent is detected, the FIP is exposed:
2023-04-28T10:45:50.616174461+00:00 stdout F 2023-04-28 10:45:50.615 3139 DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched UPDATE: FIPSetEvent(events=('update',), table='Port_Binding', conditions=None, old_conditions=None) to row=Port_Bin
ding(mac=['router'], port_security=[], type=patch, nat_addresses=['fa:16:3e:0b:a4:8e 172.24.100.75 is_chassis_resident("5d4a9317-0e15-4993-b1d0-0cb854112390")', 'fa:16:3e:3e:33:22 172.24.100.56 is_chassis_resident("4773e234-993c-46f8-a5f7
-2ea4ced5b4f8")', 'fa:16:3e:57:90:35 172.24.100.171 172.24.100.10 172.24.100.184 172.24.100.173 is_chassis_resident("cr-lrp-2a780f07-be18-45c5-a939-813d52fe55a6")', 'fa:16:3e:57:90:35 172.24.100.209 is_chassis_resident("cr-lrp-2a780f07-be
18-45c5-a939-813d52fe55a6")', 'fa:16:3e:7a:1d:90 172.24.100.102 is_chassis_resident("750a5f7c-9225-4984-b0ab-5ddbd1b71782")', 'fa:16:3e:9f:4b:bb 172.24.100.211 is_chassis_resident("4ec7cc4e-1cc3-4bf3-88a6-6664f42ffebd")', 'fa:16:3e:ad:1a:
2c 172.24.100.155 is_chassis_resident("49c7f905-d631-4b1e-88c2-edcd1a6aa60c")', 'fa:16:3e:ad:dd:ce 172.24.100.160 is_chassis_resident("937b3e8b-683c-467e-950b-a955d3c69043")', 'fa:16:3e:b7:a6:db 172.24.100.148 is_chassis_resident("2566c45
4-6e5b-443e-b0ac-7f7d4338a9a4")', 'fa:16:3e:bf:ec:5e 172.24.100.70 is_chassis_resident("f2ddd0d6-ecfc-43b1-8e62-082e7ecac695")', 'fa:16:3e:c7:de:38 172.24.100.85 is_chassis_resident("f09d3bef-540a-4ee4-8c7d-73ba5987e89d")', 'fa:16:3e:d4:7
9:55 172.24.100.35 is_chassis_resident("3c3a9516-7ffa-4def-9ca6-e1900e31ee81")', 'fa:16:3e:e5:95:81 172.24.100.241 is_chassis_resident("6f674c65-0dff-4521-bd0c-42c4d6b0037b")', 'fa:16:3e:f1:31:95 172.24.100.212 is_chassis_resident("2b9a67
8f-0c63-426e-9a8a-d19231ab2293")'], virtual_parent=[], up=[False], options={'peer': 'lrp-2a780f07-be18-45c5-a939-813d52fe55a6'}, parent_port=[], requested_additional_chassis=[], ha_chassis_group=[], external_ids={'neutron:cidrs': '172.24.
100.209/24 2001:db8::f816:3eff:fe57:9035/64', 'neutron:device_id': '18858d81-d07a-4115-b743-199b6b09198d', 'neutron:device_owner': 'network:router_gateway', 'neutron:network_name': 'neutron-b5b268c3-986b-4c80-999b-b0b6d68c654f', 'neutron:
port_capabilities': '', 'neutron:port_name': '', 'neutron:project_id': '', 'neutron:revision_number': '20', 'neutron:security_group_ids': '', 'neutron:subnet_pool_addr_scope4': '', 'neutron:subnet_pool_addr_scope6': '', 'neutron:vnic_type
': 'normal'}, additional_chassis=[], tag=[], additional_encap=[], encap=[], mirror_rules=[], datapath=2f8ccd39-7279-442a-9189-f8268c4d3e9f, chassis=[], tunnel_key=4, gateway_chassis=[], requested_chassis=[], logical_port=2a780f07-be18-45c
5-a939-813d52fe55a6) old=Port_Binding(nat_addresses=['fa:16:3e:0b:a4:8e 172.24.100.75 is_chassis_resident("5d4a9317-0e15-4993-b1d0-0cb854112390")', 'fa:16:3e:3e:33:22 172.24.100.56 is_chassis_resident("4773e234-993c-46f8-a5f7-2ea4ced5b4f8
")', 'fa:16:3e:57:90:35 172.24.100.171 172.24.100.10 172.24.100.184 172.24.100.173 is_chassis_resident("cr-lrp-2a780f07-be18-45c5-a939-813d52fe55a6")', 'fa:16:3e:57:90:35 172.24.100.209 is_chassis_resident("cr-lrp-2a780f07-be18-45c5-a939-
813d52fe55a6")', 'fa:16:3e:7a:1d:90 172.24.100.102 is_chassis_resident("750a5f7c-9225-4984-b0ab-5ddbd1b71782")', 'fa:16:3e:ad:1a:2c 172.24.100.155 is_chassis_resident("49c7f905-d631-4b1e-88c2-edcd1a6aa60c")', 'fa:16:3e:ad:dd:ce 172.24.100
.160 is_chassis_resident("937b3e8b-683c-467e-950b-a955d3c69043")', 'fa:16:3e:b7:a6:db 172.24.100.148 is_chassis_resident("2566c454-6e5b-443e-b0ac-7f7d4338a9a4")', 'fa:16:3e:bf:ec:5e 172.24.100.70 is_chassis_resident("f2ddd0d6-ecfc-43b1-8$
62-082e7ecac695")', 'fa:16:3e:c7:de:38 172.24.100.85 is_chassis_resident("f09d3bef-540a-4ee4-8c7d-73ba5987e89d")', 'fa:16:3e:d4:79:55 172.24.100.35 is_chassis_resident("3c3a9516-7ffa-4def-9ca6-e1900e31ee81")', 'fa:16:3e:e5:95:81 172.24.1$
0.241 is_chassis_resident("6f674c65-0dff-4521-bd0c-42c4d6b0037b")', 'fa:16:3e:f1:31:95 172.24.100.212 is_chassis_resident("2b9a678f-0c63-426e-9a8a-d19231ab2293")']) matches /usr/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/event.$
y:43                                                                                                                                                                                                                                         
2023-04-28T10:45:50.617666681+00:00 stdout F 2023-04-28 10:45:50.617 3139 DEBUG oslo_concurrency.lockutils [-] Lock "bgp" acquired by "ovn_bgp_agent.drivers.openstack.ovn_bgp_driver.OVNBGPDriver.expose_ip" :: waited 0.000s inner /usr/lib$
python3.9/site-packages/oslo_concurrency/lockutils.py:355                                                                                                                                                                                    
2023-04-28T10:45:50.621297855+00:00 stdout F 2023-04-28 10:45:50.621 3139 DEBUG ovn_bgp_agent.drivers.openstack.ovn_bgp_driver [-] Adding BGP route for FIP with ip ['172.24.100.211'] _expose_ip /usr/lib/python3.9/site-packages/ovn_bgp_ag$
nt/drivers/openstack/ovn_bgp_driver.py:587                                                                                                                                                                                                   


2) Remove a FIP from a VM:
$ openstack server remove floating ip kyrhrvei 172.24.100.211
As soon as the FIP is removed, its route is withdrawn.


The ovn-bgp-agent logs show that as soon as the FIPSetEvent is detected, the FIP is exposed:
2023-04-28T10:45:07.877619466+00:00 stdout F 2023-04-28 10:45:07.877 3139 DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched UPDATE: FIPUnsetEvent(events=('update',), table='Port_Binding', conditions=None, old_conditions=None) to row=Port_B
inding(mac=['router'], port_security=[], type=patch, nat_addresses=['fa:16:3e:0b:a4:8e 172.24.100.75 is_chassis_resident("5d4a9317-0e15-4993-b1d0-0cb854112390")', 'fa:16:3e:3e:33:22 172.24.100.56 is_chassis_resident("4773e234-993c-46f8-a5
f7-2ea4ced5b4f8")', 'fa:16:3e:57:90:35 172.24.100.171 172.24.100.10 172.24.100.184 172.24.100.173 is_chassis_resident("cr-lrp-2a780f07-be18-45c5-a939-813d52fe55a6")', 'fa:16:3e:57:90:35 172.24.100.209 is_chassis_resident("cr-lrp-2a780f07-
be18-45c5-a939-813d52fe55a6")', 'fa:16:3e:7a:1d:90 172.24.100.102 is_chassis_resident("750a5f7c-9225-4984-b0ab-5ddbd1b71782")', 'fa:16:3e:ad:1a:2c 172.24.100.155 is_chassis_resident("49c7f905-d631-4b1e-88c2-edcd1a6aa60c")', 'fa:16:3e:ad:d
d:ce 172.24.100.160 is_chassis_resident("937b3e8b-683c-467e-950b-a955d3c69043")', 'fa:16:3e:b7:a6:db 172.24.100.148 is_chassis_resident("2566c454-6e5b-443e-b0ac-7f7d4338a9a4")', 'fa:16:3e:bf:ec:5e 172.24.100.70 is_chassis_resident("f2ddd0
d6-ecfc-43b1-8e62-082e7ecac695")', 'fa:16:3e:c7:de:38 172.24.100.85 is_chassis_resident("f09d3bef-540a-4ee4-8c7d-73ba5987e89d")', 'fa:16:3e:d4:79:55 172.24.100.35 is_chassis_resident("3c3a9516-7ffa-4def-9ca6-e1900e31ee81")', 'fa:16:3e:e5:
95:81 172.24.100.241 is_chassis_resident("6f674c65-0dff-4521-bd0c-42c4d6b0037b")', 'fa:16:3e:f1:31:95 172.24.100.212 is_chassis_resident("2b9a678f-0c63-426e-9a8a-d19231ab2293")'], virtual_parent=[], up=[False], options={'peer': 'lrp-2a780
f07-be18-45c5-a939-813d52fe55a6'}, parent_port=[], requested_additional_chassis=[], ha_chassis_group=[], external_ids={'neutron:cidrs': '172.24.100.209/24 2001:db8::f816:3eff:fe57:9035/64', 'neutron:device_id': '18858d81-d07a-4115-b743-19
9b6b09198d', 'neutron:device_owner': 'network:router_gateway', 'neutron:network_name': 'neutron-b5b268c3-986b-4c80-999b-b0b6d68c654f', 'neutron:port_capabilities': '', 'neutron:port_name': '', 'neutron:project_id': '', 'neutron:revision_n
umber': '20', 'neutron:security_group_ids': '', 'neutron:subnet_pool_addr_scope4': '', 'neutron:subnet_pool_addr_scope6': '', 'neutron:vnic_type': 'normal'}, additional_chassis=[], tag=[], additional_encap=[], encap=[], mirror_rules=[], d
atapath=2f8ccd39-7279-442a-9189-f8268c4d3e9f, chassis=[], tunnel_key=4, gateway_chassis=[], requested_chassis=[], logical_port=2a780f07-be18-45c5-a939-813d52fe55a6) old=Port_Binding(nat_addresses=['fa:16:3e:0b:a4:8e 172.24.100.75 is_chass
is_resident("5d4a9317-0e15-4993-b1d0-0cb854112390")', 'fa:16:3e:3e:33:22 172.24.100.56 is_chassis_resident("4773e234-993c-46f8-a5f7-2ea4ced5b4f8")', 'fa:16:3e:57:90:35 172.24.100.171 172.24.100.10 172.24.100.184 172.24.100.173 is_chassis_
resident("cr-lrp-2a780f07-be18-45c5-a939-813d52fe55a6")', 'fa:16:3e:57:90:35 172.24.100.209 is_chassis_resident("cr-lrp-2a780f07-be18-45c5-a939-813d52fe55a6")', 'fa:16:3e:7a:1d:90 172.24.100.102 is_chassis_resident("750a5f7c-9225-4984-b0a
b-5ddbd1b71782")', 'fa:16:3e:9f:4b:bb 172.24.100.211 is_chassis_resident("4ec7cc4e-1cc3-4bf3-88a6-6664f42ffebd")', 'fa:16:3e:ad:1a:2c 172.24.100.155 is_chassis_resident("49c7f905-d631-4b1e-88c2-edcd1a6aa60c")', 'fa:16:3e:ad:dd:ce 172.24.1
00.160 is_chassis_resident("937b3e8b-683c-467e-950b-a955d3c69043")', 'fa:16:3e:b7:a6:db 172.24.100.148 is_chassis_resident("2566c454-6e5b-443e-b0ac-7f7d4338a9a4")', 'fa:16:3e:bf:ec:5e 172.24.100.70 is_chassis_resident("f2ddd0d6-ecfc-43b1-
8e62-082e7ecac695")', 'fa:16:3e:c7:de:38 172.24.100.85 is_chassis_resident("f09d3bef-540a-4ee4-8c7d-73ba5987e89d")', 'fa:16:3e:d4:79:55 172.24.100.35 is_chassis_resident("3c3a9516-7ffa-4def-9ca6-e1900e31ee81")', 'fa:16:3e:e5:95:81 172.24.
100.241 is_chassis_resident("6f674c65-0dff-4521-bd0c-42c4d6b0037b")', 'fa:16:3e:f1:31:95 172.24.100.212 is_chassis_resident("2b9a678f-0c63-426e-9a8a-d19231ab2293")']) matches /usr/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/event
.py:43
2023-04-28T10:45:07.879591679+00:00 stdout F 2023-04-28 10:45:07.879 3139 DEBUG oslo_concurrency.lockutils [-] Lock "bgp" acquired by "ovn_bgp_agent.drivers.openstack.ovn_bgp_driver.OVNBGPDriver.withdraw_ip" :: waited 0.000s inner /usr/li
b/python3.9/site-packages/oslo_concurrency/lockutils.py:355
2023-04-28T10:45:07.881203016+00:00 stdout F 2023-04-28 10:45:07.881 3139 DEBUG ovn_bgp_agent.drivers.openstack.ovn_bgp_driver [-] Deleting BGP route for FIP with ip ['172.24.100.211'] withdraw_ip /usr/lib/python3.9/site-packages/ovn_bgp_
agent/drivers/openstack/ovn_bgp_driver.py:729

Comment 16 errata-xmlrpc 2023-08-16 01:14:48 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Release of components for Red Hat OpenStack Platform 17.1 (Wallaby)), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2023:4577


Note You need to log in before you can comment on or make changes to this bug.