Bug 1869047 - [OSP 13][ML2/OVS] L3 HA router failure and stability issues with large floating IP count
Summary: [OSP 13][ML2/OVS] L3 HA router failure and stability issues with large floati...
Keywords:
Status: CLOSED DUPLICATE of bug 1869355
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-neutron
Version: 13.0 (Queens)
Hardware: x86_64
OS: Linux
high
high
Target Milestone: ---
: ---
Assignee: Slawek Kaplonski
QA Contact: Eran Kuris
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-08-15 21:30 UTC by Matt Flusche
Modified: 2023-12-15 18:52 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-09-21 19:55:47 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker OSP-21601 0 None None None 2023-01-20 10:10:01 UTC

Description Matt Flusche 2020-08-15 21:30:12 UTC 
Description of problem:

This specific environment has a single L3 router with about 200 FIPs.  The keepalived processes associated with the router consumes a large amount of CPU.  At times the keepalived instances will transition from active to standby and back every minute or so across the L3 agents.  There are about 350 other HA routers.  This router will also cause overall system stability on the controller nodes (high load average and pacemaker services failures).  Disabling this specific router will resolve the apparent issues.

Version-Release number of selected component (if applicable):
OSP 13

$ grep neutron installed-rpms 
openstack-neutron-12.1.1-6.el7ost.noarch                    Sun Apr 12 08:25:13 2020
openstack-neutron-common-12.1.1-6.el7ost.noarch             Sun Apr 12 08:21:38 2020
openstack-neutron-l2gw-agent-12.0.2-0.20190420004620.270972f.el7ost.noarch Sun Apr 12 08:26:48 2020
openstack-neutron-lbaas-12.0.1-0.20190803015156.b86fcef.el7ost.noarch Sun Apr 12 08:26:08 2020
openstack-neutron-lbaas-ui-4.0.1-0.20190723082436.ccf8621.el7ost.noarch Sun Apr 12 08:26:07 2020
openstack-neutron-linuxbridge-12.1.1-6.el7ost.noarch        Sun Apr 12 08:26:48 2020
openstack-neutron-metering-agent-12.1.1-6.el7ost.noarch     Sun Apr 12 08:26:48 2020
openstack-neutron-ml2-12.1.1-6.el7ost.noarch                Sun Apr 12 08:21:39 2020
openstack-neutron-openvswitch-12.1.1-6.el7ost.noarch        Sun Apr 12 08:26:46 2020
openstack-neutron-sriov-nic-agent-12.1.1-6.el7ost.noarch    Sun Apr 12 08:26:49 2020
puppet-neutron-12.4.1-10.el7ost.noarch                      Sun Apr 12 08:25:11 2020
python2-neutronclient-6.7.0-1.el7ost.noarch                 Thu Apr 18 10:02:11 2019
python2-neutron-lib-1.13.0-2.el7ost.noarch                  Sun Apr 12 08:21:24 2020
python-neutron-12.1.1-6.el7ost.noarch                       Sun Apr 12 08:21:29 2020
python-neutron-lbaas-12.0.1-0.20190803015156.b86fcef.el7ost.noarch Sun Apr 12 08:21:29 2020


How reproducible:
Unknown, This specific environment


Additional info:
I'll provide additional details in private comments
Comment 17 Slawek Kaplonski 2020-09-21 19:55:47 UTC 

*** This bug has been marked as a duplicate of bug 1869355 ***
Note You need to log in before you can comment on or make changes to this bug.