Bug 1676975 - Open files counts for octavia-api grow dramatically under normal operation
Summary: Open files counts for octavia-api grow dramatically under normal operation
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-octavia
Version: 13.0 (Queens)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: z7
: 13.0 (Queens)
Assignee: Carlos Goncalves
QA Contact: Bruna Bonguardo
URL:
Whiteboard:
Depends On: 1676971
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-02-13 17:32 UTC by Carlos Goncalves
Modified: 2019-09-10 14:08 UTC (History)
6 users (show)

Fixed In Version: openstack-octavia-2.1.0-1.el7ost
Doc Type: Bug Fix
Doc Text:
This update fixes a system load and resource utilization issue relating to the octavia API process.
Clone Of: 1676971
Environment:
Last Closed: 2019-07-10 13:01:59 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 636428 0 None None None 2019-02-13 17:33:22 UTC
OpenStack gerrit 650304 0 None None None 2019-05-13 14:08:02 UTC
Red Hat Product Errata RHBA-2019:1744 0 None None None 2019-07-10 13:02:16 UTC
Storyboard 2004993 0 None None None 2019-02-13 17:33:22 UTC

Description Carlos Goncalves 2019-02-13 17:32:01 UTC
+++ This bug was initially created as a clone of Bug #1676971 +++

It was discovered that the octavia-api process, that uses the oslo.messaging package, is orphaning amqp connections with no effort to gc old connections. This leads to a large number of open files accumulating rapidly on the host filesystem and an overall negative impact on system load and resource utilization.

Original report: https://storyboard.openstack.org/#!/story/2004993

How reproducible:
100%

Steps to Reproduce:
1. trigger dozens of all sorts of CRUD operations
2. monitor CPU and memory usage

Actual results:
CPU and virtual memory usage of octavia-api container grow with the number of LB control-plane operations.

Expected results:
CPU and memory usage should remain mostly flat with time and operations.

Additional info:
This issue was also observed on Rally runs.

Comment 11 errata-xmlrpc 2019-07-10 13:01:59 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:1744


Note You need to log in before you can comment on or make changes to this bug.