Bug 1676975

Summary:	Open files counts for octavia-api grow dramatically under normal operation
Product:	Red Hat OpenStack	Reporter:	Carlos Goncalves <cgoncalves>
Component:	openstack-octavia	Assignee:	Carlos Goncalves <cgoncalves>
Status:	CLOSED ERRATA	QA Contact:	Bruna Bonguardo <bbonguar>
Severity:	high	Docs Contact:
Priority:	high
Version:	13.0 (Queens)	CC:	astafeye, ihrachys, lpeer, majopela, mgeary, slinaber
Target Milestone:	z7	Keywords:	Triaged, ZStream
Target Release:	13.0 (Queens)
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	openstack-octavia-2.1.0-1.el7ost	Doc Type:	Bug Fix
Doc Text:	This update fixes a system load and resource utilization issue relating to the octavia API process.	Story Points:	---
Clone Of:	1676971	Environment:
Last Closed:	2019-07-10 13:01:59 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1676971
Bug Blocks:

Description Carlos Goncalves 2019-02-13 17:32:01 UTC

+++ This bug was initially created as a clone of Bug #1676971 +++

It was discovered that the octavia-api process, that uses the oslo.messaging package, is orphaning amqp connections with no effort to gc old connections. This leads to a large number of open files accumulating rapidly on the host filesystem and an overall negative impact on system load and resource utilization.

Original report: https://storyboard.openstack.org/#!/story/2004993

How reproducible:
100%

Steps to Reproduce:
1. trigger dozens of all sorts of CRUD operations
2. monitor CPU and memory usage

Actual results:
CPU and virtual memory usage of octavia-api container grow with the number of LB control-plane operations.

Expected results:
CPU and memory usage should remain mostly flat with time and operations.

Additional info:
This issue was also observed on Rally runs.

Comment 11 errata-xmlrpc 2019-07-10 13:01:59 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:1744