Bug 1479264

Summary:	Ever-increasing memory consumption causes the Controller to crash
Product:	Red Hat OpenStack	Reporter:	Stephen Kitt <skitt>
Component:	opendaylight	Assignee:	Stephen Kitt <skitt>
Status:	CLOSED ERRATA	QA Contact:	Tomas Jamrisko <tjamrisk>
Severity:	urgent	Docs Contact:
Priority:	urgent
Version:	12.0 (Pike)	CC:	itbrown, knylande, lpeer, mkolesni, nyechiel, skitt, tjamrisk, tvignaud
Target Milestone:	beta	Keywords:	Triaged
Target Release:	13.0 (Queens)
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:	opendaylight-8.0.0-1.el7ost	Doc Type:	No Doc Update
Doc Text:	undefined	Story Points:	---
Clone Of:		Environment:	N/A
Last Closed:	2018-06-27 13:33:53 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1451401

Description Stephen Kitt 2017-08-08 08:36:13 UTC

On a single-node Carbon SR1 setup, we got an OOM over the weekend. Looking at the heap dump shows 1.1GB occupied in the frontend history's closed transactions map, with 33M entries. Analysing the dump further reveals that there's a bug in the controller's transaction handling, which results in transactions being held in memory forever — which guarantees an OOM eventually, regardless of the JVM settings.

The heap dump is on https://www.sk2.org/java_pid2098.hprof.xz (269MB).

Comment 2 Stephen Kitt 2017-09-20 13:28:24 UTC

This has been fixed in Carbon upstream; it will be part of SR2.

Comment 14 errata-xmlrpc 2018-06-27 13:33:53 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:2086