Bug 1479264 – Ever-increasing memory consumption causes the Controller to crash

Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.

Bug 1479264 - Ever-increasing memory consumption causes the Controller to crash

Summary:	Ever-increasing memory consumption causes the Controller to crash

Status:	CLOSED ERRATA

Aliases:	None

Product:	Red Hat OpenStack
Classification:	Red Hat
Component:	opendaylight (Show other bugs)
Sub Component:
Version:	12.0 (Pike)
Hardware:	All Linux

Priority	urgent Severity urgent
Target Milestone:	beta
Target Release:	13.0 (Queens)
Assigned To:	Stephen Kitt
QA Contact:	Tomas Jamrisko
Docs Contact:

URL:
Whiteboard:
Keywords:	Triaged

Depends On:
Blocks:	1451401
	Show dependency tree / graph

Reported:	2017-08-08 04:36 EDT by Stephen Kitt
Modified:	2018-10-18 03:20 EDT (History)
CC List:	8 users (show)

See Also:
Fixed In Version:	opendaylight-8.0.0-1.el7ost
Doc Type:	No Doc Update
Doc Text:	undefined
Story Points:	---
Clone Of:
Environment:	N/A
Last Closed:	2018-06-27 09:33:53 EDT
Type:	Bug
Regression:	---
Mount Type:	---
Documentation:	---
CRM:
Verified Versions:
Category:	---
oVirt Team:	---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---

Attachments	(Terms of Use)
Add an attachment (proposed patch, testcase, etc.)

External Trackers
Tracker	ID	Priority	Status	Summary	Last Updated
OpenDaylight Bug	8941	None	None	None	2017-08-08 04:36 EDT
OpenDaylight gerrit	61284	None	None	None	2017-08-08 07:51 EDT
Red Hat Product Errata	RHEA-2018:2086	None	None	None	2018-06-27 09:34 EDT

Groups: None (edit)

Description Stephen Kitt 2017-08-08 04:36:13 EDT

On a single-node Carbon SR1 setup, we got an OOM over the weekend. Looking at the heap dump shows 1.1GB occupied in the frontend history's closed transactions map, with 33M entries. Analysing the dump further reveals that there's a bug in the controller's transaction handling, which results in transactions being held in memory forever — which guarantees an OOM eventually, regardless of the JVM settings.

The heap dump is on https://www.sk2.org/java_pid2098.hprof.xz (269MB).

Comment 2 Stephen Kitt 2017-09-20 09:28:24 EDT

This has been fixed in Carbon upstream; it will be part of SR2.

Comment 14 errata-xmlrpc 2018-06-27 09:33:53 EDT

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:2086

Note You need to log in before you can comment on or make changes to this bug.