Bug 1296192 - Reduce Memory usage of MiqScheduleWorker or raise memory threshold
Reduce Memory usage of MiqScheduleWorker or raise memory threshold
Status: CLOSED ERRATA
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Performance (Show other bugs)
5.5.0
Unspecified Unspecified
high Severity high
: GA
: 5.6.0
Assigned To: dmetzger
Dave Johnson
:
: 1314113 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2016-01-06 09:51 EST by Alex Krzos
Modified: 2017-12-05 10:16 EST (History)
5 users (show)

See Also:
Fixed In Version: 5.6.0.0
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-06-29 11:25:36 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
5.5.2.0 MiqScheduleWorker Memory Usage (99.39 KB, image/png)
2016-01-06 09:51 EST, Alex Krzos
no flags Details
5.5.0.13-2 MiqScheduleWorker Memory Usage (61.46 KB, image/png)
2016-01-06 09:52 EST, Alex Krzos
no flags Details
5.4.4.2 MiqScheduleWorker Memory Usage (64.37 KB, image/png)
2016-01-06 09:52 EST, Alex Krzos
no flags Details
evm.log output from pid 8220 (MiqScheduleWorker) (75.74 KB, text/plain)
2016-01-06 10:14 EST, Alex Krzos
no flags Details

  None (edit)
Description Alex Krzos 2016-01-06 09:51:06 EST
Created attachment 1112200 [details]
5.5.2.0 MiqScheduleWorker Memory Usage

Description of problem:
5.5.2.0 experiences recycling (exiting and restarting) the MiqScheduleWorker in many of the memory baseline tests.  The threshold is set to 250.megabytes in the advanced configuration.  5.5.0.13-2 and 5.4.4.2 also has the MiqScheduleWorker memory threshold set to 250.megabytes however it does not recycle in any of the same scenarios. 

This has been found in two memory baselining scenarios: Capacity and Utilization and Provisioning.

Reviewing historical for 5.5.0.13-2 and 5.4.4.2, the MiqScheduleWorker does not recycle in any of the same scenarios suggesting the additional memory growth was recently introduced.  The memory usage does increase between 5.4.4.2 to 5.5.0.13-2 as expected (Move to Ruby 2.2) however not enough to come close to the memory threshold.

Version-Release number of selected component (if applicable):
5.5.2.0

How reproducible:
~ >50% of the time (7 out of 13 tests showed this failure)

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:


Attaching Memory graphs of the MiqScheduleWorker during Capacity and Utilization memory baselining showing the following versions in the same scenario:
5.4.4.2
5.5.0.13-2
5.5.2.0
Comment 2 Alex Krzos 2016-01-06 09:52 EST
Created attachment 1112201 [details]
5.5.0.13-2 MiqScheduleWorker Memory Usage
Comment 3 Alex Krzos 2016-01-06 09:52 EST
Created attachment 1112202 [details]
5.4.4.2 MiqScheduleWorker Memory Usage
Comment 4 Alex Krzos 2016-01-06 10:14 EST
Created attachment 1112207 [details]
evm.log output from pid 8220 (MiqScheduleWorker)

Reviewing the CSV output of memory usage, the growth spurt in (rss, pss, uss) memory right before the worker exits starts between 16:36:57 and 16:37:07:


TimeStamp	RSS
2016-01-05 16:36:37.221739	208.42578125
2016-01-05 16:36:47.231512	208.4921875
2016-01-05 16:36:57.241401	208.5
2016-01-05 16:37:07.251246	219.359375
2016-01-05 16:37:17.261163	230.15234375
2016-01-05 16:37:27.266662	238.265625
2016-01-05 16:37:37.277017	245.796875
2016-01-05 16:37:47.286422	252.53125
2016-01-05 16:37:57.293079	252.53125
Comment 7 Nick Carboni 2016-03-15 16:36:21 EDT
*** Bug 1314113 has been marked as a duplicate of this bug. ***
Comment 8 Alex Krzos 2016-03-29 13:49:38 EDT
5.5.3.2 has a limit of 300MiB for the MiqScheduleWorker and I have not seen this issue occur again on this build.
Comment 10 errata-xmlrpc 2016-06-29 11:25:36 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1348

Note You need to log in before you can comment on or make changes to this bug.