Bug 1321726

Summary: Amazon realtime collection returns tens of thousands of rows vs 180 max rows for VMware causing appliance to fill swap
Product: Red Hat CloudForms Management Engine Reporter: Thomas Hennessy <thenness>
Component: C&U Capacity and UtilizationAssignee: Greg Blomquist <gblomqui>
Status: CLOSED NOTABUG QA Contact: Nandini Chandra <nachandr>
Severity: high Docs Contact:
Priority: high    
Version: 5.5.0CC: carnott, cpelland, gblomqui, jdeubel, jhardy, jocarter, kbrock, mfeifer, nachandr, obarenbo, thenness
Target Milestone: GA   
Target Release: 5.7.0   
Hardware: x86_64   
OS: Linux   
Whiteboard: c&u:ec2:perf
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-08-05 16:06:59 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
log lines for Amazon realtime collections providing row counts
none
one page pdf charting real and swap memory being filled by Amazon C&U collections none

Description Thomas Hennessy 2016-03-29 03:29:13 UTC
Created attachment 1141083 [details]
log lines for Amazon realtime collections providing row counts

Description of problem: Amazon real time collected metrics are > two orders of magnitued more than VMware resulting in all of real memory and all of swap to be fully consumed resulting in appliances effectively down


Version-Release number of selected component (if applicable):5.5.2.4


How reproducible: In customer environment, simply enabling C&U Collections for Amazon VMs is sufficient to cause the problem. I have no idea how to create the issue otherwise.


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 2 Thomas Hennessy 2016-03-29 03:37:30 UTC
Created attachment 1141085 [details]
one page pdf charting real and swap memory being filled by Amazon C&U collections

Comment 4 Thomas Hennessy 2016-03-30 20:11:01 UTC
the top_output.log.gz file is also provided in the same location.  this provides the top view of the appliance and all processes every 60 seconds so that one can see how the memory usage changes with time.

Comment 9 Nandini Chandra 2016-04-12 04:57:54 UTC
I don't have any appliance with 2 weeks worth of data, but I'd like to add that I  haven't had issues running gap collection over few weeks on vsphere in 5.5.