Bug 1397457

Summary: When using cgroups to prevent Memory leakage - Satellite server works slowly
Product: Red Hat Satellite Reporter: Sasha Segal <ssegal>
Component: PulpAssignee: satellite6-bugs <satellite6-bugs>
Status: CLOSED WONTFIX QA Contact: Katello QA List <katello-qa-list>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 6.2.0CC: abraverm, bbuckingham, bkearney, mhulan, ssegal, ttereshc
Target Milestone: UnspecifiedKeywords: Triaged
Target Release: Unused   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-01-18 13:22:51 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
Forebug debug log none

Description Sasha Segal 2016-11-22 15:08:23 UTC 
Description of problem:

We had issues when Satellite server got in swap. We were advised to implement cgroups to limit MongoDB memory usage (will grow forever because depends on host count). 2GB cgroups limit on MongoDB caused this to crash. 4GB is enough and Satellite does not get into swap anymore BUT
This works very slow. 

Version-Release number of selected component (if applicable):
6.2.0

How reproducible:

Install all-in-one Satellite.

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:
Satellite works fast. MongoDB not crashed. No Memory leakage.

Additional info:
OS version is RHEL 6.8
Comment 1 Brad Buckingham 2016-11-22 18:27:55 UTC 
Hi Sasha,

Can you attach a foreman-debug of the server where you are observing the memory growth?

Is it only mongodb memory that is growing?
Comment 2 Sasha Segal 2016-11-23 07:43:33 UTC 
Created attachment 1223048 [details]
Forebug debug log
Comment 3 Sasha Segal 2016-11-23 07:45:44 UTC 
Uploaded debug log. Looks like without cgroups only MongoDB is growing memory. 2GB cgroups limit on MongoDB makes it crash after some time.
4GB cgroups limit does not crash MongoDB but server again get in swap of ~1GB and works slowly.
Comment 4 Marek Hulan 2017-11-15 15:17:52 UTC 
Moving to Pulp component, Tanya, do you happen to know whether this is expected or any other workaround for MongoDB?
Comment 5 Tanya Tereshchenko 2017-11-16 20:38:22 UTC 
My understanding is probably 4GB is not enough for MongoDB for this setup/usage.
MongoDB is known to be greedy in terms of memory usage but at the same time it releases it quickly when needed. So if it's MongoDB which goes into swap then I guess it requires more than 4GB as a minimum.

There're some docs how to measure what's the minimum for a particular case, maybe worth trying:
https://docs.mongodb.com/manual/faq/diagnostics/#how-do-i-calculate-how-much-ram-i-need-for-my-application
Comment 6 Sasha Segal 2017-12-12 08:05:57 UTC 
Suggest close the issue, may be some online documentation related to this would be enough.
Comment 7 Bryan Kearney 2018-01-18 13:22:51 UTC 
Closing per comment 6