Bug 1045540

Summary: Misconfigured resource_limits.conf memory_limit_in_bytes causes problems with /tmp
Product: OpenShift Container Platform Reporter: Bill DeCoste <wdecoste>
Component: ContainersAssignee: Brenton Leanhardt <bleanhar>
Status: CLOSED NOTABUG QA Contact: libra bugs <libra-bugs>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 2.0.0CC: libra-onpremise-devel, wdecoste
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-01-21 19:48:57 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Bill DeCoste 2013-12-20 16:51:41 UTC 
Description of problem: I misconfigured resource_limits.conf to set memory_limit_in_bytes to 10g (instead of the intended 1g) and the following issues arose:

1) httpd segfaults due to a failure of passenger to write to /tmp unless I had logged in as root
2) tc also failed during app creation failing to write to /tmp


Version-Release number of selected component (if applicable):


How reproducible:
100% on my instance


Steps to Reproduce:
1. configure memory_limit_in_bytes to be 10g
2. attempt to restart httpd
3.

Actual results:
BOOM


Expected results:
At the least oo-diagnostics should catch this


Additional info:
Comment 2 Bill DeCoste 2013-12-20 19:38:50 UTC 
Looks like this happens even with 1g gear memory size. Only time it works is with the default 512m
Comment 3 Bill DeCoste 2013-12-20 19:40:27 UTC 
Yes, OSE2.0.0
Comment 6 Bill DeCoste 2014-01-06 18:12:14 UTC 
I do have an environment where I can recreate this. When I switched resource_limits.conf back to the default settings it works.
Comment 8 Bill DeCoste 2014-01-06 18:23:11 UTC 
It's running on a VM on my laptop. It's a clean RHEL6.5 install with OSE2.0. Nothing out of the ordinary.
Comment 9 Brenton Leanhardt 2014-01-09 19:54:41 UTC 
Would you mind attaching your exact resource_limits.conf?  On a newly created RHEL 6.5 OSE 2.x host I had no problems setting:

memory_limit_in_bytes=10737418240 # 10g 
memory_memsw_limit_in_bytes=10842275840 # 10737418240 (10g) + 104857600 (100mb)

I was able to verify under /cgroup/memory/openshift/$uuid that the values were set for my gears.  I haven't yet had any problems with application creation or restarting httpd (still digging)

I'm wondering if you gears were perhaps actually using large amounts of memory and causing problems for the rest of the system.  What was the value for memory.max_usage_in_bytes for you gears?

On a side note, not being able to write to /tmp really seems like it would be caused by something else.
Comment 10 Bill DeCoste 2014-01-21 19:29:23 UTC 
I had not increased memory_memsw_limit_in_bytes when I increased
memory_limit_in_bytes. Never had to do this in OSE1.2. Works fine when I adjust
both.
Comment 11 Brenton Leanhardt 2014-01-21 19:48:57 UTC 
That settings was new to me too. :)