Bug 459628

+++ This bug was initially created as a clone of Bug #459624 +++

Escalated to Bugzilla from IssueTracker

--- Additional comment from tao on 2008-08-20 13:59:05 EDT ---

Description of problem:
On some machines in RHEL 5.1 and 5.2 we encounter the same problem, the max memory size field is bypassed by a shell script. 
How reproducible:
I attached the script foo.ksh which we use to simulate the problem.
I attached the file /etc/security/limits.conf and the sosreport
When we run foo.ksh we acheave and bypass the limit. In our case it is a problem because we arrive to use  oom killer. 
Steps to Reproduce:
setup the file limits.conf 
run  /tmp/foo.ksh >/dev/null
ps aux to check that we have more than 60M for that process.
Thanks for your help .
Christophe. 

Actual results:

Expected results:

Additional info:
This event sent from IssueTracker by jpayne  [Support Engineering Group]
 issue 181993

--- Additional comment from tao on 2008-08-20 13:59:06 EDT ---

File uploaded: foo.ksh
This event sent from IssueTracker by jpayne  [Support Engineering Group]
 issue 181993
it_file 134626

--- Additional comment from tao on 2008-08-20 13:59:07 EDT ---

File uploaded: limits.conf

This event sent from IssueTracker by jpayne  [Support Engineering Group]
 issue 181993
it_file 134628

--- Additional comment from tao on 2008-08-20 13:59:08 EDT ---

So..in pam, this is how the limit are initialised after pl has been
populated acording to the limits.conf value.

static int init_limits(struct pam_limit_s *pl)
{
    int i;
    int retval = PAM_SUCCESS;

    D(("called."));

    for(i = 0; i < RLIM_NLIMITS; i++) {
        int r = getrlimit(i, &pl->limits[i].limit);
...
...
...

Now, man getrlimit states:

       RLIMIT_RSS
              Specifies the limit (in pages) of the process's resident
set (the number of virtual pages resident
              in  RAM).  This limit only has effect in Linux 2.4.x, x <
30, and there only affects calls to mad-
              vise() specifying MADV_WILLNEED.


It looks to me like this will never works on a RHEL5 as it uses a 2.6
kernel.






Internal Status set to 'Waiting on Support'

This event sent from IssueTracker by jpayne  [Support Engineering Group]
 issue 181993

--- Additional comment from tao on 2008-08-20 13:59:10 EDT ---

I have received confirmation from Rik van Riel that this is the default
behavior. RLIMIT_RSS is not supported in the upstream kernel, or Red Hat
2.6.x kernels.

If necessary, we can submit a request to have the man page for limits.conf
edited to correct this.

Thanks,

Justin

Internal Status set to 'Waiting on Support'

This event sent from IssueTracker by jpayne  [Support Engineering Group]
 issue 181993

--- Additional comment from tao on 2008-08-20 14:02:45 EDT ---

This bug is for the limits.conf man page. In the <item> section of the
page, rss is described:

          rss
             maximum resident set size (KB)

RLIMIT_RSS is not supported in 2.6 kernels. Request this to be removed
from RHEL5 and RHEL4 (will clone bug for RHEL4).

-Justin


Issue escalated to RHEL 5 Base OS by: jpayne.
Internal Status set to 'Waiting on Engineering'

This event sent from IssueTracker by jpayne 
 issue 181993