Bug 508327 - malloc limits in /etc/security/limits.conf ignored by pam
malloc limits in /etc/security/limits.conf ignored by pam
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: pam (Show other bugs)
x86_64 Linux
low Severity medium
: rc
: ---
Assigned To: Tomas Mraz
Depends On:
  Show dependency treegraph
Reported: 2009-06-26 11:50 EDT by Benjamin Avdicevic
Modified: 2009-06-26 15:26 EDT (History)
0 users

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2009-06-26 14:56:19 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Benjamin Avdicevic 2009-06-26 11:50:46 EDT
Description of problem:
Setting user limits in /etc/security/limits.con are ignored by kernel

Version-Release number of selected component (if applicable):
[ben@benvmpcm12 ~]$ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 5.3 (Tikanga)

[ben@benvmpcm12 ~]$ cat /proc/version
Linux version 2.6.18-128.el5 (mockbuild@hs20-bc1-7.build.redhat.com) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Wed Dec 17 11:41:38 EST 2008
How reproducible:
Easy to reproduce

Steps to Reproduce:
1.Set 'memlock' and/or 'rss' limits in limits.conf for user <user>
2.ssh to that host as <user>
3.run an application to exceede memory limit set in step 1.
Actual results:
Application runs

Expected results:
Application should fail.

Additional info:
Please see this thread which points to kernel bug:


This is simple C program I used to test this behavior:

    /* Allocate space for an array with ten elements of type int. */
    char *ptr;
    ptr =  malloc(500000000);
    if (ptr == NULL) {
    /* Memory could not be allocated, the program should handle the error here as appropriate. */
    printf("Memory could not be allocated\n");
    } else {
        /* Allocation succeeded.  Do something.  */
        printf("Memory allocation successful\n");
        free(ptr);  /* We are done with the int objects, and free the associated pointer. */
        ptr = NULL; /* The pointer must not be used again, unless re-assigned to using malloc again. */
Comment 1 Tomas Mraz 2009-06-26 14:56:19 EDT
This is known and now even commented about in the limits.conf(5) manual page.
Comment 2 Benjamin Avdicevic 2009-06-26 15:05:45 EDT

Thanks for your follow up. I am running RHEL5.3, and I cannot find comments you're referring to in man page for limits.conf(5).
Comment 3 Tomas Mraz 2009-06-26 15:26:27 EDT
rpm -q pam?

search for rss in limits.conf

I see the comment there.

Note You need to log in before you can comment on or make changes to this bug.