Bug 569714 - malloc: fallback to mmap when brk fails contravenes RLIMIT_DATA (ulimit -d)
Summary: malloc: fallback to mmap when brk fails contravenes RLIMIT_DATA (ulimit -d)
Alias: None
Product: Fedora
Classification: Fedora
Component: glibc
Version: 12
Hardware: All
OS: Linux
Target Milestone: ---
Assignee: Andreas Schwab
QA Contact: Fedora Extras Quality Assurance
Depends On:
TreeView+ depends on / blocked
Reported: 2010-03-02 06:53 UTC by Matt McCutchen
Modified: 2010-04-14 02:19 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Last Closed: 2010-03-05 10:38:20 UTC
Type: ---

Attachments (Terms of Use)
Test program that mallocs a lot of memory (399 bytes, text/x-csrc)
2010-03-02 06:53 UTC, Matt McCutchen
no flags Details

Description Matt McCutchen 2010-03-02 06:53:01 UTC
Created attachment 397246 [details]
Test program that mallocs a lot of memory

Description of problem:
Several times recently, I've hit bugs where an application enters an infinite loop allocating memory and brings the system to a halt.  I wanted to mitigate the problem by setting an RLIMIT_DATA (ulimit -d) in my login scripts.  But RLIMIT_DATA has no effect because malloc falls back to mmap when brk fails:


Supposedly the fallback is needed on systems with address space holes.  If I'm right in thinking that Linux is not such a system, we could disable the fallback.

(Note that RLIMIT_AS does not solve my problem because OOo needs to map 2.5 GB, almost all of which is read-only, but if I set the limit that high, a process that allocated that much writable memory would bog down the system.)

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. Download and compile the attached memory-hog.c .
2. (ulimit -S -d 500000 && ./memory-hog)
Actual results:
"Successfully malloc-ed 1 GB"

Expected results:
"malloc failed (i = SOMETHING), exiting"

Additional info:
malloc also uses mmap for large allocations, so they don't count toward RLIMIT_DATA.  This has been noted:


I'm guessing this won't affect me because infinite loops tend to involve small allocations.  But if it does, I can install an LD_PRELOAD library that calls mallopt(M_MMAP_MAX, 0) to disable the feature.

Comment 1 Andreas Schwab 2010-03-02 10:17:46 UTC

Comment 2 Matt McCutchen 2010-03-02 14:03:52 UTC
As I said, RLIMIT_AS does not solve my problem.  Why is it not a bug that RLIMIT_DATA does not work?

Comment 3 Matt McCutchen 2010-03-05 03:29:36 UTC
Reopening because an adequate explanation of why this is not a bug was not given.

Comment 4 Andreas Schwab 2010-03-05 09:02:37 UTC
RLIMIT_DATA works as designed.

Comment 5 Matt McCutchen 2010-03-05 09:25:50 UTC
RLIMIT_DATA is useless because whenever glibc hits the limit with brk, it just calls mmap instead.  I could understand saying that RLIMIT_DATA is broken and will be left so, but saying that it works is ludicrous.

Comment 6 Andreas Schwab 2010-03-05 10:38:20 UTC
See getrlimit(2).

Comment 7 Matt McCutchen 2010-04-14 02:19:00 UTC
See bug 582072.

Note You need to log in before you can comment on or make changes to this bug.