Created attachment 397246 [details]
Test program that mallocs a lot of memory
Description of problem:
Several times recently, I've hit bugs where an application enters an infinite loop allocating memory and brings the system to a halt. I wanted to mitigate the problem by setting an RLIMIT_DATA (ulimit -d) in my login scripts. But RLIMIT_DATA has no effect because malloc falls back to mmap when brk fails:
Supposedly the fallback is needed on systems with address space holes. If I'm right in thinking that Linux is not such a system, we could disable the fallback.
(Note that RLIMIT_AS does not solve my problem because OOo needs to map 2.5 GB, almost all of which is read-only, but if I set the limit that high, a process that allocated that much writable memory would bog down the system.)
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1. Download and compile the attached memory-hog.c .
2. (ulimit -S -d 500000 && ./memory-hog)
"Successfully malloc-ed 1 GB"
"malloc failed (i = SOMETHING), exiting"
malloc also uses mmap for large allocations, so they don't count toward RLIMIT_DATA. This has been noted:
I'm guessing this won't affect me because infinite loops tend to involve small allocations. But if it does, I can install an LD_PRELOAD library that calls mallopt(M_MMAP_MAX, 0) to disable the feature.
As I said, RLIMIT_AS does not solve my problem. Why is it not a bug that RLIMIT_DATA does not work?
Reopening because an adequate explanation of why this is not a bug was not given.
RLIMIT_DATA works as designed.
RLIMIT_DATA is useless because whenever glibc hits the limit with brk, it just calls mmap instead. I could understand saying that RLIMIT_DATA is broken and will be left so, but saying that it works is ludicrous.
See bug 582072.