Bug 569714

Summary:

malloc: fallback to mmap when brk fails contravenes RLIMIT_DATA (ulimit -d)

Product:

[Fedora] Fedora

Reporter:

Matt McCutchen <matt>

Component:

glibc

Assignee:

Andreas Schwab <schwab>

Status:

CLOSED NOTABUG

QA Contact:

Fedora Extras Quality Assurance <extras-qa>

Severity:

medium

Docs Contact:

Priority:

low

Version:

CC:

jakub, schwab

Target Milestone:

---

Keywords:

Reopened

Target Release:

---

Hardware:

All

OS:

Linux

Whiteboard:

Fixed In Version:

Doc Type:

Bug Fix

Doc Text:

Story Points:

---

Clone Of:

Environment:

Last Closed:

2010-03-05 10:38:20 UTC

Type:

---

Regression:

---

Mount Type:

---

Documentation:

---

CRM:

Verified Versions:

Category:

---

oVirt Team:

---

RHEL 7.3 requirements from Atomic Host:

Cloudforms Team:

---

Target Upstream Version:

Embargoed:

Attachments:

Description	Flags
Test program that mallocs a lot of memory	none

Description Matt McCutchen 2010-03-02 06:53:01 UTC

Created attachment 397246 [details]
Test program that mallocs a lot of memory

Description of problem:
Several times recently, I've hit bugs where an application enters an infinite loop allocating memory and brings the system to a halt.  I wanted to mitigate the problem by setting an RLIMIT_DATA (ulimit -d) in my login scripts.  But RLIMIT_DATA has no effect because malloc falls back to mmap when brk fails:

http://sourceware.org/git/?p=glibc.git;a=blob;f=malloc/malloc.c;h=763852ea3b78cba613b63d9e92e00ecdd6fcaab6;hb=HEAD#l3206

Supposedly the fallback is needed on systems with address space holes.  If I'm right in thinking that Linux is not such a system, we could disable the fallback.

(Note that RLIMIT_AS does not solve my problem because OOo needs to map 2.5 GB, almost all of which is read-only, but if I set the limit that high, a process that allocated that much writable memory would bog down the system.)

Version-Release number of selected component (if applicable):
glibc-2.11.1-1.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Download and compile the attached memory-hog.c .
2. (ulimit -S -d 500000 && ./memory-hog)
  
Actual results:
"Successfully malloc-ed 1 GB"

Expected results:
"malloc failed (i = SOMETHING), exiting"

Additional info:
malloc also uses mmap for large allocations, so they don't count toward RLIMIT_DATA.  This has been noted:

https://bugs.launchpad.net/ubuntu/+source/pam/+bug/71024/comments/10
http://www.csamuel.org/2007/11/30/linux-ulimit-memory-enforcement-oddity

I'm guessing this won't affect me because infinite loops tend to involve small allocations.  But if it does, I can install an LD_PRELOAD library that calls mallopt(M_MMAP_MAX, 0) to disable the feature.

Comment 1 Andreas Schwab 2010-03-02 10:17:46 UTC

Use RLIMIT_AS.

Comment 2 Matt McCutchen 2010-03-02 14:03:52 UTC

As I said, RLIMIT_AS does not solve my problem.  Why is it not a bug that RLIMIT_DATA does not work?

Comment 3 Matt McCutchen 2010-03-05 03:29:36 UTC

Reopening because an adequate explanation of why this is not a bug was not given.

Comment 4 Andreas Schwab 2010-03-05 09:02:37 UTC

RLIMIT_DATA works as designed.

Comment 5 Matt McCutchen 2010-03-05 09:25:50 UTC

RLIMIT_DATA is useless because whenever glibc hits the limit with brk, it just calls mmap instead.  I could understand saying that RLIMIT_DATA is broken and will be left so, but saying that it works is ludicrous.

Comment 6 Andreas Schwab 2010-03-05 10:38:20 UTC

See getrlimit(2).

Comment 7 Matt McCutchen 2010-04-14 02:19:00 UTC

See bug 582072.