Bug 598498

Summary: libc allocator in RHEL6: leaks anonymous mmap areas
Product: Red Hat Enterprise Linux 6 Reporter: Török Edwin <edwin+bugs>
Component: glibcAssignee: Andreas Schwab <schwab>
Status: CLOSED NOTABUG QA Contact: qe-baseos-tools-bugs
Severity: high Docs Contact:
Priority: low    
Version: 6.0CC: fweimer, ladar, notting
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-06-04 13:05:51 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 640286    
Bug Blocks:    
Attachments:
Description Flags
strace log
none
pmap of clamd on RHEL6
none
pmap of clamd on RHEL5
none
dmesg on RHEL6 none

Description Török Edwin 2010-06-01 14:04:55 UTC 
Description of problem:
Running ClamAV's 'make check' on RHEL6 beta I get out of memory errors when running under ulimit -v 512000.
On RHEL5 it doesn't run out of memory (or on any other system either).

If I build tcmalloc, and link with -ltcmalloc then I do not get out of memory errors. Also if I run under valgrind I don't get out of memory errors (although valgrind itself uses quite some memory)!

Version-Release number of selected component (if applicable):
/lib64/libc.so.6
GNU C Library stable release version 2.11.1, by Roland McGrath et al.
Copyright (C) 2009 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 4.4.3 20100121 (Red Hat 4.4.3-1).
Compiled on a Linux >>2.6.18-164.11.1.el5<< system on 2010-02-25.
Available extensions:
	The C stubs add-on version 2.1.2.
	crypt add-on version 2.1 by Michael Glad and others
	GNU Libidn by Simon Josefsson
	Native POSIX Threads Library by Ulrich Drepper et al
	BIND-8.2.3-T5B
	RT using linux kernel aio
For bug reporting instructions, please see:
<http://www.gnu.org/software/libc/bugs.html>.


How reproducible:
Always

Steps to Reproduce:
1. Download ClamAV-0.96.1
$ wget http://downloads.sourceforge.net/clamav/clamav-0.96.1.tar.gz
2. ./configure --disable-llvm --disable-clamav CFLAGS="-O0 -g"
3. make -j2
4. cat >clamd.conf <<EOF
DatabaseDirectory /tmp/db
LocalSocket /tmp/clamd.socket
Foreground Yes
EOF
5. mkdir /tmp/db && cat >/tmp/db/last.hdb <<EOF
aa15bcf478d165efd2065190eb473bcb:544:ClamAV-Test-File
ccc692860fe80ed8b88334356878fe26:20:Internal-Test-Signature
EOF
6. ulimit -d 512000
7. ulimit -v 512000
8. clamd/clamd -c clamd.conf 2>log&
9. clamdscan/clamdscan -c clamd.conf -m test/.split/

Actual results:
/root/git/clamav-devel/test/.split/split.clam-mew.exeab:  Can't allocate memory
ERROR
...
Total errors: 34

Expected results:
No "Can't allocate memory" errors, and "Total errors: 0"

Additional info:
See ClamAV bugreport for more details: 
https://wwws.clamav.net/bugzilla/show_bug.cgi?id=1990
https://wwws.clamav.net/bugzilla/show_bug.cgi?id=1990#c6

I'm attaching the strace and pmap output.
Comment 2 Török Edwin 2010-06-01 14:07:18 UTC 
Created attachment 418648 [details]
strace log

./libtool --mode=execute strace -Ff clamd/clamd -c clamd.conf 2>strace_log&
Comment 3 Török Edwin 2010-06-01 14:07:59 UTC 
Created attachment 418649 [details]
pmap of clamd on RHEL6

pmap of clamd on RHEL6 after clamdscan completed
Comment 4 Török Edwin 2010-06-01 14:11:27 UTC 
Created attachment 418651 [details]
pmap of clamd on RHEL5

pmap of clamd on RHEL5:
Linux 2.6.18-92.1.13.el5xen x86_64 x86_64
/lib64/libc.so.6 
GNU C Library stable release version 2.5, by Roland McGrath et al.
Copyright (C) 2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 4.1.2 20071124 (Red Hat 4.1.2-41).
Compiled on a Linux 2.6.9 system on 2008-04-11.
Available extensions:
        The C stubs add-on version 2.1.2.
        crypt add-on version 2.1 by Michael Glad and others
        GNU Libidn by Simon Josefsson
        GNU libio by Per Bothner
        NIS(YP)/NIS+ NSS modules 0.19 by Thorsten Kukuk
        Native POSIX Threads Library by Ulrich Drepper et al
        BIND-8.2.3-T5B
        RT using linux kernel aio
Thread-local storage support included.
For bug reporting instructions, please see:
<http://www.gnu.org/software/libc/bugs.html>.
Comment 5 Török Edwin 2010-06-01 14:11:58 UTC 
Created attachment 418652 [details]
dmesg on RHEL6
Comment 6 Török Edwin 2010-06-01 14:12:45 UTC 
Note that on Debian unstable with same ulimit setting I do NOT get the out of memory, although libc version is almost the same.
$ uname -mrsp
Linux 2.6.34-00179-gdebb980 x86_64 unknown
$ /lib/libc.so.6 
GNU C Library (Debian EGLIBC 2.11.1-1) stable release version 2.11.1, by Roland McGrath et al.
Copyright (C) 2009 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 4.4.4.
Compiled on a Linux 2.6.32 system on 2010-05-29.
Available extensions:
        crypt add-on version 2.1 by Michael Glad and others
        GNU Libidn by Simon Josefsson
        Native POSIX Threads Library by Ulrich Drepper et al
        BIND-8.2.3-T5B
For bug reporting instructions, please see:
<http://www.debian.org/Bugs/>.
Comment 7 Török Edwin 2010-06-01 14:13:59 UTC 
Please let me know if you need additional information to debug this problem.
Right now this is a showstopper for us on RHEL6 beta.
Comment 8 RHEL Program Management 2010-06-02 18:05:46 UTC 
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux major release.  Product Management has requested further
review of this request by Red Hat Engineering, for potential inclusion in a Red
Hat Enterprise Linux Major release.  This request is not yet committed for
inclusion.
Comment 9 Andreas Schwab 2010-06-04 13:05:51 UTC 
These are not leaks but per-thread heaps.
Comment 10 Török Edwin 2010-06-04 13:42:05 UTC 
(In reply to comment #9)
> These are not leaks but per-thread heaps.    

OK, but that doesn't explain why it uses more memory than on Debian, the libc version is similar.
Is it possible to tune/trim the size of these heaps?
Comment 11 Andreas Schwab 2010-06-04 13:55:57 UTC 
--enable-experimental-malloc
Comment 12 Török Edwin 2010-06-04 14:04:02 UTC 
Thanks, I'll see if I can do anything to lower the memory usage of ClamAV under --enable-experimental-malloc.
Will the final RHEL6 have that flag enabled?
Comment 13 Török Edwin 2010-10-05 13:22:23 UTC 
Wrote a testcase and opened #640286, since I don't seem to be able to reopen this bug.