Hide Forgot
Description of problem: Running ClamAV's 'make check' on RHEL6 beta I get out of memory errors when running under ulimit -v 512000. On RHEL5 it doesn't run out of memory (or on any other system either). If I build tcmalloc, and link with -ltcmalloc then I do not get out of memory errors. Also if I run under valgrind I don't get out of memory errors (although valgrind itself uses quite some memory)! Version-Release number of selected component (if applicable): /lib64/libc.so.6 GNU C Library stable release version 2.11.1, by Roland McGrath et al. Copyright (C) 2009 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Compiled by GNU CC version 4.4.3 20100121 (Red Hat 4.4.3-1). Compiled on a Linux >>2.6.18-164.11.1.el5<< system on 2010-02-25. Available extensions: The C stubs add-on version 2.1.2. crypt add-on version 2.1 by Michael Glad and others GNU Libidn by Simon Josefsson Native POSIX Threads Library by Ulrich Drepper et al BIND-8.2.3-T5B RT using linux kernel aio For bug reporting instructions, please see: <http://www.gnu.org/software/libc/bugs.html>. How reproducible: Always Steps to Reproduce: 1. Download ClamAV-0.96.1 $ wget http://downloads.sourceforge.net/clamav/clamav-0.96.1.tar.gz 2. ./configure --disable-llvm --disable-clamav CFLAGS="-O0 -g" 3. make -j2 4. cat >clamd.conf <<EOF DatabaseDirectory /tmp/db LocalSocket /tmp/clamd.socket Foreground Yes EOF 5. mkdir /tmp/db && cat >/tmp/db/last.hdb <<EOF aa15bcf478d165efd2065190eb473bcb:544:ClamAV-Test-File ccc692860fe80ed8b88334356878fe26:20:Internal-Test-Signature EOF 6. ulimit -d 512000 7. ulimit -v 512000 8. clamd/clamd -c clamd.conf 2>log& 9. clamdscan/clamdscan -c clamd.conf -m test/.split/ Actual results: /root/git/clamav-devel/test/.split/split.clam-mew.exeab: Can't allocate memory ERROR ... Total errors: 34 Expected results: No "Can't allocate memory" errors, and "Total errors: 0" Additional info: See ClamAV bugreport for more details: https://wwws.clamav.net/bugzilla/show_bug.cgi?id=1990 https://wwws.clamav.net/bugzilla/show_bug.cgi?id=1990#c6 I'm attaching the strace and pmap output.
Created attachment 418648 [details] strace log ./libtool --mode=execute strace -Ff clamd/clamd -c clamd.conf 2>strace_log&
Created attachment 418649 [details] pmap of clamd on RHEL6 pmap of clamd on RHEL6 after clamdscan completed
Created attachment 418651 [details] pmap of clamd on RHEL5 pmap of clamd on RHEL5: Linux 2.6.18-92.1.13.el5xen x86_64 x86_64 /lib64/libc.so.6 GNU C Library stable release version 2.5, by Roland McGrath et al. Copyright (C) 2006 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Compiled by GNU CC version 4.1.2 20071124 (Red Hat 4.1.2-41). Compiled on a Linux 2.6.9 system on 2008-04-11. Available extensions: The C stubs add-on version 2.1.2. crypt add-on version 2.1 by Michael Glad and others GNU Libidn by Simon Josefsson GNU libio by Per Bothner NIS(YP)/NIS+ NSS modules 0.19 by Thorsten Kukuk Native POSIX Threads Library by Ulrich Drepper et al BIND-8.2.3-T5B RT using linux kernel aio Thread-local storage support included. For bug reporting instructions, please see: <http://www.gnu.org/software/libc/bugs.html>.
Created attachment 418652 [details] dmesg on RHEL6
Note that on Debian unstable with same ulimit setting I do NOT get the out of memory, although libc version is almost the same. $ uname -mrsp Linux 2.6.34-00179-gdebb980 x86_64 unknown $ /lib/libc.so.6 GNU C Library (Debian EGLIBC 2.11.1-1) stable release version 2.11.1, by Roland McGrath et al. Copyright (C) 2009 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Compiled by GNU CC version 4.4.4. Compiled on a Linux 2.6.32 system on 2010-05-29. Available extensions: crypt add-on version 2.1 by Michael Glad and others GNU Libidn by Simon Josefsson Native POSIX Threads Library by Ulrich Drepper et al BIND-8.2.3-T5B For bug reporting instructions, please see: <http://www.debian.org/Bugs/>.
Please let me know if you need additional information to debug this problem. Right now this is a showstopper for us on RHEL6 beta.
This request was evaluated by Red Hat Product Management for inclusion in a Red Hat Enterprise Linux major release. Product Management has requested further review of this request by Red Hat Engineering, for potential inclusion in a Red Hat Enterprise Linux Major release. This request is not yet committed for inclusion.
These are not leaks but per-thread heaps.
(In reply to comment #9) > These are not leaks but per-thread heaps. OK, but that doesn't explain why it uses more memory than on Debian, the libc version is similar. Is it possible to tune/trim the size of these heaps?
--enable-experimental-malloc
Thanks, I'll see if I can do anything to lower the memory usage of ClamAV under --enable-experimental-malloc. Will the final RHEL6 have that flag enabled?
Wrote a testcase and opened #640286, since I don't seem to be able to reopen this bug.