Bug 175376 - Make uses too much memory
Summary: Make uses too much memory
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 4
Classification: Red Hat
Component: make
Version: 4.3
Hardware: All
OS: Linux
medium
high
Target Milestone: ---
: ---
Assignee: Petr Machata
QA Contact: Brian Brock
URL:
Whiteboard:
Depends On:
Blocks: 181409
TreeView+ depends on / blocked
 
Reported: 2005-12-09 16:41 UTC by H.J. Lu
Modified: 2015-05-05 01:32 UTC (History)
1 user (show)

Fixed In Version: RHEA-2006-0359
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2006-08-10 21:15:57 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
A patch (6.23 KB, patch)
2005-12-09 16:44 UTC, H.J. Lu
no flags Details | Diff
A testcase (29.81 KB, application/octet-stream)
2005-12-09 17:01 UTC, H.J. Lu
no flags Details
An updated patch (6.23 KB, patch)
2006-02-01 19:09 UTC, H.J. Lu
no flags Details | Diff
Automated testcase. (1.19 KB, application/x-shellscript)
2006-03-29 15:00 UTC, Petr Machata
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2006:0359 0 normal SHIPPED_LIVE make enhancement update 2006-08-09 04:00:00 UTC

Description H.J. Lu 2005-12-09 16:41:59 UTC
In some cases, make uses too much memory. For example:

[hjl@gnu-d libjava]$ make
ulimit -v 1048576; \
/usr/bin/time make -f gnu-src-gcc.deps
make[1]: Entering directory `/export/home/hjl/bugs/make/libjava'
make[1]: *** virtual memory exhausted.  Stop.
make[1]: Leaving directory `/export/home/hjl/bugs/make/libjava'
Command exited with non-zero status 2
5.94user 2.78system 0:22.33elapsed 39%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (279major+261891minor)pagefaults 0swaps
make: *** [all] Error 2

Comment 1 H.J. Lu 2005-12-09 16:44:19 UTC
Created attachment 122081 [details]
A patch

GNU make 3.80 is a HUGE memory hog. It calls xstrdup to build
dependency list. gnu-src-gcc.deps in libjava has 3000+ targets depend
the same 3000+ files, whose filenames are more than 260K. For this
dependency alone, make takes 3000*260K == 761MB.

This patch is a quick hack. It reduces the memory from 1.6GB to around
600MB for "make -f gnu-src-gcc.deps". I got

[hjl@gnu-13 libjava]$ make
ulimit -v 1048576; \
/usr/bin/time ./make -f gnu-src-gcc.deps
make[1]: Entering directory `/export/home/hjl/bugs/make/libjava'
make[1]: *** No rule to make target `../gnu/classpath/Configuration.java',
needed by `lists/gnu-src-gcc.stamp'.  Stop.
make[1]: Leaving directory `/export/home/hjl/bugs/make/libjava'
Command exited with non-zero status 2
67.10user 1.24system 1:16.63elapsed 89%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+140611minor)pagefaults 0swaps
make: *** [all] Error 2

Comment 2 H.J. Lu 2005-12-09 17:01:38 UTC
Created attachment 122083 [details]
A testcase

I don't know why my testcase wasn't uploaded when I opened the bug report.

Comment 3 Jakub Jelinek 2005-12-10 14:01:28 UTC
Have you reported the problem upstream and posted a patch there?


Comment 5 Petr Machata 2006-01-09 18:51:48 UTC
I applied the patch.  Should be in rawhide tomorrow.  I'm adding review of
make's memory management to my longterm TODO.

Comment 6 H.J. Lu 2006-02-01 19:09:03 UTC
Created attachment 123984 [details]
An updated patch

> I just want to inform you, that we got a problem report regarding your
> "memory hog" patch from 2005-Dec-08 for gmake-3.80. If compilation of
> "binutils" is started with "MALLOC_CHECK_=2", a crash in gmake occurs.
>
> I digged a bit deeper into it and noticed this:
>

My memory hog patch for make has 2 typos. This patch fixes them.

Comment 7 H.J. Lu 2006-02-01 19:09:42 UTC
Please use the new patch.

Comment 8 Petr Machata 2006-02-02 14:23:50 UTC
Thank you, it's in rawhide.

Comment 12 Bob Johnson 2006-04-11 16:53:42 UTC
This issue is on Red Hat Engineering's list of planned work items 
for the upcoming Red Hat Enterprise Linux 4.4 release.  Engineering 
resources have been assigned and barring unforeseen circumstances, Red 
Hat intends to include this item in the 4.4 release.

Comment 15 Red Hat Bugzilla 2006-08-10 21:15:57 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHEA-2006-0359.html



Note You need to log in before you can comment on or make changes to this bug.