Bug 175376 - Make uses too much memory
Make uses too much memory
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 4
Classification: Red Hat
Component: make (Show other bugs)
4.3
All Linux
medium Severity high
: ---
: ---
Assigned To: Petr Machata
Brian Brock
:
Depends On:
Blocks: 181409
  Show dependency treegraph
 
Reported: 2005-12-09 11:41 EST by H.J. Lu
Modified: 2015-05-04 21:32 EDT (History)
1 user (show)

See Also:
Fixed In Version: RHEA-2006-0359
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2006-08-10 17:15:57 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
A patch (6.23 KB, patch)
2005-12-09 11:44 EST, H.J. Lu
no flags Details | Diff
A testcase (29.81 KB, application/octet-stream)
2005-12-09 12:01 EST, H.J. Lu
no flags Details
An updated patch (6.23 KB, patch)
2006-02-01 14:09 EST, H.J. Lu
no flags Details | Diff
Automated testcase. (1.19 KB, application/x-shellscript)
2006-03-29 10:00 EST, Petr Machata
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2006:0359 normal SHIPPED_LIVE make enhancement update 2006-08-09 00:00:00 EDT

  None (edit)
Description H.J. Lu 2005-12-09 11:41:59 EST
In some cases, make uses too much memory. For example:

[hjl@gnu-d libjava]$ make
ulimit -v 1048576; \
/usr/bin/time make -f gnu-src-gcc.deps
make[1]: Entering directory `/export/home/hjl/bugs/make/libjava'
make[1]: *** virtual memory exhausted.  Stop.
make[1]: Leaving directory `/export/home/hjl/bugs/make/libjava'
Command exited with non-zero status 2
5.94user 2.78system 0:22.33elapsed 39%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (279major+261891minor)pagefaults 0swaps
make: *** [all] Error 2
Comment 1 H.J. Lu 2005-12-09 11:44:19 EST
Created attachment 122081 [details]
A patch

GNU make 3.80 is a HUGE memory hog. It calls xstrdup to build
dependency list. gnu-src-gcc.deps in libjava has 3000+ targets depend
the same 3000+ files, whose filenames are more than 260K. For this
dependency alone, make takes 3000*260K == 761MB.

This patch is a quick hack. It reduces the memory from 1.6GB to around
600MB for "make -f gnu-src-gcc.deps". I got

[hjl@gnu-13 libjava]$ make
ulimit -v 1048576; \
/usr/bin/time ./make -f gnu-src-gcc.deps
make[1]: Entering directory `/export/home/hjl/bugs/make/libjava'
make[1]: *** No rule to make target `../gnu/classpath/Configuration.java',
needed by `lists/gnu-src-gcc.stamp'.  Stop.
make[1]: Leaving directory `/export/home/hjl/bugs/make/libjava'
Command exited with non-zero status 2
67.10user 1.24system 1:16.63elapsed 89%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+140611minor)pagefaults 0swaps
make: *** [all] Error 2
Comment 2 H.J. Lu 2005-12-09 12:01:38 EST
Created attachment 122083 [details]
A testcase

I don't know why my testcase wasn't uploaded when I opened the bug report.
Comment 3 Jakub Jelinek 2005-12-10 09:01:28 EST
Have you reported the problem upstream and posted a patch there?
Comment 5 Petr Machata 2006-01-09 13:51:48 EST
I applied the patch.  Should be in rawhide tomorrow.  I'm adding review of
make's memory management to my longterm TODO.
Comment 6 H.J. Lu 2006-02-01 14:09:03 EST
Created attachment 123984 [details]
An updated patch

> I just want to inform you, that we got a problem report regarding your
> "memory hog" patch from 2005-Dec-08 for gmake-3.80. If compilation of
> "binutils" is started with "MALLOC_CHECK_=2", a crash in gmake occurs.
>
> I digged a bit deeper into it and noticed this:
>

My memory hog patch for make has 2 typos. This patch fixes them.
Comment 7 H.J. Lu 2006-02-01 14:09:42 EST
Please use the new patch.
Comment 8 Petr Machata 2006-02-02 09:23:50 EST
Thank you, it's in rawhide.
Comment 12 Bob Johnson 2006-04-11 12:53:42 EDT
This issue is on Red Hat Engineering's list of planned work items 
for the upcoming Red Hat Enterprise Linux 4.4 release.  Engineering 
resources have been assigned and barring unforeseen circumstances, Red 
Hat intends to include this item in the 4.4 release.
Comment 15 Red Hat Bugzilla 2006-08-10 17:15:57 EDT
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHEA-2006-0359.html

Note You need to log in before you can comment on or make changes to this bug.