Bug 104463

Summary: System memory is leaked with no way of recovering it
Product: [Retired] Red Hat Linux Reporter: Michael Lee Yohe <michael>
Component: kernelAssignee: Dave Jones <davej>
Status: CLOSED WONTFIX QA Contact: Brian Brock <bbrock>
Severity: high Docs Contact:
Priority: medium    
Version: 9CC: mattdm, pfrields, riel
Target Milestone: ---   
Target Release: ---   
Hardware: i686   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2004-09-30 15:41:32 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
/proc/slabinfo before init 3 (while still in init 5)
none
/proc/slabinfo after going into init 3
none
/proc/slabinfo after a fresh boot none

Description Michael Lee Yohe 2003-09-15 21:29:23 UTC
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4) Gecko/20030722
Galeon/1.3.8

Description of problem:
I have a Red Hat Linux 9, up2date -- latest errata packages, that is used for
web browsing, office applications, and compiling.  After about a week of doing
work, locking X, go home, come back, unlocking X (repeat) - more and more swap
space is consumed.  The page cache begins to dwindle to make room for something.

So, to recover the memory (just in case something leaked some X resources), I go
into runlevel 3, 'ps ax' for anything that should have been terminated, restart
several services, and then check my /proc/meminfo file.  What do I find?

The memory has not been recovered.

$ cat /proc/meminfo
        total:    used:    free:  shared: buffers:  cached:
Mem:  790355968 500113408 290242560        0 11538432 82984960
Swap: 2083848192        0 2083848192
MemTotal:       771832 kB
MemFree:        283440 kB
MemShared:           0 kB
Buffers:         11268 kB
Cached:          81040 kB
SwapCached:          0 kB
Active:          75912 kB
ActiveAnon:       4876 kB
ActiveCache:     71036 kB
Inact_dirty:     11352 kB
Inact_laundry:    6600 kB
Inact_clean:      3276 kB
Inact_target:    19428 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:       771832 kB
LowFree:        283440 kB
SwapTotal:     2035008 kB
SwapFree:      2035008 kB

This report was taken after taking the steps above.  As you can see, not very
much memory is tied up in buffers or the page cache.  Yet something is consuming
nearly 500M of physical memory.  Again, I checked 'ps ax' to see if left-overs
were in memory, but could not find any.

So - I reboot.

What do I find after a reboot (immediately going into runlevel 3)?

$ cat /proc/meminfo
        total:    used:    free:  shared: buffers:  cached:
Mem:  790355968 77197312 713158656        0 12333056 34881536
Swap: 2083848192        0 2083848192
MemTotal:       771832 kB
MemFree:        696444 kB
MemShared:           0 kB
Buffers:         12044 kB
Cached:          34064 kB
SwapCached:          0 kB
Active:          53236 kB
ActiveAnon:       9748 kB
ActiveCache:     43488 kB
Inact_dirty:       184 kB
Inact_laundry:       0 kB
Inact_clean:      2436 kB
Inact_target:    11168 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:       771832 kB
LowFree:        696444 kB
SwapTotal:     2035008 kB
SwapFree:      2035008 kB

I have over 700M free (which will eventually migrate over into the page cache as
I/O increases), but nonetheless.  free memory + page cache + buffers == memory
to be used by processes that need it.

But when you look at the numbers in the aforementioned meminfo, you still have
500M of memory consumed by something.

Here are my modules:

$ lsmod
Module                  Size  Used by    Not tainted
cryptoloop              2716   1
cipher-aes             23508   1
cryptoapi               7084   5 [cryptoloop cipher-aes]
imm                    11128   0 (unused)
parport_pc             19076   2 (autoclean)
lp                      8996   0 (autoclean)
parport                37056   2 (autoclean) [imm parport_pc lp]
sd_mod                 13516   0 (autoclean) (unused)
autofs                 13268   0 (autoclean) (unused)
ds                      8680   2
yenta_socket           13504   2
pcmcia_core            57216   0 [ds yenta_socket]
iptable_filter          2412   0 (autoclean) (unused)
ip_tables              15096   1 [iptable_filter]
e1000                  60288   1
sg                     36524   0 (autoclean)
sr_mod                 18136   0 (autoclean)
microcode               4668   0 (autoclean)
ide-scsi               12208   0
scsi_mod              107544   5 [imm sd_mod sg sr_mod ide-scsi]
ide-cd                 35712   0
cdrom                  33728   0 [sr_mod ide-cd]
loop                   12152   6 (autoclean) [cryptoloop]
lvm-mod                64000   0
keybdev                 2976   0 (unused)
mousedev                5556   1
hid                    22244   0 (unused)
input                   5856   0 [keybdev mousedev hid]
usb-uhci               26412   0 (unused)
ehci-hcd               20072   0 (unused)
usbcore                79040   1 [hid usb-uhci ehci-hcd]
ext3                   70784   6
jbd                    51924   6 [ext3]

How can I view the kernel's memory map to see what exactly is taking that HUGE
chunk of data that's being eaten up?  Rebooting to free memory is an AWFUL practice.

Version-Release number of selected component (if applicable):
kernel-2.4.20-20.9

How reproducible:
Always

Steps to Reproduce:
1. see description
    

Actual Results:  see description

Expected Results:  see description

Additional info:

Comment 1 Dave Jones 2003-09-15 22:30:48 UTC
cat /proc/slabinfo will show where the memory was allocated to.

You fact that memory seems to disappear when you lock X would suggest theres
a memory leak in whatever screensaver you are running. Are you running a
specific one, or do you have it set to random ? If its set to random, try
setting it to just blank screen for a few days, and see if the problem goes away.
If it does, its an xscreensaver bug


Comment 2 Michael Lee Yohe 2003-09-16 14:12:33 UTC
Another bug I filed relating an X resources leak problem was solved by Mike
Harris eluding to a buggy screensaver problem.  Locking X just blanks the
screen.  I was merely stating the routine - all the while, memory was problem
disappearing into the blue yonder and just hadn't noticed it.

Nonetheless, I'll hammer at this machine for the next day or two.  Go to init 3
and check the numbers.  If they are sufficiently bad (like before I rebooted it
previously), I'll post the slabinfo information for all to decipher.

BBL...

Comment 3 Michael Lee Yohe 2003-09-16 14:14:20 UTC
Also, screensavers that have been leaking memory usually free up all memory once
X disappears from existence (CTRL+ALT+BKSP or init 3).  Again, in runlevel 3 - I
have the same suite of resident apps as I did when I first booted into Red Hat
Linux, but a whole lot more memory was tied up.

Again... BBL.

Comment 4 Michael Lee Yohe 2003-10-31 22:02:16 UTC
Created attachment 95638 [details]
/proc/slabinfo before init 3 (while still in init 5)

Comment 5 Michael Lee Yohe 2003-10-31 22:03:24 UTC
Created attachment 95639 [details]
/proc/slabinfo after going into init 3

All X-related apps were killed.  I shutdown all non-essentially services to
recover as much memory as possible.

Comment 6 Michael Lee Yohe 2003-10-31 22:05:11 UTC
Created attachment 95640 [details]
/proc/slabinfo after a fresh boot

This problem is getting worse.	The last time I had to reboot, I had only 60M
in the page cache, buffers was about 5M, and free memory was about 20M.  There
was about 150M still tied up in swap (from what, who knows).  The machine was
so slow because of having to page in and out of swap that it took even forever
for reboot.

Comment 7 Michael Lee Yohe 2003-10-31 22:07:30 UTC
Lastly -

I have found that compiling large projects (in-house or stuff like
Mozilla, OpenOffice.org, etc.) will drain the available memory and
increase the amount of memory considered "active".  This is the only
way I've been able to duplicate the problem.

I would switch to the rawhide kernel, but util-linux hasn't been
updated to 2.12 yet, and I require something out of util-linux 2.12 if
I move over to the 2.4.22 rawhide kernel.

Comment 8 Michael Lee Yohe 2003-11-12 20:01:04 UTC
I found a way to recover some of the memory.  If you write a C program
to malloc() a quantity of memory, the page file does not grow very
much when the page cache is low/buffers are low/free mem is low.  The
amount of active memory begins to decrease and get allocated to the
application.  Once the malloc() program quits, the free memory
eventually gets returned to the page cache and buffers, thus speeding
up the machine.

This method does not appear aggressive enough to free up all the
"missing" memory.

Comment 9 Bugzilla owner 2004-09-30 15:41:32 UTC
Thanks for the bug report. However, Red Hat no longer maintains this version of
the product. Please upgrade to the latest version and open a new bug if the problem
persists.

The Fedora Legacy project (http://fedoralegacy.org/) maintains some older releases, 
and if you believe this bug is interesting to them, please report the problem in
the bug tracker at: http://bugzilla.fedora.us/