Bug 150209 - Over time, autofs leaks kernel memory in the size-256 slab
Summary: Over time, autofs leaks kernel memory in the size-256 slab
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 3
Classification: Red Hat
Component: kernel
Version: 3.0
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Jeff Moyer
QA Contact: Brock Organ
URL:
Whiteboard:
Depends On:
Blocks: 156320
TreeView+ depends on / blocked
 
Reported: 2005-03-03 19:33 UTC by Jeff Moyer
Modified: 2007-11-30 22:07 UTC (History)
2 users (show)

Fixed In Version: RHSA-2005-663
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2005-09-28 14:50:36 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
sysrq-m output when the problem occurs (976 bytes, text/plain)
2005-03-03 19:33 UTC, Jeff Moyer
no flags Details
contents of /proc/slabinfo (4.96 KB, text/plain)
2005-03-03 19:38 UTC, Jeff Moyer
no flags Details
Fix a memory leak in autofs4_wait (442 bytes, patch)
2005-03-04 14:35 UTC, Jeff Moyer
no flags Details | Diff


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2005:663 0 qe-ready SHIPPED_LIVE Important: Updated kernel packages available for Red Hat Enterprise Linux 3 Update 6 2005-09-28 04:00:00 UTC

Description Jeff Moyer 2005-03-03 19:33:42 UTC
Description of problem:
When running autofs and accessing/expiring mount points repeatedly, the system
will slowly leak kernel memory.  On my 128MB celeron, an over-night run of the
below test caused the system load average to hit 5, and the system to be pretty
unusable.  The size-256 slab looks like so:

size-256          365535 365535    256 24369 24369    1 :  252   63

Note that the system only has 128MB of ram, and this is taking roughly 90MB!

Version-Release number of selected component (if applicable):
2.4.21-28.ELsmp

How reproducible:
I'm not sure yet, but it is a very controlled test case, so I will know soon.

Steps to Reproduce:
1. Setup a server to export /export/<dir>
2. Create a number of directories under /export/<dir> like so:
   for n in `seq 1 48`; do mkdir $n; touch $n/$n; done
3. Now, on the client, create an automounter map to point at your server:
   (replace <IP> with the server IP)
   for n in `seq 1 48`; do echo "$n <IP>:/export/<dir>/$n" >> /etc/auto.test ; done
4. Create an entry in the auto.master on the client to point at this map:
   echo "/test /etc/auto.test --timeout=1" >> /etc/auto.master
5. Restart autofs on the client.
6. ensure that nfs services are running on the server
7. On the client, create a script that does this:
   while true; do for n in `seq 1 48`; do ls  /test/$n ; done; sleep 2; done
8. For my test, I ran two copies of this script on the client.
Actual results:
The kernel will slowly leak memory, and the system will begin to swap.  The load
average will climb, as well.

Expected results:
The kernel should not leak memory!

Additional info:
I'm in the process of trying to reproduce this without autofs in the picture.  I
get messages like this in the kernel log:

nfs_get_root: getattr error = 5
nfs_read_super: get root inode failed
nfs warning: mount version older than kernel
RPC: Can't bind to reserved port (98).

So the error cases for nfs are suspect.  If I can't reproduce this without
autofs, then I'll try to reproduce without nfs.  This can be achieved by making
the client also the server.  The client automount map would then point to
localhost, and the automound daemon will do bind mounts instead of nfs mounts.

Comment 1 Jeff Moyer 2005-03-03 19:33:42 UTC
Created attachment 111630 [details]
sysrq-m output when the problem occurs

Comment 2 Jeff Moyer 2005-03-03 19:38:47 UTC
Created attachment 111631 [details]
contents of /proc/slabinfo

Comment 3 Jeff Moyer 2005-03-04 14:35:50 UTC
Created attachment 111661 [details]
Fix a memory leak in autofs4_wait

This patch resolves the problem on my system.  Tests successfully ran over
night with no growth in the size-256 cache.

Comment 4 Jeff Moyer 2005-03-04 14:41:24 UTC
Fix the summary.

Comment 7 Jeff Moyer 2005-05-26 17:47:26 UTC
A fix for this was committed to the 2.4.21-32.1.EL kernel buld.  Setting status
to MODIFIED.

Comment 12 Ernie Petrides 2005-09-19 20:37:24 UTC
A fix for this problem was committed to the RHEL3 U6 patch
pool on 20-Apr-2005 (in kernel version 2.4.21-32.1.EL).


*** This bug has been marked as a duplicate of 160392 ***

Comment 13 Red Hat Bugzilla 2005-09-28 14:50:36 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2005-663.html



Note You need to log in before you can comment on or make changes to this bug.