Red Hat Bugzilla – Bug 178367
kernel memory leak while reading from /proc/cluster/[nodes|services]
Last modified: 2009-04-16 16:28:12 EDT
Description of problem:
If you read from /proc/cluster/services or /proc/cluster/nodes, CMAN appears to
leak memory if you watch /proc/slabinfo, specifically in the size-32 section.
Version-Release number of selected component (if applicable): 15-Jan-2006 CVS /
How reproducible: 100%
Steps to Reproduce:
1. Form a one-node cluster. Don't start fenced or rgmanager. Do not load the
DLM module or mount GFS file systems.
2. In one terminal window, run:
while : ; do cat /proc/cluster/services ; done &> /dev/null
3. In another terminal window, run:
watch --interval=1 "cat /proc/slabinfo | grep size-32\ "
Actual results: slab-32 grows unbounded, suggesting a memory leak when we read
Expected results: ?
I originally thought this was a memory leak in the DLM, which turned out to be
Created attachment 123464 [details]
Proposed patch for /proc/cluster/services memory leak
This patch contains three things:
1. Patrick Caulfield's code fix for bz 177163 as of 19 January 2006.
2. Bob Peterson's patch for adding /proc/cluster/smsg_history and
/proc/cluster/msg_history for cman-kernel debugging purposes.
3. Bob Peterson's proposed patch to fix the memory leak when using
/proc/cluster/services and /proc/cluster/nodes
The fix is mainly for debugging purposes. The patch as it stands allocates a
bunch of memory for the module that shouldn't be necessary for normal customer
situations. I'm planning to revise it to make it more useful, i.e. able to
turn it on/off with a customer command and allocate/deallocate the memory as
needed by doing insmod of a companion module or possibly by echoing something
to another file in /proc/cluster/, etc. This way, if a customer has a problem,
we can request that they turn the patch on and take information from the
failure, while still not impacting other customers.
I researched the proper way to use seq_files for /proc to make sure this fix
would work properly. For example, see:
Excerpt from that document:
"struct seq_file contains a "void *private" that can be used by the
struct seq_operations functions to hold any private data that needs
to be available to all of these related methods. For example, the
.start method might allocate some memory and save its address in
seq_file.private so that the .next and .show methods can use it,
then the .stop method would free that memory."
Also, I unit-tested this fix with printk messages to verify it was working as
planned and all kmallocs were paired with their proper kfrees.
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.