Bug 762991 (GLUSTER-1259)

Summary: Memory leaks to 5.7g usage in gnfs
Product: [Community] GlusterFS Reporter: Shehjar Tikoo <shehjart>
Component: nfsAssignee: Shehjar Tikoo <shehjart>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: high Docs Contact:
Priority: low    
Version: nfs-alphaCC: gluster-bugs
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: ---
Regression: RTP Mount Type: nfs
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Shehjar Tikoo 2010-08-02 07:58:27 UTC
Not sure where this is coming from yet but a core dump showed glusterfsd with nfs using 5.7g memory during long running tests which were running for one week. The tests included compilebench and postmark on 2 clients.

We need a thorough investigation into this.

Comment 1 Shehjar Tikoo 2010-09-23 08:54:40 UTC
Simplest way to reproduce:

o Setup a simple posix exported through nfs.

o Mount on nfs client.

o Start top at the server.

o Run the following command and see memory increase continuously.

while [ true ]; do echo Copying..;cp -R /etc/ /mnt/;echo Removing..;rm -rf /mnt/*;done

Comment 2 Shehjar Tikoo 2010-09-23 09:53:54 UTC
The leak is somewhere in the file/dir inode_t creation path. Investigating..

Comment 3 Shehjar Tikoo 2010-09-27 09:13:45 UTC
Possible inode leak paths:

o NFS LOOKUP followed by another NFS LOOKUP
o NFS LOOKUP or CREATE followed by NFS REMOVE

Comment 4 Shehjar Tikoo 2010-09-28 02:26:15 UTC
First memory leak path:

In root dir, create a file, then remove it. Just doing this results in inode of the newly created file to leak.

Commands to use:

1. touch <MNT-POINT>/testfile
2. rm -f <MNT-POINT>/testfile


Second memory leak path:

1. mkdir <MNT-POINT>/testdir
2. touch <MNT-POINT>/testdir/testfile
3. ls <MNT-POINT>/testdir

Wait 30 secs.

4. stat <MNT-POINT>/testdir

The fd for testdir has leaked by this time. I think it is because when a dir inode is used as a parent inode, it is not being unrefd correctly.

Comment 5 Vijay Bellur 2010-09-28 13:04:49 UTC
PATCH: http://patches.gluster.com/patch/5031 in master (nfs: Free inodes layer fop locals to fix memory leak)