Bug 1722698

Summary: DHT: severe memory leak in dht rename
Product: [Community] GlusterFS Reporter: Nithya Balachandran <nbalacha>
Component: distributeAssignee: Nithya Balachandran <nbalacha>
Status: CLOSED NEXTRELEASE QA Contact:
Severity: high Docs Contact:
Priority: high    
Version: mainlineCC: bugs, rhs-bugs, sankarshan, storage-qa-internal, tdesala
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1722512
: 1726294 1739337 (view as bug list) Environment:
Last Closed: 2019-07-02 11:52:04 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1722512    
Bug Blocks: 1726294, 1739337    

Description Nithya Balachandran 2019-06-21 03:36:31 UTC
+++ This bug was initially created as a clone of Bug #1722512 +++

Description of problem:

The dht rename codepath has a severe leak if it needs to create a linkto file. 
Version-Release number of selected component (if applicable):


How reproducible:
Consistently

Steps to Reproduce:
1. Create a 2x3 distribute replicate volume and fuse mount it.
2. Create 2 directories, dir1 and dir1-new in the root of the volume
3. Find 2 filenames which will hash to different subvols when created in these directories. For example, in my setup dir1/file-1 and dir1-new/newfile-1 hash to different subvols. This is necessary as the leak is in the path which creates a linkto file.
4. Run the following script and watch the memory usage for the mount process using top.



Actual results:
Memory rises steadily. Statedumps show that the number of active inodes keeps increasing.

Expected results:
Memory should not increase as there is a single file on the volume.


Additional info:

This is a regression introduced by https://code.engineering.redhat.com/gerrit/#/c/154933/ in RHGS 3.4.2

--- Additional comment from RHEL Product and Program Management on 2019-06-20 14:02:20 UTC ---

This bug is automatically being proposed for the next minor release of Red Hat Gluster Storage by setting the release flag 'rhgs‑3.5.0' to '?'. 

If this bug should be proposed for a different release, please manually change the proposed release flag.

--- Additional comment from RHEL Product and Program Management on 2019-06-20 14:05:13 UTC ---

This bug report has Keywords: Regression or TestBlocker.

Since no regressions or test blockers are allowed between releases, it is also being identified as a blocker for this release.

Please resolve ASAP.

--- Additional comment from Nithya Balachandran on 2019-06-20 14:10:03 UTC ---

(In reply to Nithya Balachandran from comment #0)
> Description of problem:
> 
> The dht rename codepath has a severe leak if it needs to create a linkto
> file. 
> Version-Release number of selected component (if applicable):
> 
> 
> How reproducible:
> Consistently
> 
> Steps to Reproduce:
> 1. Create a 2x3 distribute replicate volume and fuse mount it.
> 2. Create 2 directories, dir1 and dir1-new in the root of the volume
> 3. Find 2 filenames which will hash to different subvols when created in
> these directories. For example, in my setup dir1/file-1 and
> dir1-new/newfile-1 hash to different subvols. This is necessary as the leak
> is in the path which creates a linkto file.
> 4. Run the following script and watch the memory usage for the mount process
> using top.
> 
> 
Forgot to mention the script in the description:

while (true); do for in in {1..20000}; do touch /mnt/fuse1/dir1/file-1; mv -f /mnt/fuse1/dir1/file-1 /mnt/fuse1/dir1-new/newfile-1; done ;rm -rf /mnt/fuse1/dir1-new/*; done

Comment 1 Worker Ant 2019-06-21 03:38:32 UTC
REVIEW: https://review.gluster.org/22912 (cluster/dht:  Fixed a memleak in dht_rename_cbk) posted (#1) for review on master by N Balachandran

Comment 2 Worker Ant 2019-07-02 11:52:04 UTC
REVIEW: https://review.gluster.org/22912 (cluster/dht:  Fixed a memleak in dht_rename_cbk) merged (#5) on master by N Balachandran