Bug 1658979

Summary: High memory usage by glusterfs client process
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Jay Samson <jpankaja>
Component: glusterfsAssignee: Sunny Kumar <sunkumar>
Status: CLOSED WORKSFORME QA Contact: Bala Konda Reddy M <bmekala>
Severity: medium Docs Contact:
Priority: medium    
Version: rhgs-3.4CC: jpankaja, nravinas, rhs-bugs, sankarshan, sunkumar, vbellur, woluty
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-02-07 04:17:26 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1647277    
Bug Blocks:    
Attachments:
Description Flags
glusterfs client statedump none

Comment 6 wang 2019-01-25 08:22:42 UTC 
Created attachment 1523446 [details]
glusterfs client statedump

I have the same problem。but my glusterfs version is 3.13.2。
（一）、glusterfs version info
glusterfs 3.13.2
Repository revision: git://git.gluster.org/glusterfs.git
Copyright (c) 2006-2016 Red Hat, Inc. <https://www.gluster.org/>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
It is licensed to you under your choice of the GNU Lesser
General Public License, version 3 or any later version (LGPLv3
or later), or the GNU General Public License, version 2 (GPLv2),
in all cases as published by the Free Software Foundation.
（二）、glusterfs memory info（three days ago，the glusterfs RSS has reached 150G，I have to remount the glusterfs）
 PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND                                                                                  
 3610 root      20   0 8204272 7.160g   4388 S   1.7  2.9   1948:13 glusterfs                                                                                
 3658 root      20   0 4219832 712408   4540 S   0.0  0.3   1291:37 glusterfsd                                                                               
 3708 root      20   0 3231104 389320   4424 S   0.3  0.1  44:40.75 glusterfsd                                                                               
 3726 root      20   0 3239296 263720   4428 S   0.3  0.1  72:39.66 glusterfsd                                                                               
 3682 root      20   0 3303044 236828   4424 S   0.7  0.1  53:37.38 glusterfsd                                                                               
 3752 root      20   0 3296900 224340   4420 S   0.7  0.1  71:58.79 glusterfsd
（三）、glusterfs volume info
Volume Name: GFSvolume
Type: Distributed-Replicate
Volume ID: f77eedac-95da-40e7-a1cb-8fd0c097ef75
Status: Started
Snapshot Count: 0
Number of Bricks: 24 x 3 = 72
Transport-type: tcp
Bricks:
Brick1: XXXXXXXXXXXXXXXXXXX01-32-146:/GFSbrick1/GFSdata
Brick2: XXXXXXXXXXXXXXXXXXX02-32-37:/GFSbrick1/GFSdata
Brick3: XXXXXXXXXXXXXXXXXXX03-32-130:/GFSbrick1/GFSdata
Brick4: XXXXXXXXXXXXXXXXXXX04-32-48:/GFSbrick1/GFSdata
Brick5: XXXXXXXXXXXXXXXXXXX05-32-132:/GFSbrick1/GFSdata
Brick6: XXXXXXXXXXXXXXXXXXX06-32-84:/GFSbrick1/GFSdata
Brick7: XXXXXXXXXXXXXXXXXXX01-32-146:/GFSbrick2/GFSdata
Brick8: XXXXXXXXXXXXXXXXXXX02-32-37:/GFSbrick2/GFSdata
Brick9: XXXXXXXXXXXXXXXXXXX03-32-130:/GFSbrick2/GFSdata
Brick10: XXXXXXXXXXXXXXXXXXX04-32-48:/GFSbrick2/GFSdata
Brick11: XXXXXXXXXXXXXXXXXXX05-32-132:/GFSbrick2/GFSdata
Brick12: XXXXXXXXXXXXXXXXXXX06-32-84:/GFSbrick2/GFSdata
Brick13: XXXXXXXXXXXXXXXXXXX01-32-146:/GFSbrick3/GFSdata
Brick14: XXXXXXXXXXXXXXXXXXX02-32-37:/GFSbrick3/GFSdata
Brick15: XXXXXXXXXXXXXXXXXXX03-32-130:/GFSbrick3/GFSdata
Brick16: XXXXXXXXXXXXXXXXXXX04-32-48:/GFSbrick3/GFSdata
Brick17: XXXXXXXXXXXXXXXXXXX05-32-132:/GFSbrick3/GFSdata
Brick18: XXXXXXXXXXXXXXXXXXX06-32-84:/GFSbrick3/GFSdata
Brick19: XXXXXXXXXXXXXXXXXXX01-32-146:/GFSbrick4/GFSdata
Brick20: XXXXXXXXXXXXXXXXXXX02-32-37:/GFSbrick4/GFSdata
Brick21: XXXXXXXXXXXXXXXXXXX03-32-130:/GFSbrick4/GFSdata
Brick22: XXXXXXXXXXXXXXXXXXX04-32-48:/GFSbrick4/GFSdata
Brick23: XXXXXXXXXXXXXXXXXXX05-32-132:/GFSbrick4/GFSdata
Brick24: XXXXXXXXXXXXXXXXXXX06-32-84:/GFSbrick4/GFSdata
Brick25: XXXXXXXXXXXXXXXXXXX01-32-146:/GFSbrick5/GFSdata
Brick26: XXXXXXXXXXXXXXXXXXX02-32-37:/GFSbrick5/GFSdata
Brick27: XXXXXXXXXXXXXXXXXXX03-32-130:/GFSbrick5/GFSdata
Brick28: XXXXXXXXXXXXXXXXXXX04-32-48:/GFSbrick5/GFSdata
Brick29: XXXXXXXXXXXXXXXXXXX05-32-132:/GFSbrick5/GFSdata
Brick30: XXXXXXXXXXXXXXXXXXX06-32-84:/GFSbrick5/GFSdata
Brick31: XXXXXXXXXXXXXXXXXXX01-32-146:/GFSbrick6/GFSdata
Brick32: XXXXXXXXXXXXXXXXXXX02-32-37:/GFSbrick6/GFSdata
Brick33: XXXXXXXXXXXXXXXXXXX03-32-130:/GFSbrick6/GFSdata
Brick34: XXXXXXXXXXXXXXXXXXX04-32-48:/GFSbrick6/GFSdata
Brick35: XXXXXXXXXXXXXXXXXXX05-32-132:/GFSbrick6/GFSdata
Brick36: XXXXXXXXXXXXXXXXXXX06-32-84:/GFSbrick6/GFSdata
Brick37: XXXXXXXXXXXXXXXXXXX01-32-146:/GFSbrick7/GFSdata
Brick38: XXXXXXXXXXXXXXXXXXX02-32-37:/GFSbrick7/GFSdata
Brick39: XXXXXXXXXXXXXXXXXXX03-32-130:/GFSbrick7/GFSdata
Brick40: XXXXXXXXXXXXXXXXXXX04-32-48:/GFSbrick7/GFSdata
Brick41: XXXXXXXXXXXXXXXXXXX05-32-132:/GFSbrick7/GFSdata
Brick42: XXXXXXXXXXXXXXXXXXX06-32-84:/GFSbrick7/GFSdata
Brick43: XXXXXXXXXXXXXXXXXXX01-32-146:/GFSbrick8/GFSdata
Brick44: XXXXXXXXXXXXXXXXXXX02-32-37:/GFSbrick8/GFSdata
Brick45: XXXXXXXXXXXXXXXXXXX03-32-130:/GFSbrick8/GFSdata
Brick46: XXXXXXXXXXXXXXXXXXX04-32-48:/GFSbrick8/GFSdata
Brick47: XXXXXXXXXXXXXXXXXXX05-32-132:/GFSbrick8/GFSdata
Brick48: XXXXXXXXXXXXXXXXXXX06-32-84:/GFSbrick8/GFSdata
Brick49: XXXXXXXXXXXXXXXXXXX01-32-146:/GFSbrick9/GFSdata
Brick50: XXXXXXXXXXXXXXXXXXX02-32-37:/GFSbrick9/GFSdata
Brick51: XXXXXXXXXXXXXXXXXXX03-32-130:/GFSbrick9/GFSdata
Brick52: XXXXXXXXXXXXXXXXXXX04-32-48:/GFSbrick9/GFSdata
Brick53: XXXXXXXXXXXXXXXXXXX05-32-132:/GFSbrick9/GFSdata
Brick54: XXXXXXXXXXXXXXXXXXX06-32-84:/GFSbrick9/GFSdata
Brick55: XXXXXXXXXXXXXXXXXXX01-32-146:/GFSbrick10/GFSdata
Brick56: XXXXXXXXXXXXXXXXXXX02-32-37:/GFSbrick10/GFSdata
Brick57: XXXXXXXXXXXXXXXXXXX03-32-130:/GFSbrick10/GFSdata
Brick58: XXXXXXXXXXXXXXXXXXX04-32-48:/GFSbrick10/GFSdata
Brick59: XXXXXXXXXXXXXXXXXXX05-32-132:/GFSbrick10/GFSdata
Brick60: XXXXXXXXXXXXXXXXXXX06-32-84:/GFSbrick10/GFSdata
Brick61: XXXXXXXXXXXXXXXXXXX01-32-146:/GFSbrick11/GFSdata
Brick62: XXXXXXXXXXXXXXXXXXX02-32-37:/GFSbrick11/GFSdata
Brick63: XXXXXXXXXXXXXXXXXXX03-32-130:/GFSbrick11/GFSdata
Brick64: XXXXXXXXXXXXXXXXXXX04-32-48:/GFSbrick11/GFSdata
Brick65: XXXXXXXXXXXXXXXXXXX05-32-132:/GFSbrick11/GFSdata
Brick66: XXXXXXXXXXXXXXXXXXX06-32-84:/GFSbrick11/GFSdata
Brick67: XXXXXXXXXXXXXXXXXXX01-32-146:/GFSbrick12/GFSdata
Brick68: XXXXXXXXXXXXXXXXXXX02-32-37:/GFSbrick12/GFSdata
Brick69: XXXXXXXXXXXXXXXXXXX03-32-130:/GFSbrick12/GFSdata
Brick70: XXXXXXXXXXXXXXXXXXX04-32-48:/GFSbrick12/GFSdata
Brick71: XXXXXXXXXXXXXXXXXXX05-32-132:/GFSbrick12/GFSdata
Brick72: XXXXXXXXXXXXXXXXXXX06-32-84:/GFSbrick12/GFSdata
Options Reconfigured:
server.event-threads: 12
performance.io-thread-count: 32
server.allow-insecure: on
cluster.self-heal-window-size: 128
performance.client-io-threads: off
nfs.disable: on
transport.address-family: inet
features.quota: off
features.inode-quota: off
performance.read-ahead: on
performance.cache-size: 64GB
network.ping-timeout: 30
performance.write-behind-window-size: 1GB

（四）、mount info
XXXXXXXXXXXXXXXX03-32-130:/GFSvolume on /GFSvolume type fuse.glusterfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)