Description of problem: glusterfs process consuming more and more memory even on clusters with unused volumes, where "gluster volume status" is the only activity Version-Release number of selected component (if applicable): glusterfs-7.2-1.el7.x86_64 but the problem is known for me for a longer time (back to version 3) How reproducible: just use "gluster volume status" in a loop Steps to Reproduce: 1. check glusterfs process memory usage 2. while (true); do gluster volume status ; sleep 1; done 3. wait couple of hours 4. check glusterfs process memory usage Actual results: memory usage grows up to the oom Expected results: Additional info:
Moving it to the right component as the leak is not with glusterd process.
one more thing... it looks like it leaks more (only?) if using ssl (client.ssl/server.ssl "on"). i've always used clusters with ssl but i've spotted some cluster without ssl and with big uptime and without "memleak like" behaviour. i've run loop again on cssluster after making ssl off and after 1 hour the difference in memory usage is negligible (still, going to wait more hours) my custom settings (3 nodes): gluster volume set volume0 cluster.server-quorum-type server gluster volume set volume0 diagnostics.count-fop-hits yes gluster volume set volume0 diagnostics.latency-measurement yes gluster volume set volume0 client.ssl on gluster volume set volume0 server.ssl on gluster volume set volume0 ssl.cipher-list 'HIGH:!SSLv2:!SSLv3:!TLSv1:!TLSv1.1:TLSv1.2:!3DES:!RC4:!aNULL:!ADH'
I remember Mohit fixing a memleak issue in a SSL enabled environment. Not sure, whether we have the fix in release 7.1 Mohit, please confirm. Thanks, Sanju
REVIEW: https://review.gluster.org/24073 (rpc: Cleanup SSL specific data at the time of freeing rpc object) posted (#1) for review on release-7 by MOHIT AGRAWAL
The patch is not backported in release-7 so posted a patch to resolve the same.
REVIEW: https://review.gluster.org/24073 (rpc: Cleanup SSL specific data at the time of freeing rpc object) merged (#3) on release-7 by Rinku Kothiya
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days