Created attachment 1393376 [details]
Contains statedump files for before and after the test for each version, results of the test for each version, and the test script

Description of problem:
On a replicate 3 volume spread across 3 servers, the client with version 3.13.2 uses significantly more memory than 3.12.5 (which is already over 1G)

Version-Release number of selected component (if applicable):
3.13.2

How reproducible:
We have seen this problem 4/4 test runs

Steps to Reproduce:
1. Create a volume that is of type replicate 3 across 3 different servers
2. Unzip the attached zip file and extract mem_test.sh onto the on of the servers in the cluster
3. Update the script to replace the variables with your own paths for mount, brick, etc (replace "/path/to..." with your locations)
4. run the script with the following command "./mem-test.sh -r 10 -t 10 -m read -s" This will create create 1 million files with 10 threads writing 100,000 files each and then the script will read each file from the mount.
5. Repeat this on a site with gluster version 3.12 and 3.13

Actual results:
The gluster_processes.csv file shows that the client for 3.13 is using ~5G of memory while in 3.12 it was using ~1G. These values can be seen in the client_mem_res column. NOTE: if the gluster_process.csv values have no unit then they are in KB

Expected results:
The memory usage should not increase that much between versions. If anything it should decrease.

Additional info:
In addition to the script, the zip file also contains
- gluster_processes_<version-number>.csv files show the results of our test runs for each version
- before_test_<version-number>.glusterdump files show the statedump for the glusterfs process BEFORE the tests start for each version
- after_test_<version-number>.glusterdump files show the statedump for the glusterfs process AFTER the tests start for each version