Bug 1399584 - Memory leaks after glfs_new+set_volfile_server+init+fini
Summary: Memory leaks after glfs_new+set_volfile_server+init+fini
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: libgfapi
Version: mainline
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Mateusz Slupny
QA Contact: Sudhir D
URL:
Whiteboard:
Depends On:
Blocks: 1397177
TreeView+ depends on / blocked
 
Reported: 2016-11-29 10:43 UTC by Mateusz Slupny
Modified: 2018-08-29 03:35 UTC (History)
2 users (show)

Fixed In Version: glusterfs-4.1.3 (or later)
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-08-29 03:35:33 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)

Description Mateusz Slupny 2016-11-29 10:43:36 UTC
Description of problem:

Extends the test from https://bugzilla.redhat.com/show_bug.cgi?id=1397177.

After running a simple application valgrind reports several memory leaks. Loop is added to highlight actual repeating leaks and not occurrences one-time initialization.

Application code:

#include <glusterfs/api/glfs.h>

void glusterCatastrophe()
{
  glfs_t* gluster = glfs_new("hot");
  glfs_set_volfile_server(gluster, "tcp", "10.10.27.20", 24007);
  glfs_init(gluster);
  glfs_fini(gluster);
}

int main()
{
  for (int i = 0; i < 10; ++i)
  {
    glusterCatastrophe();
  }

  return 0;
}

valgrind output:

==31753== Memcheck, a memory error detector
==31753== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==31753== Using Valgrind-3.12.0.SVN and LibVEX; rerun with -h for copyright info
==31753== Command: ./simple
==31753== 
==31753== 
==31753== HEAP SUMMARY:
==31753==     in use at exit: 137,819,294 bytes in 11,711 blocks
==31753==   total heap usage: 47,402 allocs, 35,691 frees, 164,418,897 bytes allocated
==31753== 
==31753== 780 bytes in 10 blocks are definitely lost in loss record 253 of 459
==31753==    at 0x4C2CB3F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBBCF: __gf_malloc (mem-pool.c:142)
==31753==    by 0x68BBEBB: gf_vasprintf (mem-pool.c:221)
==31753==    by 0x68BBFD0: gf_asprintf (mem-pool.c:240)
==31753==    by 0x687F3F8: data_from_int32 (dict.c:721)
==31753==    by 0x6881A21: dict_set_int32 (dict.c:1789)
==31753==    by 0x6D9E594: rpc_transport_inet_options_build (rpc-transport.c:681)
==31753==    by 0x6278FBD: glfs_mgmt_init (glfs-mgmt.c:917)
==31753==    by 0x6274863: glfs_volumes_init (glfs.c:266)
==31753==    by 0x6275D6B: glfs_init_common (glfs.c:989)
==31753==    by 0x6275F21: glfs_init@@GFAPI_3.4.0 (glfs.c:1034)
==31753==    by 0x1088D2: glusterCatastrophe() (in /home/matt/dev/glusterfs_memory_leak/simple)
==31753== 
==31753== 820 bytes in 10 blocks are definitely lost in loss record 259 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x6887817: gf_strdup (mem-pool.h:187)
==31753==    by 0x6888B6B: gf_log_init (logging.c:734)
==31753==    by 0x6275B4D: glfs_set_logging@@GFAPI_3.4.0 (glfs.c:903)
==31753==    by 0x62757A7: glfs_new@@GFAPI_3.4.0 (glfs.c:778)
==31753==    by 0x1088A3: glusterCatastrophe() (in /home/matt/dev/glusterfs_memory_leak/simple)
==31753==    by 0x1088FB: main (in /home/matt/dev/glusterfs_memory_leak/simple)
==31753== 
==31753== 840 bytes in 10 blocks are definitely lost in loss record 262 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x6D9C89D: gf_strdup (mem-pool.h:187)
==31753==    by 0x6D9E4F3: rpc_transport_inet_options_build (rpc-transport.c:667)
==31753==    by 0x6278FBD: glfs_mgmt_init (glfs-mgmt.c:917)
==31753==    by 0x6274863: glfs_volumes_init (glfs.c:266)
==31753==    by 0x6275D6B: glfs_init_common (glfs.c:989)
==31753==    by 0x6275F21: glfs_init@@GFAPI_3.4.0 (glfs.c:1034)
==31753==    by 0x1088D2: glusterCatastrophe() (in /home/matt/dev/glusterfs_memory_leak/simple)
==31753==    by 0x1088FB: main (in /home/matt/dev/glusterfs_memory_leak/simple)
==31753== 
==31753== 840 bytes in 10 blocks are definitely lost in loss record 263 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x687E5ED: dict_set_lk (dict.c:376)
==31753==    by 0x687E7C8: dict_set (dict.c:422)
==31753==    by 0x6882538: dict_set_dynstr (dict.c:2223)
==31753==    by 0x6D9E521: rpc_transport_inet_options_build (rpc-transport.c:673)
==31753==    by 0x6278FBD: glfs_mgmt_init (glfs-mgmt.c:917)
==31753==    by 0x6274863: glfs_volumes_init (glfs.c:266)
==31753==    by 0x6275D6B: glfs_init_common (glfs.c:989)
==31753==    by 0x6275F21: glfs_init@@GFAPI_3.4.0 (glfs.c:1034)
==31753==    by 0x1088D2: glusterCatastrophe() (in /home/matt/dev/glusterfs_memory_leak/simple)
==31753==    by 0x1088FB: main (in /home/matt/dev/glusterfs_memory_leak/simple)
==31753== 
==31753== 840 bytes in 10 blocks are definitely lost in loss record 264 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x687E5ED: dict_set_lk (dict.c:376)
==31753==    by 0x687E7C8: dict_set (dict.c:422)
==31753==    by 0x6881A4C: dict_set_int32 (dict.c:1795)
==31753==    by 0x6D9E594: rpc_transport_inet_options_build (rpc-transport.c:681)
==31753==    by 0x6278FBD: glfs_mgmt_init (glfs-mgmt.c:917)
==31753==    by 0x6274863: glfs_volumes_init (glfs.c:266)
==31753==    by 0x6275D6B: glfs_init_common (glfs.c:989)
==31753==    by 0x6275F21: glfs_init@@GFAPI_3.4.0 (glfs.c:1034)
==31753==    by 0x1088D2: glusterCatastrophe() (in /home/matt/dev/glusterfs_memory_leak/simple)
==31753==    by 0x1088FB: main (in /home/matt/dev/glusterfs_memory_leak/simple)
==31753== 
==31753== 840 bytes in 10 blocks are definitely lost in loss record 265 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x54ED71E: dht_init_subvolumes (dht-helper.c:1042)
==31753==    by 0x557573A: dht_init (dht-shared.c:757)
==31753==    by 0x68858C0: __xlator_init (xlator.c:403)
==31753==    by 0x68859E1: xlator_init (xlator.c:428)
==31753==    by 0x68CE7BA: glusterfs_graph_init (graph.c:320)
==31753==    by 0x68CF39F: glusterfs_graph_activate (graph.c:670)
==31753==    by 0x6276EBF: glfs_process_volfp (glfs-mgmt.c:78)
==31753==    by 0x62783E3: glfs_mgmt_getspec_cbk (glfs-mgmt.c:604)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753== 
==31753== 870 bytes in 10 blocks are definitely lost in loss record 268 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x687E5ED: dict_set_lk (dict.c:376)
==31753==    by 0x687E7C8: dict_set (dict.c:422)
==31753==    by 0x6882452: dict_set_str (dict.c:2186)
==31753==    by 0x6D9E5FD: rpc_transport_inet_options_build (rpc-transport.c:688)
==31753==    by 0x6278FBD: glfs_mgmt_init (glfs-mgmt.c:917)
==31753==    by 0x6274863: glfs_volumes_init (glfs.c:266)
==31753==    by 0x6275D6B: glfs_init_common (glfs.c:989)
==31753==    by 0x6275F21: glfs_init@@GFAPI_3.4.0 (glfs.c:1034)
==31753==    by 0x1088D2: glusterCatastrophe() (in /home/matt/dev/glusterfs_memory_leak/simple)
==31753==    by 0x1088FB: main (in /home/matt/dev/glusterfs_memory_leak/simple)
==31753== 
==31753== 880 bytes in 10 blocks are definitely lost in loss record 276 of 459
==31753==    at 0x4C2CB3F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBBCF: __gf_malloc (mem-pool.c:142)
==31753==    by 0x688F1BC: gf_dnscache_init (common-utils.c:379)
==31753==    by 0x4E59760: init (io-stats.c:3786)
==31753==    by 0x68858C0: __xlator_init (xlator.c:403)
==31753==    by 0x68859E1: xlator_init (xlator.c:428)
==31753==    by 0x68CE7BA: glusterfs_graph_init (graph.c:320)
==31753==    by 0x68CF39F: glusterfs_graph_activate (graph.c:670)
==31753==    by 0x6276EBF: glfs_process_volfp (glfs-mgmt.c:78)
==31753==    by 0x62783E3: glfs_mgmt_getspec_cbk (glfs-mgmt.c:604)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753== 
==31753== 880 bytes in 10 blocks are definitely lost in loss record 277 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x5C5231B: init (quick-read.c:1034)
==31753==    by 0x68858C0: __xlator_init (xlator.c:403)
==31753==    by 0x68859E1: xlator_init (xlator.c:428)
==31753==    by 0x68CE7BA: glusterfs_graph_init (graph.c:320)
==31753==    by 0x68CF39F: glusterfs_graph_activate (graph.c:670)
==31753==    by 0x6276EBF: glfs_process_volfp (glfs-mgmt.c:78)
==31753==    by 0x62783E3: glfs_mgmt_getspec_cbk (glfs-mgmt.c:604)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753==    by 0x6D9E0D9: rpc_transport_notify (rpc-transport.c:537)
==31753== 
==31753== 950 bytes in 10 blocks are definitely lost in loss record 282 of 459
==31753==    at 0x4C2CB3F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBBCF: __gf_malloc (mem-pool.c:142)
==31753==    by 0x68BBEBB: gf_vasprintf (mem-pool.c:221)
==31753==    by 0x68BBFD0: gf_asprintf (mem-pool.c:240)
==31753==    by 0x5575C76: dht_init (dht-shared.c:822)
==31753==    by 0x68858C0: __xlator_init (xlator.c:403)
==31753==    by 0x68859E1: xlator_init (xlator.c:428)
==31753==    by 0x68CE7BA: glusterfs_graph_init (graph.c:320)
==31753==    by 0x68CF39F: glusterfs_graph_activate (graph.c:670)
==31753==    by 0x6276EBF: glfs_process_volfp (glfs-mgmt.c:78)
==31753==    by 0x62783E3: glfs_mgmt_getspec_cbk (glfs-mgmt.c:604)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753== 
==31753== 960 bytes in 10 blocks are definitely lost in loss record 286 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x54ED75E: dht_init_subvolumes (dht-helper.c:1048)
==31753==    by 0x557573A: dht_init (dht-shared.c:757)
==31753==    by 0x68858C0: __xlator_init (xlator.c:403)
==31753==    by 0x68859E1: xlator_init (xlator.c:428)
==31753==    by 0x68CE7BA: glusterfs_graph_init (graph.c:320)
==31753==    by 0x68CF39F: glusterfs_graph_activate (graph.c:670)
==31753==    by 0x6276EBF: glfs_process_volfp (glfs-mgmt.c:78)
==31753==    by 0x62783E3: glfs_mgmt_getspec_cbk (glfs-mgmt.c:604)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753== 
==31753== 960 bytes in 10 blocks are definitely lost in loss record 287 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x54ED7E5: dht_init_subvolumes (dht-helper.c:1060)
==31753==    by 0x557573A: dht_init (dht-shared.c:757)
==31753==    by 0x68858C0: __xlator_init (xlator.c:403)
==31753==    by 0x68859E1: xlator_init (xlator.c:428)
==31753==    by 0x68CE7BA: glusterfs_graph_init (graph.c:320)
==31753==    by 0x68CF39F: glusterfs_graph_activate (graph.c:670)
==31753==    by 0x6276EBF: glfs_process_volfp (glfs-mgmt.c:78)
==31753==    by 0x62783E3: glfs_mgmt_getspec_cbk (glfs-mgmt.c:604)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753== 
==31753== 1,010 bytes in 10 blocks are definitely lost in loss record 291 of 459
==31753==    at 0x4C2CB3F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBBCF: __gf_malloc (mem-pool.c:142)
==31753==    by 0x68BBEBB: gf_vasprintf (mem-pool.c:221)
==31753==    by 0x68BBFD0: gf_asprintf (mem-pool.c:240)
==31753==    by 0x5575C1C: dht_init (dht-shared.c:818)
==31753==    by 0x68858C0: __xlator_init (xlator.c:403)
==31753==    by 0x68859E1: xlator_init (xlator.c:428)
==31753==    by 0x68CE7BA: glusterfs_graph_init (graph.c:320)
==31753==    by 0x68CF39F: glusterfs_graph_activate (graph.c:670)
==31753==    by 0x6276EBF: glfs_process_volfp (glfs-mgmt.c:78)
==31753==    by 0x62783E3: glfs_mgmt_getspec_cbk (glfs-mgmt.c:604)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753== 
==31753== 1,050 bytes in 10 blocks are definitely lost in loss record 293 of 459
==31753==    at 0x4C2CB3F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBBCF: __gf_malloc (mem-pool.c:142)
==31753==    by 0x68BBEBB: gf_vasprintf (mem-pool.c:221)
==31753==    by 0x68BBFD0: gf_asprintf (mem-pool.c:240)
==31753==    by 0x5575C49: dht_init (dht-shared.c:820)
==31753==    by 0x68858C0: __xlator_init (xlator.c:403)
==31753==    by 0x68859E1: xlator_init (xlator.c:428)
==31753==    by 0x68CE7BA: glusterfs_graph_init (graph.c:320)
==31753==    by 0x68CF39F: glusterfs_graph_activate (graph.c:670)
==31753==    by 0x6276EBF: glfs_process_volfp (glfs-mgmt.c:78)
==31753==    by 0x62783E3: glfs_mgmt_getspec_cbk (glfs-mgmt.c:604)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753== 
==31753== 1,300 bytes in 10 blocks are definitely lost in loss record 304 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x68836EB: dict_allocate_and_serialize (dict.c:2797)
==31753==    by 0x627873F: glfs_volfile_fetch (glfs-mgmt.c:690)
==31753==    by 0x6278D6B: mgmt_rpc_notify (glfs-mgmt.c:846)
==31753==    by 0x6DA2410: rpc_clnt_notify (rpc-clnt.c:993)
==31753==    by 0x6D9E0D9: rpc_transport_notify (rpc-transport.c:537)
==31753==    by 0xD9B7677: socket_connect_finish (socket.c:2341)
==31753==    by 0xD9B7873: socket_event_handler (socket.c:2384)
==31753==    by 0x68F92AC: event_dispatch_epoll_handler (event-epoll.c:571)
==31753==    by 0x68F969A: event_dispatch_epoll_worker (event-epoll.c:674)
==31753==    by 0x6B78709: start_thread (pthread_create.c:333)
==31753== 
==31753== 1,400 bytes in 10 blocks are definitely lost in loss record 306 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x68BC9BB: mem_get (mem-pool.c:513)
==31753==    by 0x68BC7DE: mem_get0 (mem-pool.c:446)
==31753==    by 0x687E568: dict_set_lk (dict.c:358)
==31753==    by 0x687E7C8: dict_set (dict.c:422)
==31753==    by 0x6881A4C: dict_set_int32 (dict.c:1795)
==31753==    by 0x6D9E594: rpc_transport_inet_options_build (rpc-transport.c:681)
==31753==    by 0x6278FBD: glfs_mgmt_init (glfs-mgmt.c:917)
==31753==    by 0x6274863: glfs_volumes_init (glfs.c:266)
==31753==    by 0x6275D6B: glfs_init_common (glfs.c:989)
==31753==    by 0x6275F21: glfs_init@@GFAPI_3.4.0 (glfs.c:1034)
==31753== 
==31753== 1,400 bytes in 10 blocks are definitely lost in loss record 307 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x68BC9BB: mem_get (mem-pool.c:513)
==31753==    by 0x68BC7DE: mem_get0 (mem-pool.c:446)
==31753==    by 0x687E568: dict_set_lk (dict.c:358)
==31753==    by 0x687E7C8: dict_set (dict.c:422)
==31753==    by 0x6882452: dict_set_str (dict.c:2186)
==31753==    by 0x6D9E5FD: rpc_transport_inet_options_build (rpc-transport.c:688)
==31753==    by 0x6278FBD: glfs_mgmt_init (glfs-mgmt.c:917)
==31753==    by 0x6274863: glfs_volumes_init (glfs.c:266)
==31753==    by 0x6275D6B: glfs_init_common (glfs.c:989)
==31753==    by 0x6275F21: glfs_init@@GFAPI_3.4.0 (glfs.c:1034)
==31753== 
==31753== 1,640 bytes in 10 blocks are definitely lost in loss record 313 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x68BC9BB: mem_get (mem-pool.c:513)
==31753==    by 0x68BC7DE: mem_get0 (mem-pool.c:446)
==31753==    by 0x687DABD: get_new_data (dict.c:40)
==31753==    by 0x687FA26: data_from_dynstr (dict.c:900)
==31753==    by 0x688250D: dict_set_dynstr (dict.c:2217)
==31753==    by 0x6D9E521: rpc_transport_inet_options_build (rpc-transport.c:673)
==31753==    by 0x6278FBD: glfs_mgmt_init (glfs-mgmt.c:917)
==31753==    by 0x6274863: glfs_volumes_init (glfs.c:266)
==31753==    by 0x6275D6B: glfs_init_common (glfs.c:989)
==31753==    by 0x6275F21: glfs_init@@GFAPI_3.4.0 (glfs.c:1034)
==31753== 
==31753== 1,640 bytes in 10 blocks are definitely lost in loss record 314 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x68BC9BB: mem_get (mem-pool.c:513)
==31753==    by 0x68BC7DE: mem_get0 (mem-pool.c:446)
==31753==    by 0x687DABD: get_new_data (dict.c:40)
==31753==    by 0x687F3C2: data_from_int32 (dict.c:716)
==31753==    by 0x6881A21: dict_set_int32 (dict.c:1789)
==31753==    by 0x6D9E594: rpc_transport_inet_options_build (rpc-transport.c:681)
==31753==    by 0x6278FBD: glfs_mgmt_init (glfs-mgmt.c:917)
==31753==    by 0x6274863: glfs_volumes_init (glfs.c:266)
==31753==    by 0x6275D6B: glfs_init_common (glfs.c:989)
==31753==    by 0x6275F21: glfs_init@@GFAPI_3.4.0 (glfs.c:1034)
==31753== 
==31753== 1,640 bytes in 10 blocks are definitely lost in loss record 315 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x68BC9BB: mem_get (mem-pool.c:513)
==31753==    by 0x68BC7DE: mem_get0 (mem-pool.c:446)
==31753==    by 0x687DABD: get_new_data (dict.c:40)
==31753==    by 0x687F973: str_to_data (dict.c:878)
==31753==    by 0x6882427: dict_set_str (dict.c:2180)
==31753==    by 0x6D9E5FD: rpc_transport_inet_options_build (rpc-transport.c:688)
==31753==    by 0x6278FBD: glfs_mgmt_init (glfs-mgmt.c:917)
==31753==    by 0x6274863: glfs_volumes_init (glfs.c:266)
==31753==    by 0x6275D6B: glfs_init_common (glfs.c:989)
==31753==    by 0x6275F21: glfs_init@@GFAPI_3.4.0 (glfs.c:1034)
==31753== 
==31753== 1,680 bytes in 10 blocks are definitely lost in loss record 317 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x54ED7A8: dht_init_subvolumes (dht-helper.c:1054)
==31753==    by 0x557573A: dht_init (dht-shared.c:757)
==31753==    by 0x68858C0: __xlator_init (xlator.c:403)
==31753==    by 0x68859E1: xlator_init (xlator.c:428)
==31753==    by 0x68CE7BA: glusterfs_graph_init (graph.c:320)
==31753==    by 0x68CF39F: glusterfs_graph_activate (graph.c:670)
==31753==    by 0x6276EBF: glfs_process_volfp (glfs-mgmt.c:78)
==31753==    by 0x62783E3: glfs_mgmt_getspec_cbk (glfs-mgmt.c:604)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753== 
==31753== 2,080 bytes in 20 blocks are definitely lost in loss record 330 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x4E583A6: ios_init_top_stats (io-stats.c:3438)
==31753==    by 0x4E594F0: init (io-stats.c:3758)
==31753==    by 0x68858C0: __xlator_init (xlator.c:403)
==31753==    by 0x68859E1: xlator_init (xlator.c:428)
==31753==    by 0x68CE7BA: glusterfs_graph_init (graph.c:320)
==31753==    by 0x68CF39F: glusterfs_graph_activate (graph.c:670)
==31753==    by 0x6276EBF: glfs_process_volfp (glfs-mgmt.c:78)
==31753==    by 0x62783E3: glfs_mgmt_getspec_cbk (glfs-mgmt.c:604)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753== 
==31753== 2,250 bytes in 30 blocks are definitely lost in loss record 333 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x58188EB: init (afr.c:515)
==31753==    by 0x68858C0: __xlator_init (xlator.c:403)
==31753==    by 0x68859E1: xlator_init (xlator.c:428)
==31753==    by 0x68CE7BA: glusterfs_graph_init (graph.c:320)
==31753==    by 0x68CF39F: glusterfs_graph_activate (graph.c:670)
==31753==    by 0x6276EBF: glfs_process_volfp (glfs-mgmt.c:78)
==31753==    by 0x62783E3: glfs_mgmt_getspec_cbk (glfs-mgmt.c:604)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753==    by 0x6D9E0D9: rpc_transport_notify (rpc-transport.c:537)
==31753== 
==31753== 2,560 bytes in 10 blocks are definitely lost in loss record 340 of 459
==31753==    at 0x4C2CB3F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x657DA8A: regcomp (regcomp.c:479)
==31753==    by 0x5574101: dht_init_regex (dht-shared.c:359)
==31753==    by 0x5575848: dht_init (dht-shared.c:778)
==31753==    by 0x68858C0: __xlator_init (xlator.c:403)
==31753==    by 0x68859E1: xlator_init (xlator.c:428)
==31753==    by 0x68CE7BA: glusterfs_graph_init (graph.c:320)
==31753==    by 0x68CF39F: glusterfs_graph_activate (graph.c:670)
==31753==    by 0x6276EBF: glfs_process_volfp (glfs-mgmt.c:78)
==31753==    by 0x62783E3: glfs_mgmt_getspec_cbk (glfs-mgmt.c:604)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753== 
==31753== 2,940 bytes in 30 blocks are definitely lost in loss record 349 of 459
==31753==    at 0x4C2CB3F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBBCF: __gf_malloc (mem-pool.c:142)
==31753==    by 0x68BBEBB: gf_vasprintf (mem-pool.c:221)
==31753==    by 0x68BBFD0: gf_asprintf (mem-pool.c:240)
==31753==    by 0x5818A8C: init (afr.c:555)
==31753==    by 0x68858C0: __xlator_init (xlator.c:403)
==31753==    by 0x68859E1: xlator_init (xlator.c:428)
==31753==    by 0x68CE7BA: glusterfs_graph_init (graph.c:320)
==31753==    by 0x68CF39F: glusterfs_graph_activate (graph.c:670)
==31753==    by 0x6276EBF: glfs_process_volfp (glfs-mgmt.c:78)
==31753==    by 0x62783E3: glfs_mgmt_getspec_cbk (glfs-mgmt.c:604)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753== 
==31753== 6,660 bytes in 90 blocks are definitely lost in loss record 368 of 459
==31753==    at 0x4C2CB3F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBBCF: __gf_malloc (mem-pool.c:142)
==31753==    by 0x68BBEBB: gf_vasprintf (mem-pool.c:221)
==31753==    by 0x68BBFD0: gf_asprintf (mem-pool.c:240)
==31753==    by 0x687F7B5: data_from_uint32 (dict.c:823)
==31753==    by 0x6881DA0: dict_set_uint32 (dict.c:1925)
==31753==    by 0x52C53E8: client_setvolume (client-handshake.c:1373)
==31753==    by 0x52C6139: client_dump_version_cbk (client-handshake.c:1662)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753==    by 0x6D9E0D9: rpc_transport_notify (rpc-transport.c:537)
==31753==    by 0xD9B734D: socket_event_poll_in (socket.c:2265)
==31753== 
==31753== 6,660 bytes in 90 blocks are definitely lost in loss record 369 of 459
==31753==    at 0x4C2CB3F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBBCF: __gf_malloc (mem-pool.c:142)
==31753==    by 0x68BBEBB: gf_vasprintf (mem-pool.c:221)
==31753==    by 0x68BBFD0: gf_asprintf (mem-pool.c:240)
==31753==    by 0x687F4C4: data_from_int16 (dict.c:741)
==31753==    by 0x68818F9: dict_set_int16 (dict.c:1744)
==31753==    by 0x52C545D: client_setvolume (client-handshake.c:1381)
==31753==    by 0x52C6139: client_dump_version_cbk (client-handshake.c:1662)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753==    by 0x6D9E0D9: rpc_transport_notify (rpc-transport.c:537)
==31753==    by 0xD9B734D: socket_event_poll_in (socket.c:2265)
==31753== 
==31753== 7,200 bytes in 90 blocks are definitely lost in loss record 371 of 459
==31753==    at 0x4C2CB3F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBBCF: __gf_malloc (mem-pool.c:142)
==31753==    by 0x68BBEBB: gf_vasprintf (mem-pool.c:221)
==31753==    by 0x68BBFD0: gf_asprintf (mem-pool.c:240)
==31753==    by 0x687F3F8: data_from_int32 (dict.c:721)
==31753==    by 0x6881A21: dict_set_int32 (dict.c:1789)
==31753==    by 0x52C505B: client_setvolume (client-handshake.c:1302)
==31753==    by 0x52C6139: client_dump_version_cbk (client-handshake.c:1662)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753==    by 0x6D9E0D9: rpc_transport_notify (rpc-transport.c:537)
==31753==    by 0xD9B734D: socket_event_poll_in (socket.c:2265)
==31753== 
==31753== 7,560 bytes in 90 blocks are definitely lost in loss record 372 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x687E5ED: dict_set_lk (dict.c:376)
==31753==    by 0x687E7C8: dict_set (dict.c:422)
==31753==    by 0x6882452: dict_set_str (dict.c:2186)
==31753==    by 0x52C5371: client_setvolume (client-handshake.c:1366)
==31753==    by 0x52C6139: client_dump_version_cbk (client-handshake.c:1662)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753==    by 0x6D9E0D9: rpc_transport_notify (rpc-transport.c:537)
==31753==    by 0xD9B734D: socket_event_poll_in (socket.c:2265)
==31753==    by 0xD9B78B2: socket_event_handler (socket.c:2395)
==31753== 
==31753== 7,650 bytes in 90 blocks are definitely lost in loss record 375 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x687E5ED: dict_set_lk (dict.c:376)
==31753==    by 0x687E7C8: dict_set (dict.c:422)
==31753==    by 0x6881A4C: dict_set_int32 (dict.c:1795)
==31753==    by 0x52C505B: client_setvolume (client-handshake.c:1302)
==31753==    by 0x52C6139: client_dump_version_cbk (client-handshake.c:1662)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753==    by 0x6D9E0D9: rpc_transport_notify (rpc-transport.c:537)
==31753==    by 0xD9B734D: socket_event_poll_in (socket.c:2265)
==31753==    by 0xD9B78B2: socket_event_handler (socket.c:2395)
==31753== 
==31753== 7,650 bytes in 90 blocks are definitely lost in loss record 376 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x687E5ED: dict_set_lk (dict.c:376)
==31753==    by 0x687E7C8: dict_set (dict.c:422)
==31753==    by 0x6882538: dict_set_dynstr (dict.c:2223)
==31753==    by 0x52C5254: client_setvolume (client-handshake.c:1349)
==31753==    by 0x52C6139: client_dump_version_cbk (client-handshake.c:1662)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753==    by 0x6D9E0D9: rpc_transport_notify (rpc-transport.c:537)
==31753==    by 0xD9B734D: socket_event_poll_in (socket.c:2265)
==31753==    by 0xD9B78B2: socket_event_handler (socket.c:2395)
==31753== 
==31753== 7,830 bytes in 90 blocks are definitely lost in loss record 378 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x687E5ED: dict_set_lk (dict.c:376)
==31753==    by 0x687E7C8: dict_set (dict.c:422)
==31753==    by 0x6882452: dict_set_str (dict.c:2186)
==31753==    by 0x52C52C7: client_setvolume (client-handshake.c:1357)
==31753==    by 0x52C6139: client_dump_version_cbk (client-handshake.c:1662)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753==    by 0x6D9E0D9: rpc_transport_notify (rpc-transport.c:537)
==31753==    by 0xD9B734D: socket_event_poll_in (socket.c:2265)
==31753==    by 0xD9B78B2: socket_event_handler (socket.c:2395)
==31753== 
==31753== 7,920 bytes in 90 blocks are definitely lost in loss record 380 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x687E5ED: dict_set_lk (dict.c:376)
==31753==    by 0x687E7C8: dict_set (dict.c:422)
==31753==    by 0x6881924: dict_set_int16 (dict.c:1750)
==31753==    by 0x52C545D: client_setvolume (client-handshake.c:1381)
==31753==    by 0x52C6139: client_dump_version_cbk (client-handshake.c:1662)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753==    by 0x6D9E0D9: rpc_transport_notify (rpc-transport.c:537)
==31753==    by 0xD9B734D: socket_event_poll_in (socket.c:2265)
==31753==    by 0xD9B78B2: socket_event_handler (socket.c:2395)
==31753== 
==31753== 8,010 bytes in 90 blocks are definitely lost in loss record 381 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x687E5ED: dict_set_lk (dict.c:376)
==31753==    by 0x687E7C8: dict_set (dict.c:422)
==31753==    by 0x6881DCB: dict_set_uint32 (dict.c:1931)
==31753==    by 0x52C53E8: client_setvolume (client-handshake.c:1373)
==31753==    by 0x52C6139: client_dump_version_cbk (client-handshake.c:1662)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753==    by 0x6D9E0D9: rpc_transport_notify (rpc-transport.c:537)
==31753==    by 0xD9B734D: socket_event_poll_in (socket.c:2265)
==31753==    by 0xD9B78B2: socket_event_handler (socket.c:2395)
==31753== 
==31753== 8,320 bytes in 80 blocks are definitely lost in loss record 384 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x4E5827A: ios_init_top_stats (io-stats.c:3426)
==31753==    by 0x4E594F0: init (io-stats.c:3758)
==31753==    by 0x68858C0: __xlator_init (xlator.c:403)
==31753==    by 0x68859E1: xlator_init (xlator.c:428)
==31753==    by 0x68CE7BA: glusterfs_graph_init (graph.c:320)
==31753==    by 0x68CF39F: glusterfs_graph_activate (graph.c:670)
==31753==    by 0x6276EBF: glfs_process_volfp (glfs-mgmt.c:78)
==31753==    by 0x62783E3: glfs_mgmt_getspec_cbk (glfs-mgmt.c:604)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753== 
==31753== 10,136 (320 direct, 9,816 indirect) bytes in 2 blocks are definitely lost in loss record 388 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x6DA36A1: rpcclnt_cbk_program_register (rpc-clnt.c:1506)
==31753==    by 0x627910E: glfs_mgmt_init (glfs-mgmt.c:944)
==31753==    by 0x6274863: glfs_volumes_init (glfs.c:266)
==31753==    by 0x6275D6B: glfs_init_common (glfs.c:989)
==31753==    by 0x6275F21: glfs_init@@GFAPI_3.4.0 (glfs.c:1034)
==31753==    by 0x1088D2: glusterCatastrophe() (in /home/matt/dev/glusterfs_memory_leak/simple)
==31753==    by 0x1088FB: main (in /home/matt/dev/glusterfs_memory_leak/simple)
==31753== 
==31753== 11,430 bytes in 90 blocks are definitely lost in loss record 390 of 459
==31753==    at 0x4C2CB3F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBBCF: __gf_malloc (mem-pool.c:142)
==31753==    by 0x68BBEBB: gf_vasprintf (mem-pool.c:221)
==31753==    by 0x68BBFD0: gf_asprintf (mem-pool.c:240)
==31753==    by 0x52C51E5: client_setvolume (client-handshake.c:1339)
==31753==    by 0x52C6139: client_dump_version_cbk (client-handshake.c:1662)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753==    by 0x6D9E0D9: rpc_transport_notify (rpc-transport.c:537)
==31753==    by 0xD9B734D: socket_event_poll_in (socket.c:2265)
==31753==    by 0xD9B78B2: socket_event_handler (socket.c:2395)
==31753==    by 0x68F92AC: event_dispatch_epoll_handler (event-epoll.c:571)
==31753== 
==31753== 12,600 bytes in 90 blocks are definitely lost in loss record 391 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x68BC9BB: mem_get (mem-pool.c:513)
==31753==    by 0x68BC7DE: mem_get0 (mem-pool.c:446)
==31753==    by 0x687E568: dict_set_lk (dict.c:358)
==31753==    by 0x687E7C8: dict_set (dict.c:422)
==31753==    by 0x6881A4C: dict_set_int32 (dict.c:1795)
==31753==    by 0x52C505B: client_setvolume (client-handshake.c:1302)
==31753==    by 0x52C6139: client_dump_version_cbk (client-handshake.c:1662)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753==    by 0x6D9E0D9: rpc_transport_notify (rpc-transport.c:537)
==31753== 
==31753== 12,600 bytes in 90 blocks are definitely lost in loss record 392 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x68BC9BB: mem_get (mem-pool.c:513)
==31753==    by 0x68BC7DE: mem_get0 (mem-pool.c:446)
==31753==    by 0x687E568: dict_set_lk (dict.c:358)
==31753==    by 0x687E7C8: dict_set (dict.c:422)
==31753==    by 0x6882538: dict_set_dynstr (dict.c:2223)
==31753==    by 0x52C5254: client_setvolume (client-handshake.c:1349)
==31753==    by 0x52C6139: client_dump_version_cbk (client-handshake.c:1662)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753==    by 0x6D9E0D9: rpc_transport_notify (rpc-transport.c:537)
==31753== 
==31753== 12,600 bytes in 90 blocks are definitely lost in loss record 393 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x68BC9BB: mem_get (mem-pool.c:513)
==31753==    by 0x68BC7DE: mem_get0 (mem-pool.c:446)
==31753==    by 0x687E568: dict_set_lk (dict.c:358)
==31753==    by 0x687E7C8: dict_set (dict.c:422)
==31753==    by 0x6882452: dict_set_str (dict.c:2186)
==31753==    by 0x52C52C7: client_setvolume (client-handshake.c:1357)
==31753==    by 0x52C6139: client_dump_version_cbk (client-handshake.c:1662)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753==    by 0x6D9E0D9: rpc_transport_notify (rpc-transport.c:537)
==31753== 
==31753== 12,600 bytes in 90 blocks are definitely lost in loss record 394 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x68BC9BB: mem_get (mem-pool.c:513)
==31753==    by 0x68BC7DE: mem_get0 (mem-pool.c:446)
==31753==    by 0x687E568: dict_set_lk (dict.c:358)
==31753==    by 0x687E7C8: dict_set (dict.c:422)
==31753==    by 0x6882452: dict_set_str (dict.c:2186)
==31753==    by 0x52C5371: client_setvolume (client-handshake.c:1366)
==31753==    by 0x52C6139: client_dump_version_cbk (client-handshake.c:1662)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753==    by 0x6D9E0D9: rpc_transport_notify (rpc-transport.c:537)
==31753== 
==31753== 12,600 bytes in 90 blocks are definitely lost in loss record 395 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x68BC9BB: mem_get (mem-pool.c:513)
==31753==    by 0x68BC7DE: mem_get0 (mem-pool.c:446)
==31753==    by 0x687E568: dict_set_lk (dict.c:358)
==31753==    by 0x687E7C8: dict_set (dict.c:422)
==31753==    by 0x6881DCB: dict_set_uint32 (dict.c:1931)
==31753==    by 0x52C53E8: client_setvolume (client-handshake.c:1373)
==31753==    by 0x52C6139: client_dump_version_cbk (client-handshake.c:1662)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753==    by 0x6D9E0D9: rpc_transport_notify (rpc-transport.c:537)
==31753== 
==31753== 12,600 bytes in 90 blocks are definitely lost in loss record 396 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x68BC9BB: mem_get (mem-pool.c:513)
==31753==    by 0x68BC7DE: mem_get0 (mem-pool.c:446)
==31753==    by 0x687E568: dict_set_lk (dict.c:358)
==31753==    by 0x687E7C8: dict_set (dict.c:422)
==31753==    by 0x6881924: dict_set_int16 (dict.c:1750)
==31753==    by 0x52C545D: client_setvolume (client-handshake.c:1381)
==31753==    by 0x52C6139: client_dump_version_cbk (client-handshake.c:1662)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753==    by 0x6D9E0D9: rpc_transport_notify (rpc-transport.c:537)
==31753== 
==31753== 14,760 bytes in 90 blocks are definitely lost in loss record 404 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x68BC9BB: mem_get (mem-pool.c:513)
==31753==    by 0x68BC7DE: mem_get0 (mem-pool.c:446)
==31753==    by 0x687DABD: get_new_data (dict.c:40)
==31753==    by 0x687F973: str_to_data (dict.c:878)
==31753==    by 0x6882427: dict_set_str (dict.c:2180)
==31753==    by 0x6D9D1F3: rpc_transport_load (rpc-transport.c:234)
==31753==    by 0x6DA2640: rpc_clnt_connection_init (rpc-clnt.c:1053)
==31753==    by 0x6DA294C: rpc_clnt_new (rpc-clnt.c:1142)
==31753==    by 0x5291029: client_init_rpc (client.c:2523)
==31753==    by 0x5291B50: init (client.c:2746)
==31753== 
==31753== 14,760 bytes in 90 blocks are definitely lost in loss record 405 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x68BC9BB: mem_get (mem-pool.c:513)
==31753==    by 0x68BC7DE: mem_get0 (mem-pool.c:446)
==31753==    by 0x687DABD: get_new_data (dict.c:40)
==31753==    by 0x687F3C2: data_from_int32 (dict.c:716)
==31753==    by 0x6881A21: dict_set_int32 (dict.c:1789)
==31753==    by 0x52C505B: client_setvolume (client-handshake.c:1302)
==31753==    by 0x52C6139: client_dump_version_cbk (client-handshake.c:1662)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753==    by 0x6D9E0D9: rpc_transport_notify (rpc-transport.c:537)
==31753== 
==31753== 14,760 bytes in 90 blocks are definitely lost in loss record 406 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x68BC9BB: mem_get (mem-pool.c:513)
==31753==    by 0x68BC7DE: mem_get0 (mem-pool.c:446)
==31753==    by 0x687DABD: get_new_data (dict.c:40)
==31753==    by 0x687FA26: data_from_dynstr (dict.c:900)
==31753==    by 0x688250D: dict_set_dynstr (dict.c:2217)
==31753==    by 0x52C5254: client_setvolume (client-handshake.c:1349)
==31753==    by 0x52C6139: client_dump_version_cbk (client-handshake.c:1662)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753==    by 0x6D9E0D9: rpc_transport_notify (rpc-transport.c:537)
==31753== 
==31753== 14,760 bytes in 90 blocks are definitely lost in loss record 407 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x68BC9BB: mem_get (mem-pool.c:513)
==31753==    by 0x68BC7DE: mem_get0 (mem-pool.c:446)
==31753==    by 0x687DABD: get_new_data (dict.c:40)
==31753==    by 0x687F973: str_to_data (dict.c:878)
==31753==    by 0x6882427: dict_set_str (dict.c:2180)
==31753==    by 0x52C52C7: client_setvolume (client-handshake.c:1357)
==31753==    by 0x52C6139: client_dump_version_cbk (client-handshake.c:1662)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753==    by 0x6D9E0D9: rpc_transport_notify (rpc-transport.c:537)
==31753== 
==31753== 14,760 bytes in 90 blocks are definitely lost in loss record 408 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x68BC9BB: mem_get (mem-pool.c:513)
==31753==    by 0x68BC7DE: mem_get0 (mem-pool.c:446)
==31753==    by 0x687DABD: get_new_data (dict.c:40)
==31753==    by 0x687F973: str_to_data (dict.c:878)
==31753==    by 0x6882427: dict_set_str (dict.c:2180)
==31753==    by 0x52C5371: client_setvolume (client-handshake.c:1366)
==31753==    by 0x52C6139: client_dump_version_cbk (client-handshake.c:1662)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753==    by 0x6D9E0D9: rpc_transport_notify (rpc-transport.c:537)
==31753== 
==31753== 14,760 bytes in 90 blocks are definitely lost in loss record 409 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x68BC9BB: mem_get (mem-pool.c:513)
==31753==    by 0x68BC7DE: mem_get0 (mem-pool.c:446)
==31753==    by 0x687DABD: get_new_data (dict.c:40)
==31753==    by 0x687F77F: data_from_uint32 (dict.c:818)
==31753==    by 0x6881DA0: dict_set_uint32 (dict.c:1925)
==31753==    by 0x52C53E8: client_setvolume (client-handshake.c:1373)
==31753==    by 0x52C6139: client_dump_version_cbk (client-handshake.c:1662)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753==    by 0x6D9E0D9: rpc_transport_notify (rpc-transport.c:537)
==31753== 
==31753== 14,760 bytes in 90 blocks are definitely lost in loss record 410 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x68BC9BB: mem_get (mem-pool.c:513)
==31753==    by 0x68BC7DE: mem_get0 (mem-pool.c:446)
==31753==    by 0x687DABD: get_new_data (dict.c:40)
==31753==    by 0x687F48D: data_from_int16 (dict.c:736)
==31753==    by 0x68818F9: dict_set_int16 (dict.c:1744)
==31753==    by 0x52C545D: client_setvolume (client-handshake.c:1381)
==31753==    by 0x52C6139: client_dump_version_cbk (client-handshake.c:1662)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753==    by 0x6D9E0D9: rpc_transport_notify (rpc-transport.c:537)
==31753== 
==31753== 16,560 bytes in 30 blocks are definitely lost in loss record 414 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x689AB7F: __inode_create (inode.c:614)
==31753==    by 0x689CA43: __inode_table_init_root (inode.c:1557)
==31753==    by 0x689CDE3: inode_table_new (inode.c:1638)
==31753==    by 0x57F6BAE: afr_selfheal_daemon_init (afr-self-heald.c:976)
==31753==    by 0x5818B07: init (afr.c:569)
==31753==    by 0x68858C0: __xlator_init (xlator.c:403)
==31753==    by 0x68859E1: xlator_init (xlator.c:428)
==31753==    by 0x68CE7BA: glusterfs_graph_init (graph.c:320)
==31753==    by 0x68CF39F: glusterfs_graph_activate (graph.c:670)
==31753==    by 0x6276EBF: glfs_process_volfp (glfs-mgmt.c:78)
==31753== 
==31753== 18,000 bytes in 30 blocks are definitely lost in loss record 415 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x57F6BF2: afr_selfheal_daemon_init (afr-self-heald.c:980)
==31753==    by 0x5818B07: init (afr.c:569)
==31753==    by 0x68858C0: __xlator_init (xlator.c:403)
==31753==    by 0x68859E1: xlator_init (xlator.c:428)
==31753==    by 0x68CE7BA: glusterfs_graph_init (graph.c:320)
==31753==    by 0x68CF39F: glusterfs_graph_activate (graph.c:670)
==31753==    by 0x6276EBF: glfs_process_volfp (glfs-mgmt.c:78)
==31753==    by 0x62783E3: glfs_mgmt_getspec_cbk (glfs-mgmt.c:604)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753== 
==31753== 18,000 bytes in 30 blocks are definitely lost in loss record 416 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x57F6CB9: afr_selfheal_daemon_init (afr-self-heald.c:993)
==31753==    by 0x5818B07: init (afr.c:569)
==31753==    by 0x68858C0: __xlator_init (xlator.c:403)
==31753==    by 0x68859E1: xlator_init (xlator.c:428)
==31753==    by 0x68CE7BA: glusterfs_graph_init (graph.c:320)
==31753==    by 0x68CF39F: glusterfs_graph_activate (graph.c:670)
==31753==    by 0x6276EBF: glfs_process_volfp (glfs-mgmt.c:78)
==31753==    by 0x62783E3: glfs_mgmt_getspec_cbk (glfs-mgmt.c:604)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753== 
==31753== 35,640 (2,240 direct, 33,400 indirect) bytes in 10 blocks are definitely lost in loss record 422 of 459
==31753==    at 0x4C2CA6F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x4C2EDEF: realloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x657C4FD: re_compile_internal (regcomp.c:747)
==31753==    by 0x657DAE9: regcomp (regcomp.c:498)
==31753==    by 0x5574101: dht_init_regex (dht-shared.c:359)
==31753==    by 0x5575848: dht_init (dht-shared.c:778)
==31753==    by 0x68858C0: __xlator_init (xlator.c:403)
==31753==    by 0x68859E1: xlator_init (xlator.c:428)
==31753==    by 0x68CE7BA: glusterfs_graph_init (graph.c:320)
==31753==    by 0x68CF39F: glusterfs_graph_activate (graph.c:670)
==31753==    by 0x6276EBF: glfs_process_volfp (glfs-mgmt.c:78)
==31753==    by 0x62783E3: glfs_mgmt_getspec_cbk (glfs-mgmt.c:604)
==31753== 
==31753== 37,850 bytes in 490 blocks are definitely lost in loss record 424 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x691B241: gf_strdup (mem-pool.h:187)
==31753==    by 0x691C324: volume_option (graph.y:240)
==31753==    by 0x691B7F1: graphyyparse (graph.y:69)
==31753==    by 0x691D18C: glusterfs_graph_construct (graph.y:590)
==31753==    by 0x6276D9F: glfs_process_volfp (glfs-mgmt.c:54)
==31753==    by 0x62783E3: glfs_mgmt_getspec_cbk (glfs-mgmt.c:604)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753==    by 0x6D9E0D9: rpc_transport_notify (rpc-transport.c:537)
==31753==    by 0xD9B734D: socket_event_poll_in (socket.c:2265)
==31753== 
==31753== 37,890 bytes in 10 blocks are definitely lost in loss record 425 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x6277F22: glusterfs_oldvolfile_update (glfs-mgmt.c:490)
==31753==    by 0x627840B: glfs_mgmt_getspec_cbk (glfs-mgmt.c:610)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753==    by 0x6D9E0D9: rpc_transport_notify (rpc-transport.c:537)
==31753==    by 0xD9B734D: socket_event_poll_in (socket.c:2265)
==31753==    by 0xD9B78B2: socket_event_handler (socket.c:2395)
==31753==    by 0x68F92AC: event_dispatch_epoll_handler (event-epoll.c:571)
==31753==    by 0x68F969A: event_dispatch_epoll_worker (event-epoll.c:674)
==31753==    by 0x6B78709: start_thread (pthread_create.c:333)
==31753== 
==31753== 59,040 bytes in 670 blocks are definitely lost in loss record 430 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x687E5ED: dict_set_lk (dict.c:376)
==31753==    by 0x687E7C8: dict_set (dict.c:422)
==31753==    by 0x6882538: dict_set_dynstr (dict.c:2223)
==31753==    by 0x691C349: volume_option (graph.y:241)
==31753==    by 0x691B7F1: graphyyparse (graph.y:69)
==31753==    by 0x691D18C: glusterfs_graph_construct (graph.y:590)
==31753==    by 0x6276D9F: glfs_process_volfp (glfs-mgmt.c:54)
==31753==    by 0x62783E3: glfs_mgmt_getspec_cbk (glfs-mgmt.c:604)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753== 
==31753== 69,840 (2,880 direct, 66,960 indirect) bytes in 30 blocks are definitely lost in loss record 433 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x57F6DB5: afr_selfheal_daemon_init (afr-self-heald.c:1010)
==31753==    by 0x5818B07: init (afr.c:569)
==31753==    by 0x68858C0: __xlator_init (xlator.c:403)
==31753==    by 0x68859E1: xlator_init (xlator.c:428)
==31753==    by 0x68CE7BA: glusterfs_graph_init (graph.c:320)
==31753==    by 0x68CF39F: glusterfs_graph_activate (graph.c:670)
==31753==    by 0x6276EBF: glfs_process_volfp (glfs-mgmt.c:78)
==31753==    by 0x62783E3: glfs_mgmt_getspec_cbk (glfs-mgmt.c:604)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753==    by 0x6DA2355: rpc_clnt_notify (rpc-clnt.c:961)
==31753== 
==31753== 74,200 bytes in 530 blocks are definitely lost in loss record 434 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x68BC9BB: mem_get (mem-pool.c:513)
==31753==    by 0x68BC7DE: mem_get0 (mem-pool.c:446)
==31753==    by 0x687E568: dict_set_lk (dict.c:358)
==31753==    by 0x687E7C8: dict_set (dict.c:422)
==31753==    by 0x6882538: dict_set_dynstr (dict.c:2223)
==31753==    by 0x691C349: volume_option (graph.y:241)
==31753==    by 0x691B7F1: graphyyparse (graph.y:69)
==31753==    by 0x691D18C: glusterfs_graph_construct (graph.y:590)
==31753==    by 0x6276D9F: glfs_process_volfp (glfs-mgmt.c:54)
==31753==    by 0x62783E3: glfs_mgmt_getspec_cbk (glfs-mgmt.c:604)
==31753== 
==31753== 95,120 bytes in 580 blocks are definitely lost in loss record 436 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x68BC9BB: mem_get (mem-pool.c:513)
==31753==    by 0x68BC7DE: mem_get0 (mem-pool.c:446)
==31753==    by 0x687DABD: get_new_data (dict.c:40)
==31753==    by 0x687FA26: data_from_dynstr (dict.c:900)
==31753==    by 0x688250D: dict_set_dynstr (dict.c:2217)
==31753==    by 0x691C349: volume_option (graph.y:241)
==31753==    by 0x691B7F1: graphyyparse (graph.y:69)
==31753==    by 0x691D18C: glusterfs_graph_construct (graph.y:590)
==31753==    by 0x6276D9F: glfs_process_volfp (glfs-mgmt.c:54)
==31753==    by 0x62783E3: glfs_mgmt_getspec_cbk (glfs-mgmt.c:604)
==31753== 
==31753== 125,064 (3,600 direct, 121,464 indirect) bytes in 10 blocks are definitely lost in loss record 438 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x68BF3BC: iobuf_create_stdalloc_arena (iobuf.c:367)
==31753==    by 0x68BF738: iobuf_pool_new (iobuf.c:431)
==31753==    by 0x627410B: glusterfs_ctx_defaults_init (glfs.c:89)
==31753==    by 0x627577A: glfs_new@@GFAPI_3.4.0 (glfs.c:772)
==31753==    by 0x1088A3: glusterCatastrophe() (in /home/matt/dev/glusterfs_memory_leak/simple)
==31753==    by 0x1088FB: main (in /home/matt/dev/glusterfs_memory_leak/simple)
==31753== 
==31753== 256,080 (3,600 direct, 252,480 indirect) bytes in 30 blocks are definitely lost in loss record 446 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x68F3E39: eh_new (event-history.c:21)
==31753==    by 0x57F6D77: afr_selfheal_daemon_init (afr-self-heald.c:1005)
==31753==    by 0x5818B07: init (afr.c:569)
==31753==    by 0x68858C0: __xlator_init (xlator.c:403)
==31753==    by 0x68859E1: xlator_init (xlator.c:428)
==31753==    by 0x68CE7BA: glusterfs_graph_init (graph.c:320)
==31753==    by 0x68CF39F: glusterfs_graph_activate (graph.c:670)
==31753==    by 0x6276EBF: glfs_process_volfp (glfs-mgmt.c:78)
==31753==    by 0x62783E3: glfs_mgmt_getspec_cbk (glfs-mgmt.c:604)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753== 
==31753== 8,360,238 (192 direct, 8,360,046 indirect) bytes in 2 blocks are definitely lost in loss record 454 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x6885399: xlator_dynload (xlator.c:263)
==31753==    by 0x68854D2: xlator_set_type (xlator.c:295)
==31753==    by 0x62745B1: create_master (glfs.c:194)
==31753==    by 0x6275D23: glfs_init_common (glfs.c:981)
==31753==    by 0x6275F21: glfs_init@@GFAPI_3.4.0 (glfs.c:1034)
==31753==    by 0x1088D2: glusterCatastrophe() (in /home/matt/dev/glusterfs_memory_leak/simple)
==31753==    by 0x1088FB: main (in /home/matt/dev/glusterfs_memory_leak/simple)
==31753== 
==31753== 28,729,220 (3,200 direct, 28,726,020 indirect) bytes in 8 blocks are definitely lost in loss record 456 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x6DA2822: rpc_clnt_new (rpc-clnt.c:1112)
==31753==    by 0x6279014: glfs_mgmt_init (glfs-mgmt.c:927)
==31753==    by 0x6274863: glfs_volumes_init (glfs.c:266)
==31753==    by 0x6275D6B: glfs_init_common (glfs.c:989)
==31753==    by 0x6275F21: glfs_init@@GFAPI_3.4.0 (glfs.c:1034)
==31753==    by 0x1088D2: glusterCatastrophe() (in /home/matt/dev/glusterfs_memory_leak/simple)
==31753==    by 0x1088FB: main (in /home/matt/dev/glusterfs_memory_leak/simple)
==31753== 
==31753== 47,186,680 (1,120 direct, 47,185,560 indirect) bytes in 10 blocks are definitely lost in loss record 459 of 459
==31753==    at 0x4C2EB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31753==    by 0x68BBACF: __gf_calloc (mem-pool.c:117)
==31753==    by 0x4E3CCEB: ios_create_sample_buf (io-stats.c:508)
==31753==    by 0x4E3CE15: ios_init_sample_buf (io-stats.c:547)
==31753==    by 0x4E596BF: init (io-stats.c:3778)
==31753==    by 0x68858C0: __xlator_init (xlator.c:403)
==31753==    by 0x68859E1: xlator_init (xlator.c:428)
==31753==    by 0x68CE7BA: glusterfs_graph_init (graph.c:320)
==31753==    by 0x68CF39F: glusterfs_graph_activate (graph.c:670)
==31753==    by 0x6276EBF: glfs_process_volfp (glfs-mgmt.c:78)
==31753==    by 0x62783E3: glfs_mgmt_getspec_cbk (glfs-mgmt.c:604)
==31753==    by 0x6DA1DFD: rpc_clnt_handle_reply (rpc-clnt.c:790)
==31753== 
==31753== LEAK SUMMARY:
==31753==    definitely lost: 672,672 bytes in 4,922 blocks
==31753==    indirectly lost: 84,755,746 bytes in 3,646 blocks
==31753==      possibly lost: 52,263,754 bytes in 301 blocks
==31753==    still reachable: 127,122 bytes in 2,842 blocks
==31753==         suppressed: 0 bytes in 0 blocks
==31753== Reachable blocks (those to which a pointer was found) are not shown.
==31753== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==31753== 
==31753== For counts of detected and suppressed errors, rerun with: -v
==31753== ERROR SUMMARY: 116 errors from 116 contexts (suppressed: 0 from 0)

Actual results:

Several memory leaks are reported by valgrind.

Expected results:

No memory leaks should be reported.

Additional info:

Volume "hot" consists only of a single local brick.

Comment 1 Worker Ant 2016-11-29 11:19:23 UTC
REVIEW: http://review.gluster.org/15967 (libgfapi: fix major memory leaks after cleanup) posted (#1) for review on master by Anonymous Coward (mateusz.slupny)

Comment 2 Worker Ant 2016-11-30 11:51:12 UTC
REVIEW: http://review.gluster.org/15967 (libgfapi: fix major memory leaks after cleanup) posted (#2) for review on master by Mateusz Slupny (mateusz.slupny)

Comment 3 Worker Ant 2016-12-01 08:48:19 UTC
REVIEW: http://review.gluster.org/15967 (libgfapi: fix major memory leaks after cleanup) posted (#3) for review on master by Mateusz Slupny (mateusz.slupny)

Comment 4 Amar Tumballi 2018-08-29 03:35:33 UTC
This update is done in bulk based on the state of the patch and the time since last activity. If the issue is still seen, please reopen the bug.


Note You need to log in before you can comment on or make changes to this bug.