Bug 813757

Description of problem:

Log messages are complaining that the mem-pool size for the dict->data is less (thus completely making use of the mem-pool allocated to them), and are callocing for further allocations.

 [0x7ff6aec75891]))) 0-mem-pool: Mem pool is full. Callocing mem
[2012-04-17 23:06:15.051756] D [mem-pool.c:466:mem_get] (-->/usr/local/lib/libglusterfs.so.0(dict_set+0xa5) [0x7ff6aec3982b] (-->/usr/local/li
b/libglusterfs.so.0(+0x13636) [0x7ff6aec39636] (-->/usr/local/lib/libglusterfs.so.0(mem_get0+0x7b) [0x7ff6aec75891]))) 0-mem-pool: Mem pool is
 full. Callocing mem
[2012-04-17 23:06:15.051813] D [mem-pool.c:466:mem_get] (-->/usr/local/lib/glusterfs/3git/xlator/mgmt/glusterd.so(_heal_volume_add_shd_rsp+0x2
f6) [0x7ff6aaec3cf4] (-->/usr/local/lib/libglusterfs.so.0(data_copy+0x8d) [0x7ff6aec39176] (-->/usr/local/lib/libglusterfs.so.0(mem_get0+0x7b)
 [0x7ff6aec75891]))) 0-mem-pool: Mem pool is full. Callocing mem
[2012-04-17 23:06:15.052156] D [mem-pool.c:466:mem_get] (-->/usr/local/lib/libglusterfs.so.0(dict_set+0xa5) [0x7ff6aec3982b] (-->/usr/local/li
b/libglusterfs.so.0(+0x13636) [0x7ff6aec39636] (-->/usr/local/lib/libglusterfs.so.0(mem_get0+0x7b) [0x7ff6aec75891]))) 0-mem-pool: Mem pool is
 full. Callocing mem
[2012-04-17 23:06:15.052210] D [mem-pool.c:466:mem_get] (-->/usr/local/lib/glusterfs/3git/xlator/mgmt/glusterd.so(_heal_volume_add_shd_rsp+0x2
f6) [0x7ff6aaec3cf4] (-->/usr/local/lib/libglusterfs.so.0(data_copy+0x8d) [0x7ff6aec39176] (-->/usr/local/lib/libglusterfs.so.0(mem_get0+0x7b)
 [0x7ff6aec75891]))) 0-mem-pool: Mem pool is full. Callocing mem
[2012-04-17 23:06:15.052602] D [mem-pool.c:466:mem_get] (-->/usr/local/lib/libglusterfs.so.0(dict_set+0xa5) [0x7ff6aec3982b] (-->/usr/local/li
b/libglusterfs.so.0(+0x13636) [0x7ff6aec39636] (-->/usr/local/lib/libglusterfs.so.0(mem_get0+0x7b) [0x7ff6aec75891]))) 0-mem-pool: Mem pool is
 full. Callocing mem
[2012-04-17 23:06:15.052660] D [mem-pool.c:466:mem_get] (-->/usr/local/lib/glusterfs/3git/xlator/mgmt/glusterd.so(_heal_volume_add_shd_rsp+0x2
f6) [0x7ff6aaec3cf4] (-->/usr/local/lib/libglusterfs.so.0(data_copy+0x8d) [0x7ff6aec39176] (-->/usr/local/lib/libglusterfs.so.0(mem_get0+0x7b)
 [0x7ff6aec75891]))) 0-mem-pool: Mem pool is full. Callocing mem
[2012-04-17 23:06:15.053047] D [mem-pool.c:466:mem_get] (-->/usr/local/lib/libglusterfs.so.0(dict_set+0xa5) [0x7ff6aec3982b] (-->/usr/local/li
b/libglusterfs.so.0(+0x13636) [0x7ff6aec39636] (-->/usr/local/lib/libglusterfs.so.0(mem_get0+0x7b) [0x7ff6aec75891]))) 0-mem-pool: Mem pool is
 full. Callocing mem
[2012-04-17 23:06:15.053078] D [glusterd-op-sm.c:3338:glusterd_handle_node_rsp] 0-: Returning 0
[2012-04-17 23:06:15.053089] D [glusterd-op-sm.c:4465:glusterd_op_sm_inject_event] 0-glusterd: Enqueue event: 'GD_OP_EVENT_ALL_ACK'
[2012-04-17 23:06:15.055393] D [glusterd-op-sm.c:4089:glusterd_op_ac_rcvd_brick_op_acc] 0-: Returning 0
[2012-04-17 23:06:15.055438] D [glusterd-utils.c:4576:glusterd_sm_tr_log_transition_add] 0-glusterd: Transitioning from 'Brick op sent' to 'Br
ick op sent' due to event 'GD_OP_EVENT_RCVD_ACC'
[2012-04-17 23:06:15.055447] D [glusterd-utils.c:4578:glusterd_sm_tr_log_transition_add] 0-: returning 0
[2012-04-17 23:06:15.055455] D [glusterd-op-sm.c:4537:glusterd_op_sm] 0-: Dequeued event of type: 'GD_OP_EVENT_ALL_ACK'
[2012-04-17 23:06:15.056087] D [glusterd-utils.c:938:glusterd_volinfo_find] 0-: Volume mirror found
[2012-04-17 23:06:15.056123] D [glusterd-utils.c:946:glusterd_volinfo_find] 0-: Returning 0
[2012-04-17 23:06:15.544308] D [mem-pool.c:466:mem_get] (-->/usr/local/lib/libglusterfs.so.0(dict_set+0xa5) [0x7ff6aec3982b] (-->/usr/local/li
b/libglusterfs.so.0(+0x13636) [0x7ff6aec39636] (-->/usr/local/lib/libglusterfs.so.0(mem_get0+0x7b) [0x7ff6aec75891]))) 0-mem-pool: Mem pool is
 full. Callocing mem
[2012-04-17 23:06:15.544512] D [mem-pool.c:466:mem_get] (-->/usr/local/lib/libglusterfs.so.0(dict_set+0xa5) [0x7ff6aec3982b] (-->/usr/local/li
b/libglusterfs.so.0(+0x13636) [0x7ff6aec39636] (-->/usr/local/lib/libglusterfs.so.0(mem_get0+0x7b) [0x7ff6aec75891]))) 0-mem-pool: Mem pool is
 full. Callocing mem
[2012-04-17 23:06:15.544687] D [mem-pool.c:466:mem_get] (-->/usr/local/lib/libglusterfs.so.0(dict_set+0xa5) [0x7ff6aec3982b] (-->/usr/local/li
b/libglusterfs.so.0(+0x13636) [0x7ff6aec39636] (-->/usr/local/lib/libglusterfs.so.0(mem_get0+0x7b) [0x7ff6aec75891]))) 0-mem-pool: Mem pool is
 full. Callocing mem
[2012-04-17 23:06:15.544858] D [mem-pool.c:466:mem_get] (-->/usr/local/lib/libglusterfs.so.0(dict_set+0xa5) [0x7ff6aec3982b] (-->/usr/local/lib/libglusterfs.so.0(+0x13636) [0x7ff6aec39636] (-->/usr/local/lib/libglusterfs.so.0(mem_get0+0x7b) [0x7ff6aec75891]))) 0-mem-pool: Mem pool is full. Callocing mem

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. Create a replicate volume (in this case 3 replicas), start it and mount it
2. Start untarring linux-kernel tarball and immidietly bring 2 bricks down
3. After many entries have been created on the mount point (aroung 10,000), bring up the 2 bricks (volume start force)
4. Start self-healing by issuing the gluster volume heal command. 
  
Actual results:

Because during self-heal it has to do many dict opearions it runs out of the dict->data pool and has to do calloc for getting more data_t structure..

Expected results:

Mempool size should be increased such that data_t structures are allocated from out internal mempool only instead of doing calloc.


Additional info:


#define GF_MEMPOOL_COUNT_OF_DICT_T        4096
/* Considering 4 key/value pairs in a dictionary on an average */
#define GF_MEMPOOL_COUNT_OF_DATA_T        (GF_MEMPOOL_COUNT_OF_DICT_T * 4)
#define GF_MEMPOOL_COUNT_OF_DATA_PAIR_T   (GF_MEMPOOL_COUNT_OF_DICT_T * 4)


 grep GF_MEMPOOL_COUNT_OF_DATA_T
glusterfsd/src/glusterfsd.c:        ctx->dict_data_pool = mem_pool_new (data_t, GF_MEMPOOL_COUNT_OF_DATA_T);
glusterfsd/src/glusterfsd.h:#define GF_MEMPOOL_COUNT_OF_DATA_T        (GF_MEMPOOL_COUNT_OF_DICT_T * 4)