Bug 1361681

Description of problem:
** time difference between 2 subsequent statedump of glusterfs(fuse mount) process is 2 hours **
statedump2:
[mount/fuse.fuse - usage-type gf_fuse_mt_iov_base memusage]
size=4294967290
num_allocs=2
max_size=4294967295
max_num_allocs=5
total_allocs=4007351


statedump 3
[mount/fuse.fuse - usage-type gf_fuse_mt_iov_base memusage]
size=4294967187
num_allocs=2
max_size=4294967295
max_num_allocs=5
total_allocs=8996888

statedump 4
[mount/fuse.fuse - usage-type gf_fuse_mt_iov_base memusage]
size=4294966952
num_allocs=1
max_size=4294967295
max_num_allocs=5
total_allocs=13664729

The mem_type gf_fuse_mt_iov_base consuming almost 4.29 GB but not exceeding the max size for gf_fuse_mt_iov_base. There was no significant increase in memory consumption by gf_fuse_mt_iov_base since statedump 2 but seems to be the reason for glusterfs process being picked up by OOM killer.

It also observed that the client system was showing 3.3GB free memory even though there was no rsync' was going on volume mount point. There may be possibility of high memory intensive processes running on the system as well.


In fuse_thread_proc() function:
                iov_in[0].iov_base = GF_CALLOC (1, msg0_size,
                                                gf_fuse_mt_iov_base);

                if (!iobuf || !iov_in[0].iov_base) {
                        gf_log (this->name, GF_LOG_ERROR,
                                "Out of memory");
                        if (iobuf)
                                iobuf_unref (iobuf);
                        GF_FREE (iov_in[0].iov_base);
                        sleep (10);
                        continue;
                }
......
                if (finh->opcode == FUSE_WRITE)
                        msg = iov_in[1].iov_base;
                else {
                        if (res > msg0_size) {
                                void *b = GF_REALLOC (iov_in[0].iov_base, res);
                                if (b) {
                                        iov_in[0].iov_base = b;
                                        finh = (fuse_in_header_t *)
                                                 iov_in[0].iov_base;
                                }
                                else {
                                        gf_log ("glusterfs-fuse", GF_LOG_ERROR,
                                                "Out of memory");
                                        send_fuse_err (this, finh, ENOMEM);

                                        goto cont_err;
                                }
                        }

It seems like we are leaking memory by over-riding iov_in[0].iov_base without freeing the existing memory-ptr in the variable?

Re-alloc may return different ptr than the one passed.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info: