This crash was seen over rdma transport gdb) bt #0 0x00000038b5271d71 in _int_free () from /lib64/libc.so.6 #1 0x00000038b527273b in free () from /lib64/libc.so.6 #2 0x00002b1e70c09758 in __gf_free (free_ptr=0x1d0dcfc8) at mem-pool.c:291 #3 0x00002b1e70bf1d83 in gf_timer_call_cancel (ctx=0x1d0a8010, event=0x1d0dcfc8) at timer.c:132 #4 0x00002b1e70e4b21d in rpc_clnt_reconnect (trans_ptr=0x2aaab0001a18) at rpc-clnt.c:378 #5 0x00002b1e70bf1f4f in gf_timer_proc (ctx=0x1d0a8010) at timer.c:177 #6 0x00000038b5a064a7 in start_thread () from /lib64/libpthread.so.0 #7 0x00000038b52d3c2d in clone () from /lib64/libc.so.6 (gdb) info thr 5 Thread 4599 0x00000038b52d4018 in epoll_wait () from /lib64/libc.so.6 4 Thread 4600 0x00000038b5a0e4e8 in do_sigwait () from /lib64/libpthread.so.0 3 Thread 4601 0x00000038b5a0d5cb in read () from /lib64/libpthread.so.0 2 Thread 4602 0x00000038b5a0d5cb in read () from /lib64/libpthread.so.0 * 1 Thread 4604 0x00000038b5271d71 in _int_free () from /lib64/libc.so.6
This crash is happening in 'free()', that means its a genuine case of double free. Just reviewing the code, and see that in 'fini()' of rdma.c we are not doing event_unregister() of the socket, which means, we may get stale events on the transport.
PATCH: http://patches.gluster.com/patch/4847 in master (rdma: do event_unregister() in fini() too.)