After running a session of fileop with 100x100x100 directories and files. We did a "rm -rf" of the mounted directory. Simultaneously we added a volume and gave rebalance on the servers. The GNFS server crashed with following bt ---------------------------------------------------------------------------- ---------------------------------------------------------------------------- (gdb) bt #0 0x00002aaaab970808 in nfs3svc_access_cbk (frame=0x2ae1d517dccc, cookie=<value optimized out>, this=<value optimized out>, op_ret=<value optimized out>, op_errno=117, buf=0x2aaab86b67c0) at nfs3.c:1343 #1 0x00002aaaab9650a3 in nfs_fop_stat_cbk (frame=0x2ae1d517dccc, cookie=0x2aaaab98c23d, this=0x0, op_ret=117, op_errno=117, buf=0x2aaab86b67c0) at nfs-fops.c:356 #2 0x00002aaaab741e8c in io_stats_stat_cbk (frame=0x2ae1d4f6e7ec, cookie=<value optimized out>, this=<value optimized out>, op_ret=0, op_errno=117, buf=0x2aaab86b67c0) at io-stats.c:487 #3 0x00002ae1d408abd9 in default_stat_cbk (frame=0x2ae1d4f74a68, cookie=<value optimized out>, this=<value optimized out>, op_ret=0, op_errno=117, buf=0x2aaab86b67c0) at defaults.c:55 #4 0x00002ae1d408abd9 in default_stat_cbk (frame=0x2ae1d4f75224, cookie=<value optimized out>, this=<value optimized out>, op_ret=0, op_errno=117, buf=0x2aaab86b67c0) at defaults.c:55 #5 0x00002ae1d408abd9 in default_stat_cbk (frame=0x2ae1d4f743b4, cookie=<value optimized out>, this=<value optimized out>, op_ret=0, op_errno=117, buf=0x2aaab86b67c0) at defaults.c:55 #6 0x00002aaaaaf107fd in wb_stat_cbk (frame=0x2ae1d4f74b70, cookie=<value optimized out>, this=<value optimized out>, op_ret=0, op_errno=117, buf=0x2aaab86b67c0) at write-behind.c:629 #7 0x00002aaaaaceeb54 in dht_attr_cbk (frame=0x2ae1d4f6d34c, cookie=<value optimized out>, this=0x155fc588, op_ret=<value optimized out>, op_errno=<value optimized out>, stbuf=<value optimized out>) at dht-common.c:1179 #8 0x00002aaaaaac7bc7 in client3_1_stat_cbk (req=<value optimized out>, iov=<value optimized out>, count=<value optimized out>, myframe=0x2ae1d4f74648) at client3_1-fops.c:367 #9 0x00002ae1d42db252 in rpc_clnt_handle_reply (clnt=0x1561bf88, pollin=<value optimized out>) at rpc-clnt.c:752 #10 0x00002ae1d42db44d in rpc_clnt_notify (trans=<value optimized out>, mydata=0x1561bfb8, event=<value optimized out>, data=0x75) at rpc-clnt.c:865 #11 0x00002ae1d42d66ec in rpc_transport_notify (this=0x2ae1d517dccc, event=3094046656, data=0x75) at rpc-transport.c:1142 #12 0x00002aaaaeed230f in socket_event_poll_in (this=0x1561c1a8) at socket.c:1619 #13 0x00002aaaaeed2498 in socket_event_handler (fd=<value optimized out>, idx=1, data=0x1561c1a8, poll_in=1, poll_out=0, poll_err=0) at socket.c:1733 #14 0x00002ae1d409dc37 in event_dispatch_epoll (event_pool=0x155e9198) at event.c:812 #15 0x00000000004046fd in main (argc=7, argv=0x7fff2ebf8998) at glusterfsd.c:1410 ----------------------------------------------------------------------------- -----------------------------------------------------------------------------
Fixed with the patch http://patches.gluster.com/patch/5461/