Description of problem:
-----------------------

Had a 4 node cluster.Created a 128G data set using iozone from 4 clients.

An attempt was made to clean from the mount point.

Ganesha crashed on one of the nodes in the meantime.

BT from core :

(gdb) bt
#0  0x0000000000533300 in mdcache_key_cmp (k1=0x78, k2=0x7fb6d0001bb8)
    at /usr/src/debug/nfs-ganesha-2.4-rc1-0.1.1-Source/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_int.h:75
#1  0x0000000000533787 in cih_fh_cmpf (lhs=0x60, rhs=0x7fb6d0001ba0)
    at /usr/src/debug/nfs-ganesha-2.4-rc1-0.1.1-Source/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_hash.h:155
#2  0x00000000005337c0 in cih_fhcache_inline_lookup (tree=0x10cebb0, key=0x7fb6d0001ba0)
    at /usr/src/debug/nfs-ganesha-2.4-rc1-0.1.1-Source/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_hash.h:179
#3  0x000000000053411c in cih_remove_checked (entry=0x7fb6d0001820)
    at /usr/src/debug/nfs-ganesha-2.4-rc1-0.1.1-Source/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_hash.h:400
#4  0x0000000000538a16 in _mdcache_kill_entry (entry=0x7fb6d0001820, 
    file=0x57c530 "/root/rpmbuild/BUILD/nfs-ganesha-2.4-rc1-0.1.1-Source/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_int.h", line=673, function=0x57cdc0 <__func__.21013> "_mdc_unreachable")
    at /usr/src/debug/nfs-ganesha-2.4-rc1-0.1.1-Source/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_helpers.c:1479
#5  0x000000000052b696 in _mdc_unreachable (entry=0x7fb6d0001820, 
    file=0x57c660 "/root/rpmbuild/BUILD/nfs-ganesha-2.4-rc1-0.1.1-Source/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_handle.c", line=1151, function=0x57ce51 <__func__.21708> "mdcache_unlink")
    at /usr/src/debug/nfs-ganesha-2.4-rc1-0.1.1-Source/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_int.h:673
#6  0x000000000052fac2 in mdcache_unlink (dir_hdl=0x7fb728c19aa8, obj_hdl=0x7fb6d0001858, 
    name=0x7fb410000aa0 "iozone.DUMMY.1")
    at /usr/src/debug/nfs-ganesha-2.4-rc1-0.1.1-Source/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_handle.c:1151
#7  0x000000000043263e in fsal_remove (parent=0x7fb728c19aa8, name=0x7fb410000aa0 "iozone.DUMMY.1")
    at /usr/src/debug/nfs-ganesha-2.4-rc1-0.1.1-Source/FSAL/fsal_helper.c:1541
#8  0x0000000000479298 in nfs4_op_remove (op=0x7fb578001540, data=0x7fb75af4bd90, resp=0x7fb410000960)
    at /usr/src/debug/nfs-ganesha-2.4-rc1-0.1.1-Source/Protocols/NFS/nfs4_op_remove.c:104
#9  0x000000000045cf86 in nfs4_Compound (arg=0x7fb578000aa8, req=0x7fb5780008e8, res=0x7fb4100014b0)
    at /usr/src/debug/nfs-ganesha-2.4-rc1-0.1.1-Source/Protocols/NFS/nfs4_Compound.c:734
#10 0x000000000044ad6b in nfs_rpc_execute (reqdata=0x7fb5780008c0)
    at /usr/src/debug/nfs-ganesha-2.4-rc1-0.1.1-Source/MainNFSD/nfs_worker_thread.c:1281
#11 0x000000000044b625 in worker_run (ctx=0x12063e0)
    at /usr/src/debug/nfs-ganesha-2.4-rc1-0.1.1-Source/MainNFSD/nfs_worker_thread.c:1548
#12 0x0000000000500657 in fridgethr_start_routine (arg=0x12063e0)
    at /usr/src/debug/nfs-ganesha-2.4-rc1-0.1.1-Source/support/fridgethr.c:550
---Type <return> to continue, or q <return> to quit---
#13 0x00007fb7e2acbdc5 in start_thread () from /lib64/libpthread.so.0
#14 0x00007fb7e218b1cd in clone () from /lib64/libc.so.6
(gdb) 
(gdb) 
(gdb) 



Version-Release number of selected component (if applicable):
--------------------------------------------------------------

glusterfs-ganesha-3.8.1-0.4.git56fcf39.el7rhgs.x86_64
nfs-ganesha-2.4-0.rc1.el7.x86_64


How reproducible:
-----------------

Reporting the first occurrence.

Steps to Reproduce:
-------------------

1. Create a huge data set using iozone

2. rm -rf <mountpoint> -v


Actual results:
---------------

Ganesha crashes on 1/4 nodes.

Expected results:
----------------

No Ganesha crash.

Additional info:
---------------

2*2 volume
Client and Server OS : RHEL 7.2
256 Worker Threads were configured and rpc count was set to 128 on all clients