| Summary: | Ganesha crashes on rm -rf from mount point | |||
|---|---|---|---|---|
| Product: | Red Hat Gluster Storage | Reporter: | Ambarish <asoman> | |
| Component: | nfs-ganesha | Assignee: | Kaleb KEITHLEY <kkeithle> | |
| Status: | CLOSED NOTABUG | QA Contact: | Shashank Raj <sraj> | |
| Severity: | high | Docs Contact: | ||
| Priority: | unspecified | |||
| Version: | rhgs-3.1 | CC: | rhs-bugs, sashinde, storage-qa-internal | |
| Target Milestone: | --- | |||
| Target Release: | --- | |||
| Hardware: | x86_64 | |||
| OS: | Linux | |||
| Whiteboard: | ||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | ||
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 1373262 (view as bug list) | Environment: | ||
| Last Closed: | 2016-09-05 15:51:35 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Bug Depends On: | ||||
| Bug Blocks: | 1373262 | |||
Closing this as wrongly opened a downstream one |
Description of problem: ----------------------- Had a 4 node cluster.Created a 128G data set using iozone from 4 clients. An attempt was made to clean from the mount point. Ganesha crashed on one of the nodes in the meantime. BT from core : (gdb) bt #0 0x0000000000533300 in mdcache_key_cmp (k1=0x78, k2=0x7fb6d0001bb8) at /usr/src/debug/nfs-ganesha-2.4-rc1-0.1.1-Source/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_int.h:75 #1 0x0000000000533787 in cih_fh_cmpf (lhs=0x60, rhs=0x7fb6d0001ba0) at /usr/src/debug/nfs-ganesha-2.4-rc1-0.1.1-Source/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_hash.h:155 #2 0x00000000005337c0 in cih_fhcache_inline_lookup (tree=0x10cebb0, key=0x7fb6d0001ba0) at /usr/src/debug/nfs-ganesha-2.4-rc1-0.1.1-Source/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_hash.h:179 #3 0x000000000053411c in cih_remove_checked (entry=0x7fb6d0001820) at /usr/src/debug/nfs-ganesha-2.4-rc1-0.1.1-Source/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_hash.h:400 #4 0x0000000000538a16 in _mdcache_kill_entry (entry=0x7fb6d0001820, file=0x57c530 "/root/rpmbuild/BUILD/nfs-ganesha-2.4-rc1-0.1.1-Source/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_int.h", line=673, function=0x57cdc0 <__func__.21013> "_mdc_unreachable") at /usr/src/debug/nfs-ganesha-2.4-rc1-0.1.1-Source/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_helpers.c:1479 #5 0x000000000052b696 in _mdc_unreachable (entry=0x7fb6d0001820, file=0x57c660 "/root/rpmbuild/BUILD/nfs-ganesha-2.4-rc1-0.1.1-Source/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_handle.c", line=1151, function=0x57ce51 <__func__.21708> "mdcache_unlink") at /usr/src/debug/nfs-ganesha-2.4-rc1-0.1.1-Source/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_int.h:673 #6 0x000000000052fac2 in mdcache_unlink (dir_hdl=0x7fb728c19aa8, obj_hdl=0x7fb6d0001858, name=0x7fb410000aa0 "iozone.DUMMY.1") at /usr/src/debug/nfs-ganesha-2.4-rc1-0.1.1-Source/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_handle.c:1151 #7 0x000000000043263e in fsal_remove (parent=0x7fb728c19aa8, name=0x7fb410000aa0 "iozone.DUMMY.1") at /usr/src/debug/nfs-ganesha-2.4-rc1-0.1.1-Source/FSAL/fsal_helper.c:1541 #8 0x0000000000479298 in nfs4_op_remove (op=0x7fb578001540, data=0x7fb75af4bd90, resp=0x7fb410000960) at /usr/src/debug/nfs-ganesha-2.4-rc1-0.1.1-Source/Protocols/NFS/nfs4_op_remove.c:104 #9 0x000000000045cf86 in nfs4_Compound (arg=0x7fb578000aa8, req=0x7fb5780008e8, res=0x7fb4100014b0) at /usr/src/debug/nfs-ganesha-2.4-rc1-0.1.1-Source/Protocols/NFS/nfs4_Compound.c:734 #10 0x000000000044ad6b in nfs_rpc_execute (reqdata=0x7fb5780008c0) at /usr/src/debug/nfs-ganesha-2.4-rc1-0.1.1-Source/MainNFSD/nfs_worker_thread.c:1281 #11 0x000000000044b625 in worker_run (ctx=0x12063e0) at /usr/src/debug/nfs-ganesha-2.4-rc1-0.1.1-Source/MainNFSD/nfs_worker_thread.c:1548 #12 0x0000000000500657 in fridgethr_start_routine (arg=0x12063e0) at /usr/src/debug/nfs-ganesha-2.4-rc1-0.1.1-Source/support/fridgethr.c:550 ---Type <return> to continue, or q <return> to quit--- #13 0x00007fb7e2acbdc5 in start_thread () from /lib64/libpthread.so.0 #14 0x00007fb7e218b1cd in clone () from /lib64/libc.so.6 (gdb) (gdb) (gdb) Version-Release number of selected component (if applicable): -------------------------------------------------------------- glusterfs-ganesha-3.8.1-0.4.git56fcf39.el7rhgs.x86_64 nfs-ganesha-2.4-0.rc1.el7.x86_64 How reproducible: ----------------- Reporting the first occurrence. Steps to Reproduce: ------------------- 1. Create a huge data set using iozone 2. rm -rf <mountpoint> -v Actual results: --------------- Ganesha crashes on 1/4 nodes. Expected results: ---------------- No Ganesha crash. Additional info: --------------- 2*2 volume Client and Server OS : RHEL 7.2 256 Worker Threads were configured and rpc count was set to 128 on all clients