Bug 783045

Summary: glusterd crashed with illegal instruction when tried to create dist-rep volume after building with lefence
Product: [Community] GlusterFS Reporter: M S Vishwanath Bhat <vbhat>
Component: glusterdAssignee: krishnan parthasarathi <kparthas>
Status: CLOSED NOTABUG QA Contact:
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: pre-releaseCC: gluster-bugs, mzywusko, nsathyan
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-01-20 03:38:27 EST Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Attachments:
Description Flags
glusterd log file none

Description M S Vishwanath Bhat 2012-01-19 01:42:57 EST
Created attachment 556187 [details]
glusterd log file

Description of problem:
Build the source with -lefence option. started glusterd and tried to create dist-rep volume. glusterd crashed with signal 4 Illegal Instruction. 

Version-Release number of selected component (if applicable):
master

How reproducible:
2/2

Steps to Reproduce:
1. Build the source with -lefence
2. start glusterd
3. create volume via cli
  
Actual results:
glusterd crahsed

Expected results:
glusterd should not crash

Additional info:

from core...

Reading symbols from /lib/libgcc_s.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libgcc_s.so.1
Core was generated by `glusterd -LDEBUG'.
Program terminated with signal 4, Illegal instruction.
#0  0x00007fabb4f61e87 in kill () from /lib/libc.so.6
(gdb) bt
#0  0x00007fabb4f61e87 in kill () from /lib/libc.so.6
#1  0x00007fabb5d8efc2 in EF_Abort () from /usr/lib/libefence.so.0
#2  0x00007fabb5d8e713 in free () from /usr/lib/libefence.so.0
#3  0x00007fabb23063e9 in ibv_fork_init () from /usr/lib/libibverbs.so.1
#4  0x00007fabb2518c85 in gf_rdma_init (this=0x7fabb29e4d40) at rdma.c:3967
#5  0x00007fabb251b2bf in init (this=0x7fabb29e4d40) at rdma.c:4856
#6  0x00007fabb58f6701 in rpc_transport_load (ctx=0x7fabb608fe28, options=0x7fabb6170fc0, trans_name=0x7fabb29fafe8 "rdma.management") at rpc-transport.c:322
#7  0x00007fabb58f2e9f in rpcsvc_transport_create (svc=0x7fabb3dcbf50, options=0x7fabb6170fc0, name=0x7fabb29fafe8 "rdma.management") at rpcsvc.c:1317
#8  0x00007fabb58f311c in rpcsvc_create_listener (svc=0x7fabb3dcbf50, options=0x7fabb6170fc0, name=0x7fabb29fafe8 "rdma.management") at rpcsvc.c:1384
#9  0x00007fabb58f33cd in rpcsvc_create_listeners (svc=0x7fabb3dcbf50, options=0x7fabb6170fc0, name=0x7fabb616efec "management") at rpcsvc.c:1463
#10 0x00007fabb2a5573d in init (this=0x7fabb616c6b8) at glusterd.c:887
#11 0x00007fabb5b1d3de in __xlator_init (xl=0x7fabb616c6b8) at xlator.c:364
#12 0x00007fabb5b1d508 in xlator_init (xl=0x7fabb616c6b8) at xlator.c:387
#13 0x00007fabb5b5c287 in glusterfs_graph_init (graph=0x7fabb615cf38) at graph.c:300
#14 0x00007fabb5b5ca43 in glusterfs_graph_activate (graph=0x7fabb615cf38, ctx=0x7fabb608fe28) at graph.c:490
#15 0x0000000000407a18 in glusterfs_process_volfp (ctx=0x7fabb608fe28, fp=0x7fabb615adc8) at glusterfsd.c:1485
#16 0x0000000000407b71 in glusterfs_volumes_init (ctx=0x7fabb608fe28) at glusterfsd.c:1537
#17 0x0000000000407d98 in main (argc=2, argv=0x7fffbad12f48) at glusterfsd.c:1597

I have attached the glusterd log file and archived the core file.
Comment 1 krishnan parthasarathi 2012-01-20 03:38:27 EST
From the backtrace we can see that the illegal free comes from ibv_fork_init, which is part of libibverbs. glusterd doesn't pass any information that can influence the way ibv_fork_init did its initialisation. So closing this is "NOT A BUG", since it is not an issue with glusterd ("innocent until proven guilty").
 
The unfortunate side-effect is that we cannot run tests using gluster compiled with libefence, when tranport underneath is rdma.