Bug 770561

Summary: glusterd core dumps upon remove-brick operation from a stripe-distribute volume
Product: [Community] GlusterFS Reporter: shylesh <shmohan>
Component: glusterdAssignee: krishnan parthasarathi <kparthas>
Status: CLOSED CURRENTRELEASE QA Contact: shylesh <shmohan>
Severity: high Docs Contact:
Priority: high    
Version: mainlineCC: gluster-bugs, nsathyan
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: glusterfs-3.4.0 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-07-24 17:43:44 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: 3.3.0qa42 Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 817967    

Description shylesh 2011-12-27 12:27:34 UTC
Description of problem:

glusterd crashed upon remove-brick operation
Version-Release number of selected component (if applicable):
Mainline

How reproducible:
often

Steps to Reproduce:
1.create a 2x3 stripe-replicate volume 
2.execute the command "gluster volume remove-brick $vol $b5 $b6"

  
Actual results:
glusterd crashed

Expected results:


Additional info:
bt from dump
=============
#0  0x00000031a3e32905 in raise () from /lib64/libc.so.6
#1  0x00000031a3e340e5 in abort () from /lib64/libc.so.6
#2  0x00000031a3e2b9be in __assert_fail_base () from /lib64/libc.so.6
#3  0x00000031a3e2ba80 in __assert_fail () from /lib64/libc.so.6
#4  0x00007f8fb68dce15 in volume_volgen_graph_build_clusters (graph=0x7fffe82960e0, volinfo=0x18557d0)
    at glusterd-volgen.c:2208
#5  0x00007f8fb68dd07c in client_graph_builder (graph=0x7fffe82960e0, volinfo=0x18557d0, set_dict=0x1863d90, param=0x0)
    at glusterd-volgen.c:2249
#6  0x00007f8fb68d977a in build_graph_generic (graph=0x7fffe82960e0, volinfo=0x18557d0, mod_dict=0x1337040, param=0x0, 
    builder=0x7f8fb68dcfd2 <client_graph_builder>) at glusterd-volgen.c:1196
#7  0x00007f8fb68dd2dc in build_client_graph (graph=0x7fffe82960e0, volinfo=0x18557d0, mod_dict=0x1337040)
    at glusterd-volgen.c:2291
#8  0x00007f8fb68dfd78 in generate_single_transport_client_volfile (volinfo=0x18557d0, 
    filepath=0x7fffe8296200 "/etc/glusterd/vols/rep/rep-fuse.vol", dict=0x1337040) at glusterd-volgen.c:2863
#9  0x00007f8fb68e01ef in generate_client_volfiles (volinfo=0x18557d0) at glusterd-volgen.c:2929
#10 0x00007f8fb68e0455 in glusterd_create_volfiles_and_notify_services (volinfo=0x18557d0) at glusterd-volgen.c:2968
#11 0x00007f8fb691278b in glusterd_op_remove_brick (dict=0x1e66980, op_errstr=0x7fffe82984a8) at glusterd-brick-ops.c:1514
#12 0x00007f8fb68a0565 in glusterd_op_commit_perform (op=GD_OP_REMOVE_BRICK, dict=0x1e66980, op_errstr=0x7fffe82984a8, 
    rsp_dict=0x0) at glusterd-op-sm.c:2320
#13 0x00007f8fb689d780 in glusterd_op_ac_send_commit_op (event=0x1863ce0, ctx=0x130d590) at glusterd-op-sm.c:1681
#14 0x00007f8fb68a38db in glusterd_op_sm () at glusterd-op-sm.c:3321
#15 0x00007f8fb68bfc10 in glusterd3_1_stage_op_cbk (req=0x7f8fb5f1904c, iov=0x7f8fb5f1908c, count=1, 
    myframe=0x7f8fb800a0d4) at glusterd-rpc-ops.c:905
#16 0x00007f8fb944a28f in rpc_clnt_handle_reply (clnt=0x12efe80, pollin=0x186a560) at rpc-clnt.c:789
#17 0x00007f8fb944a84e in rpc_clnt_notify (trans=0x12f01a0, mydata=0x12efeb0, event=RPC_TRANSPORT_MSG_RECEIVED, 
    data=0x186a560) at rpc-clnt.c:908
#18 0x00007f8fb9444582 in rpc_transport_notify (this=0x12f01a0, event=RPC_TRANSPORT_MSG_RECEIVED, data=0x186a560)
    at rpc-transport.c:498
#19 0x00007f8fb6592028 in socket_event_poll_in (this=0x12f01a0) at socket.c:1675
#20 0x00007f8fb6592a7e in socket_event_handler (fd=11, idx=2, data=0x12f01a0, poll_in=1, poll_out=0, poll_err=0)
    at socket.c:1790
#21 0x00007f8fb96c62d5 in event_dispatch_epoll_handler (event_pool=0x12e0b80, events=0x12ebea0, i=0) at event.c:794
#22 0x00007f8fb96c668d in event_dispatch_epoll (event_pool=0x12e0b80) at event.c:856
#23 0x00007f8fb96c6cec in event_dispatch (event_pool=0x12e0b80) at event.c:956
#24 0x000000000040aab4 in main (argc=1, argv=0x7fffe8299348) at glusterfsd.c:1601

Comment 1 Anand Avati 2012-01-04 19:32:07 UTC
CHANGE: http://review.gluster.com/2577 (glusterd: Added checks for remove-brick handler.) merged in master by Vijay Bellur (vijay)

Comment 2 shylesh 2012-05-23 16:02:55 UTC
No crash happens