Bug 805798 - glusterd crash with volume start
Summary: glusterd crash with volume start
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: glusterd
Version: pre-release
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Amar Tumballi
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks: 817967
TreeView+ depends on / blocked
 
Reported: 2012-03-22 07:38 UTC by Saurabh
Modified: 2016-01-19 06:10 UTC (History)
5 users (show)

Fixed In Version: glusterfs-3.4.0
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-07-24 17:42:17 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)

Description Saurabh 2012-03-22 07:38:22 UTC
Description of problem:

#0  0x00007fe6af13fb8a in rpc_clnt_record (clnt=0x19df550, call_frame=0x7fe6a9ff2c50, prog=0x7fe6abec9280, procnum=1, hdrlen=64, 
    rpchdr=0x7fe6a9ff2ad0, callid=3) at rpc-clnt.c:1296
1296	        au.pid                   = call_frame->root->pid;
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.47.el6_2.9.x86_64 libgcc-4.4.6-3.el6.x86_64
(gdb) bt
#0  0x00007fe6af13fb8a in rpc_clnt_record (clnt=0x19df550, call_frame=0x7fe6a9ff2c50, prog=0x7fe6abec9280, procnum=1, hdrlen=64, 
    rpchdr=0x7fe6a9ff2ad0, callid=3) at rpc-clnt.c:1296
#1  0x00007fe6af14021e in rpc_clnt_submit (rpc=0x19df550, prog=0x7fe6abec9280, procnum=1, 
    cbkfn=0x7fe6abc9e83d <gd_syncop_mgmt_lock_cbk>, proghdr=0x7fe6a9ff2bb0, proghdrcount=1, progpayload=0x0, progpayloadcount=0, 
    iobref=0x7fe69c000a40, frame=0x7fe6a9ff2c50, rsphdr=0x0, rsphdr_count=0, rsp_payload=0x0, rsp_payload_count=0, rsp_iobref=0x0)
    at rpc-clnt.c:1462
#2  0x00007fe6abc9e809 in gd_syncop_submit_request (rpc=0x19df550, req=0x7fe6a9ff2c40, cookie=0x7fe6a9ff2c50, prog=0x7fe6abec9280, 
    procnum=1, cbkfn=0x7fe6abc9e83d <gd_syncop_mgmt_lock_cbk>, xdrproc=0x7fe6aef20715 <xdr_gd1_mgmt_cluster_lock_req>)
    at glusterd-syncop.c:67
#3  0x00007fe6abc9ea4b in gd_syncop_mgmt_lock (rpc=0x19df550, 
    my_uuid=0x19d4e98 "\262\251X\360\370|F_\224\301\256\352[cU\303/etc/glusterd", recv_uuid=0x7fe6a9ff2f60 "")
    at glusterd-syncop.c:131
#4  0x00007fe6abc3de21 in gd_sync_task_begin (data=0x19b329c) at glusterd-handler.c:991
#5  0x00007fe6af3a6e7a in synctask_wrap (old_task=0x19d8fe0) at syncop.c:128
#6  0x0000003364e43610 in ?? () from /lib64/libc.so.6
#7  0x0000000000000000 in ?? ()
(gdb) 

Version-Release number of selected component (if applicable):
git head at,
commit 1522ac5c15516156bb33f8d47b0c0e7de6cd411f
Author: Kaushal M <kaushal>
Date:   Tue Mar 20 18:05:18 2012 +0530


How reproducible:
always

Steps to Reproduce:
1. create a distribute replicate volume
2. try to start this volume
3.
  
Actual results:

the start command fails while generating core

Expected results:
the volume should be in start mode

Additional info:


[2012-03-22 03:37:06.228546] I [glusterd-utils.c:267:glusterd_lock] 0-glusterd: Cluster lock held by b2a958f0-f87c-465f-94c1-aeea5b6355c3
pending frames:

patchset: git://git.gluster.com/glusterfs.git
signal received: 11
time of crash: 2012-03-22 03:37:06
configuration details:
argp 1
backtrace 1
dlfcn 1
fdatasync 1
libpthread 1
llistxattr 1
setfsid 1
spinlock 1
epoll.h 1
xattr.h 1
st_atim.tv_nsec 1
package-string: glusterfs 3git
/lib64/libc.so.6[0x3364e32900]
/root/git/inst/lib/libgfrpc.so.0(rpc_clnt_record+0x7a)[0x7fe6af13fb8a]
/root/git/inst/lib/libgfrpc.so.0(rpc_clnt_submit+0x1fb)[0x7fe6af14021e]
/root/git/inst/lib/glusterfs/3git/xlator/mgmt/glusterd.so(gd_syncop_submit_request+0x1c4)[0x7fe6abc9e809]
/root/git/inst/lib/glusterfs/3git/xlator/mgmt/glusterd.so(gd_syncop_mgmt_lock+0xca)[0x7fe6abc9ea4b]
/root/git/inst/lib/glusterfs/3git/xlator/mgmt/glusterd.so(gd_sync_task_begin+0x120)[0x7fe6abc3de21]
/root/git/inst/lib/libglusterfs.so.0(synctask_wrap+0x38)[0x7fe6af3a6e7a]
/lib64/libc.so.6[0x3364e43610]

Comment 1 Amar Tumballi 2012-03-22 09:44:10 UTC
Can you see if http://review.gluster.com/3000 fixes the issue for you?

Comment 2 Amar Tumballi 2012-03-25 07:20:13 UTC
should be fixed in latest master as synctasks are reverted from glusterd ops

Comment 3 Gowrishankar Rajaiyan 2012-05-18 08:45:08 UTC
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~#
[root@dhcp201-181 ~]# gluster volume create test4 replica 2 dhcp201-181.englab.pnq.redhat.com:/tmp/test4 dhcp201-154.englab.pnq.redhat.com:/tmp/test4 dhcp201-214.englab.pnq.redhat.com:/tmp/test4 dhcp201-127.englab.pnq.redhat.com:/tmp/test4
Creation of volume test4 has been successful. Please start the volume to access data.
[root@dhcp201-181 ~]# 
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~#


#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~#
Volume Name: test4
Type: Distributed-Replicate
Volume ID: 05f7e3a6-b0fb-4133-9489-1f52e7507e44
Status: Created
Number of Bricks: 2 x 2 = 4
Transport-type: tcp
Bricks:
Brick1: dhcp201-181.englab.pnq.redhat.com:/tmp/test4
Brick2: dhcp201-154.englab.pnq.redhat.com:/tmp/test4
Brick3: dhcp201-214.englab.pnq.redhat.com:/tmp/test4
Brick4: dhcp201-127.englab.pnq.redhat.com:/tmp/test4
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~#


#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~#
# gluster volume start test4 
Starting volume test4 has been successful
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~#


Verified: glusterfs-server-3.3.0beta4-1.x86_64


Note You need to log in before you can comment on or make changes to this bug.