Bug 972713

Summary: [RHS] glusterd crash seen during "volume create"
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Prasanth <pprakash>
Component: glusterfsAssignee: krishnan parthasarathi <kparthas>
Status: CLOSED ERRATA QA Contact: Prasanth <pprakash>
Severity: medium Docs Contact:
Priority: high    
Version: 2.1CC: dtsang, knarra, mmahoney, nsathyan, pprakash, rhs-bugs, shaines, ssampat, vbellur
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: glusterfs-3.4.0.12rhs-beta3-1 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-09-23 22:35:38 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
sosreport
none
core none

Description Prasanth 2013-06-10 13:00:25 UTC 
Description of problem:

glusterd crashed during volume create:

----------
[root@qa-vm05 vdsm]# gluster volume create vol11  qa-vm05.lab.eng.blr.redhat.com:/home/11 qa-vm06.lab.eng.blr.redhat.com:/home/11
Connection failed. Please check if gluster daemon is operational.
[root@qa-vm05 vdsm]# 
[root@qa-vm05 vdsm]# service glusterd status
glusterd dead but pid file exists


[2013-06-10 00:50:59.242344] W [dict.c:328:dict_set] (-->/usr/lib64/glusterfs/3.4.0.8rhs/xlator/mgmt/glusterd.so(gd_sync_task_begin+0x295) [0x7fe03795a945] (-->/usr/lib64/glusterfs/3.4.0.8rhs/xlator/mgmt/glusterd.so(gd_brick_op_phase+0xc9) [0x7fe0379594f9] (-->/usr/lib64/glusterfs/3.4.0.8rhs/xlator/mgmt/glusterd.so(gd_syncop_mgmt_brick_op+0x26f) [0x7fe0379593df]))) 0-dict: !this || !value for key=index
[2013-06-10 00:50:59.242364] E [glusterd-syncop.c:545:gd_syncop_mgmt_brick_op] 0-management: Error setting index on brick status rsp dict
pending frames:
frame : type(0) op(0)
frame : type(0) op(0)

patchset: git://git.gluster.com/glusterfs.git
signal received: 11
time of crash: 2013-06-10 00:50:59configuration details:
argp 1
backtrace 1
dlfcn 1
fdatasync 1
libpthread 1
llistxattr 1
setfsid 1
spinlock 1
epoll.h 1
xattr.h 1
st_atim.tv_nsec 1
package-string: glusterfs 3.4.0.8rhs
/lib64/libc.so.6[0x3237832920]
/usr/lib64/glusterfs/3.4.0.8rhs/xlator/mgmt/glusterd.so(gd_unlock_op_phase+0x19e)[0x7fe03795a5de]
/usr/lib64/glusterfs/3.4.0.8rhs/xlator/mgmt/glusterd.so(gd_sync_task_begin+0xd9)[0x7fe03795a789]
/usr/lib64/glusterfs/3.4.0.8rhs/xlator/mgmt/glusterd.so(glusterd_op_begin_synctask+0x3b)[0x7fe03795aaab]
/usr/lib64/glusterfs/3.4.0.8rhs/xlator/mgmt/glusterd.so(__glusterd_handle_status_volume+0x24c)[0x7fe0378f78ac]
/usr/lib64/glusterfs/3.4.0.8rhs/xlator/mgmt/glusterd.so(glusterd_big_locked_handler+0x3f)[0x7fe0378f826f]
/usr/lib64/libglusterfs.so.0(synctask_wrap+0x12)[0x3238844c02]
/lib64/libc.so.6[0x3237843b70]
---------

# rpm -qa |grep glusterfs
glusterfs-fuse-3.4.0.8rhs-1.el6rhs.x86_64
glusterfs-geo-replication-3.4.0.8rhs-1.el6rhs.x86_64
glusterfs-rdma-3.4.0.8rhs-1.el6rhs.x86_64
glusterfs-3.4.0.8rhs-1.el6rhs.x86_64
glusterfs-server-3.4.0.8rhs-1.el6rhs.x86_64
----------

Version-Release number of selected component (if applicable): glusterfs-3.4.0.8rhs-1.el6


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:
Comment 2 Prasanth 2013-06-10 13:11:15 UTC 
Created attachment 759177 [details]
sosreport
Comment 3 Prasanth 2013-06-10 13:16:55 UTC 
Created attachment 759189 [details]
core
Comment 4 krishnan parthasarathi 2013-07-11 08:44:23 UTC 
The following patches,
 7503237 syncop: synctask shouldn't yawn, it could miss a 'wake
 2b525e1 syncop: Remove task from syncbarrier's waitq before 'wake
 3496933 syncop: Update synctask state appropriately
have been available since glusterfs-3.4.0.9 builds, which should have fixed the issues of OOM-kills and crashes.
Please verify if this still happens with the latest packages.
Moving to ON_QA for verification.
Comment 5 Prasanth 2013-07-23 09:03:09 UTC 
Verified in glusterfs-3.4.0.12rhs-beta3-1

No crash seen in this glusterfs version during volume create operation.
Comment 6 Scott Haines 2013-09-23 22:35:38 UTC 
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. 

For information on the advisory, and where to find the updated files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1262.html