Bug 787516

Summary: glusterd is killed due to signal 11 (SIGSEGV) and systemd doesn't catch it
Product: [Community] GlusterFS Reporter: tuxdna
Component: glusterdAssignee: krishnan parthasarathi <kparthas>
Status: CLOSED DUPLICATE QA Contact:
Severity: high Docs Contact:
Priority: unspecified    
Version: 3.2.5CC: gluster-bugs, nsathyan
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-02-06 13:36:30 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
ABRT formatted core dump none

Description tuxdna 2012-02-05 22:18:39 UTC
Created attachment 559512 [details]
ABRT formatted core dump

Description of problem:
On Fedora 16 x64_64, on starting glusterd, glusterd is killed due to signal 11 (SIGSEGV). 
systemd still says that glusterd is running


Version-Release number of selected component (if applicable):
# rpm -qa 'gluster*'
glusterfs-3.2.5-6.fc16.x86_64
glusterfs-geo-replication-3.2.5-6.fc16.x86_64
glusterfs-fuse-3.2.5-6.fc16.x86_64
glusterfs-server-3.2.5-6.fc16.x86_64
glusterfs-vim-3.2.5-6.fc16.x86_64


How reproducible:
Everytime I start the glusterd service.

Steps to Reproduce:
1. sudo yum install glusterfs-server
2. sudo service glusterd start
3. tail -f /var/log/messages
4. sudo service glusterd status

Actual results:
glusterd fails to start

Expected results:
glusterd should not fail to start


Additional info:
http://pastie.org/3323980
Core dump attached

Comment 1 tuxdna 2012-02-05 22:44:27 UTC
I am adding the stack trace below which might be of help:

[root@fedorabox ccpp-2012-02-06-01:48:21-6604]# gdb /usr/sbin/glusterd coredump 
GNU gdb (GDB) Fedora (7.3.50.20110722-10.fc16)
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/sbin/glusterd...Reading symbols from /usr/lib/debug/usr/sbin/glusterfsd.debug...done.
done.
[New LWP 6604]
[New LWP 6673]
[New LWP 6609]
Missing separate debuginfo for 
Try: yum --disablerepo='*' --enablerepo='*-debuginfo' install /usr/lib/debug/.build-id/3b/62f3e9203bb91707aed487b38269ab91bb6996
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/usr/sbin/glusterd -p /run/glusterd.pid'.
Program terminated with signal 11, Segmentation fault.
#0  glusterd_friend_sm () at glusterd-sm.c:934
934	                list_for_each_entry_safe (event, tmp, &gd_friend_sm_queue, list) {
Missing separate debuginfos, use: debuginfo-install libgcc-4.6.2-1.fc16.x86_64
(gdb) bt
#0  glusterd_friend_sm () at glusterd-sm.c:934
#1  0x00007ff2bc4f671f in glusterd_peer_rpc_notify (rpc=0x257d9e0, mydata=0x2573440, event=<optimized out>, data=<optimized out>) at glusterd-handler.c:3883
#2  0x00007ff2bdf939e2 in rpc_clnt_start (rpc=<optimized out>) at rpc-clnt.c:1022
#3  0x00007ff2bc509018 in glusterd_rpc_create (rpc=0x7fffa6ae3938, options=<optimized out>, notify_fn=0x7ff2bc4f6540 <glusterd_peer_rpc_notify>, 
    notify_data=0x2573440) at glusterd-handler.c:3279
#4  0x00007ff2bc50966d in glusterd_friend_add (hoststr=0x257e320 "server", port=0, state=GD_FRIEND_STATE_DEFAULT, uuid=0x7fffa6ae5a00, rpc=0x257d9e0, 
    friend=<optimized out>, restore=_gf_true, args=0x7fffa6ae39c0) at glusterd-handler.c:3378
#5  0x00007ff2bc540033 in glusterd_store_retrieve_peers (this=<optimized out>) at glusterd-store.c:2248
#6  0x00007ff2bc540488 in glusterd_restore () at glusterd-store.c:2310
#7  0x00007ff2bc4f63fc in init (this=0x2570f10) at glusterd.c:695
#8  0x00007ff2be1b86b2 in __xlator_init (xl=0x2570f10) at xlator.c:1418
#9  xlator_init (xl=0x2570f10) at xlator.c:1441
#10 0x00007ff2be1e5a01 in glusterfs_graph_init (graph=<optimized out>) at graph.c:345
#11 0x00007ff2be1e6258 in glusterfs_graph_activate (graph=0x256c710, ctx=0x256b010) at graph.c:524
#12 0x0000000000405fe5 in glusterfs_process_volfp (ctx=0x256b010, fp=0x256ca80) at glusterfsd.c:1400
#13 0x0000000000406221 in glusterfs_volumes_init (ctx=0x256b010) at glusterfsd.c:1452
#14 0x00000000004039ce in main (argc=<optimized out>, argv=0x7fffa6aee388) at glusterfsd.c:1505
(gdb)

Comment 2 tuxdna 2012-02-05 23:03:00 UTC
Tail  of /var/log/glusterfs/etc-glusterfs-glusterd.vol.log for the last run at:
http://pastie.org/3324286

Comment 3 krishnan parthasarathi 2012-02-06 13:36:30 UTC

*** This bug has been marked as a duplicate of bug 786006 ***