Bug 883697 - glusterd crashes when volume start fails
Summary: glusterd crashes when volume start fails
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat Storage
Component: glusterfs
Version: 2.0
Hardware: All
OS: All
medium
medium
Target Milestone: ---
: ---
Assignee: krishnan parthasarathi
QA Contact: Shruti Sampat
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2012-12-05 06:33 UTC by Shruti Sampat
Modified: 2015-11-03 23:05 UTC (History)
8 users (show)

Fixed In Version: glusterfs-3.4.0qa4-1
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-09-23 22:34:41 UTC
Embargoed:


Attachments (Terms of Use)
vdsm logs from server 1 (645.41 KB, text/x-log)
2012-12-05 06:33 UTC, Shruti Sampat
no flags Details
vdsm logs from server 2 (15.29 MB, text/x-log)
2012-12-05 06:35 UTC, Shruti Sampat
no flags Details
gluster logs from server 1 (492.46 KB, text/x-log)
2012-12-05 06:36 UTC, Shruti Sampat
no flags Details
gluster logs from server 2 (237.86 KB, text/x-log)
2012-12-05 06:37 UTC, Shruti Sampat
no flags Details

Description Shruti Sampat 2012-12-05 06:33:41 UTC
Created attachment 657996 [details]
vdsm logs from server 1

Description of problem:
---------------------------------------
When trying to start a volume from the UI, when the brick directories are absent, volume start fails, and glusterd crashes.

The following is seen in the gluster logs - 
---------------------------------------
patchset: git://git.gluster.com/glusterfs.git
signal received: 11
time of crash: 2012-12-04 23:04:42configuration details:
argp 1
backtrace 1
dlfcn 1
fdatasync 1
libpthread 1
llistxattr 1
setfsid 1
spinlock 1
epoll.h 1
xattr.h 1
st_atim.tv_nsec 1
package-string: glusterfs 3.4.0qa2
/lib64/libc.so.6[0x327e832900]
/lib64/libc.so.6[0x327e8753b5]
/lib64/libc.so.6[0x327e877cd8]
/usr/lib64/libglusterfs.so.0(synctask_destroy+0x22)[0x35d96496d2]
/usr/lib64/libglusterfs.so.0(syncenv_processor+0x20)[0x35d9649aa0]
/lib64/libpthread.so.0[0x327f0077f1]
/lib64/libc.so.6(clone+0x6d)[0x327e8e5ccd]
---------

Version-Release number of selected component (if applicable):
2.1-qa18.el6ev

How reproducible:
Frequently

Steps to Reproduce:
1. Create a distribute volume with bricks present on each node, for a two-node cluster.
2. Remove the brick directories from one of the servers.
3. Try to start the volume from the UI.
  
Actual results:
The Events tab reports "Could not start Gluster volume <vol-name>" and glusterd is found to crash.

Expected results:
glusterd should not crash

Additional info:
glusterfs version on each of the storage servers - glusterfs-3.4.0qa2-1.el6rhs.x86_64

Comment 1 Shruti Sampat 2012-12-05 06:35:47 UTC
Created attachment 657997 [details]
vdsm logs from server 2

Comment 2 Shruti Sampat 2012-12-05 06:36:48 UTC
Created attachment 657998 [details]
gluster logs from server 1

Comment 3 Shruti Sampat 2012-12-05 06:37:18 UTC
Created attachment 657999 [details]
gluster logs from server 2

Comment 5 Shireesh 2012-12-05 08:30:58 UTC
Can this be simulated without using the UI? i.e. Perform all operations directly using gluster cli and see if it still crashes.

Comment 6 Shruti Sampat 2012-12-05 08:35:07 UTC
This issue can be reproduced by executing the following steps from gluster CLI - 

1. Create a distribute volume with bricks present on each node, for a two-node cluster.
2. Remove the brick directories from one of the servers.
3. Execute "gluster volume start <vol-name>" from the other server (the one which        still has brick directories)

glusterd crashes after executing the volume start command from the other server.

Comment 7 Shireesh 2012-12-05 08:42:26 UTC
Clearly sounds like a bug in glusterfs. Changing the component.

Comment 8 Amar Tumballi 2012-12-05 08:59:39 UTC
Shruti, can you please check if this is fixed in 3.4.0qa4?

Comment 9 Shruti Sampat 2012-12-05 10:21:03 UTC
Checked in 3.4.0qa4, Amar. This issue is not occurring.

Comment 10 Amar Tumballi 2012-12-05 12:04:42 UTC
marking it for ON_QA with 3.4.0qa4-1 as fixed in version (as per comment#9)

Comment 11 Shruti Sampat 2012-12-11 08:50:58 UTC
Verified in glusterfs version 3.4.0qa4.

Comment 12 Scott Haines 2013-09-23 22:34:41 UTC
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. 

For information on the advisory, and where to find the updated files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1262.html


Note You need to log in before you can comment on or make changes to this bug.