Bug 761934 (GLUSTER-202)

Summary: Crash in server_inodelk_resume
Product: [Community] GlusterFS Reporter: Vikas Gorur <vikas>
Component: protocolAssignee: Vijay Bellur <vbellur>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: high Docs Contact:
Priority: low    
Version: mainlineCC: gluster-bugs, vijay, vinayak
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Vikas Gorur 2009-08-11 10:04:46 UTC
Running the test http://johnleach.co.uk/downloads/glusterfs/glusterfs-mirror-test-suite-3.tar.gz results in a reproducible segfault with git-master code.

To run the test, download and untar. Then run

$ ruby rub.rb

#0  0xb7f22371 in inode_ref (inode=0x0) at ../../../libglusterfs/src/inode.c:367
#1  0xb7516692 in server_inodelk_resume (frame=0x80a2c70, this=0x8056fa8, 
    volume=0x8098848 "mirror", loc=0x80a0a3c, cmd=13, flock=0x80a0a58)
    at ../../../../../xlators/protocol/server/src/server-protocol.c:5555
#2  0xb7f2be2e in call_resume_wind (stub=0x80a0a20)
    at ../../../libglusterfs/src/call-stub.c:2557
#3  0xb7f3096f in call_resume (stub=0x80a0a20)
    at ../../../libglusterfs/src/call-stub.c:4174
#4  0xb7516ae8 in server_inodelk (frame=0x80a2c70, bound_xl=0x8056a30, hdr=0x809ac50, 
    hdrlen=104, iobuf=0x0)
    at ../../../../../xlators/protocol/server/src/server-protocol.c:5633
#5  0xb751c503 in protocol_server_interpret (this=0x8056fa8, trans=0x8059d88, 
    hdr_p=0x809ac50 "", hdrlen=104, iobuf=0x0)
    at ../../../../../xlators/protocol/server/src/server-protocol.c:7448
#6  0xb751d157 in protocol_server_pollin (this=0x8056fa8, trans=0x8059d88)
    at ../../../../../xlators/protocol/server/src/server-protocol.c:7729
#7  0xb751d2ad in notify (this=0x8056fa8, event=2, data=0x8059d88)
    at ../../../../../xlators/protocol/server/src/server-protocol.c:7785
#8  0xb7f14a58 in xlator_notify (xl=0x8056fa8, event=2, data=0x8059d88)
    at ../../../libglusterfs/src/xlator.c:925
#9  0xb7f44663 in socket_event_poll_in (this=0x8059d88)
    at ../../../../transport/socket/src/socket.c:713
#10 0xb7f44968 in socket_event_handler (fd=7, idx=1, data=0x8059d88, poll_in=1, 
    poll_out=0, poll_err=0) at ../../../../transport/socket/src/socket.c:813
#11 0xb7f347d0 in event_dispatch_epoll_handler (event_pool=0x80519a8, events=0x8059178, 
    i=0) at ../../../libglusterfs/src/event.c:804
#12 0xb7f349a1 in event_dispatch_epoll (event_pool=0x80519a8)
    at ../../../libglusterfs/src/event.c:867
#13 0xb7f34cfc in event_dispatch (event_pool=0x80519a8)
    at ../../../libglusterfs/src/event.c:975
#14 0x0804d67b in main (argc=6, argv=0xbfefe704)
    at ../../../glusterfsd/src/glusterfsd.c:1291

Comment 1 Vinayak Hegde 2009-09-07 03:41:06 UTC
logfile of crashed server is as below.

Version      : glusterfs 2.1.0git built on Sep  7 2009 10:14:11
git: git://git.sv.gnu.org/gluster.git
Starting Time: 2009-09-07 11:31:04
Command line : glusterfsd -f /home/vinayak/volfiles/afr-server1.vol -l /home/vinayak/logs/afr-server1.log -L DEBUG
PID          : 25406
System name  : Linux
Nodename     : laptop
Kernel Release : 2.6.28-15-generic
Hardware Identifier: i686

Given volfile:
+------------------------------------------------------------------------------+
  1: volume posix
  2:   type storage/posix
  3:   option directory /tmp/export1
  4: end-volume
  5:
  6: volume locks
  7:   type features/locks
  8:   subvolumes posix
  9: end-volume
 10:
 11: volume brick
 12:   type performance/io-threads
 13:   option thread-count 8
 14:   subvolumes locks
 15: end-volume
 16:
 17: volume server
 18:   type protocol/server
 19:   option transport-type tcp
 20:   option listen-port 9332
 21:   option auth.addr.brick.allow *
 22:   subvolumes brick
 23: end-volume
 24:

+----------------------------------------------------------------------------+------------------------------------------------------------------------------+
[2009-09-07 11:31:04] D [glusterfsd.c:1265:main] glusterfs: running in pid 25406
[2009-09-07 11:31:04] D [io-threads.c:2930:init] brick: io-threads: Autoscaling: off, min_threads: 8, max_threads: 8
[2009-09-07 11:31:04] D [transport.c:145:transport_load] transport: attempt to load file /usr/local/lib/glusterfs/2.1.0git/transport/socket.so
[2009-09-07 11:31:04] W [xlator.c:656:validate_xlator_volume_options] server: option 'listen-port' is deprecated, preferred is 'transport.socket.listen-port', continuing with correction
[2009-09-07 11:31:04] D [xlator.c:285:_volume_option_value_validate] server: no range check required for 'option transport.socket.listen-port 9332'
[2009-09-07 11:31:04] N [glusterfsd.c:1284:main] glusterfs: Successfully started
[2009-09-07 11:32:22] D [addr.c:190:gf_auth] brick: allowed = "*", received addr = "127.0.0.1"
[2009-09-07 11:32:22] N [server-protocol.c:7011:mop_setvolume] server: accepted client from 127.0.0.1:1021
[2009-09-07 11:32:22] D [addr.c:190:gf_auth] brick: allowed = "*", received addr = "127.0.0.1"
[2009-09-07 11:32:22] N [server-protocol.c:7011:mop_setvolume] server: accepted client from 127.0.0.1:1020
pending frames:
frame : type(1) op(INODELK)

patchset: git://git.sv.gnu.org/gluster.git
signal received: 11
time of crash: 2009-09-07 11:32:23
configuration details:
argp 1
backtrace 1
dlfcn 1
fdatasync 1
libpthread 1
llistxattr 1
setfsid 1
spinlock 1
epoll.h 1
xattr.h 1
st_atim.tv_nsec 1
package-string: glusterfs 2.1.0git
[0xb7f35400]
/usr/local/lib/glusterfs/2.1.0git/xlator/protocol/server.so(server_inodelk_resume+0x196)[0xb74e6b16]
/usr/local/lib/libglusterfs.so.0(call_resume+0x2f7)[0xb7f19947]
/usr/local/lib/glusterfs/2.1.0git/xlator/protocol/server.so(server_inodelk+0x1e3)[0xb74e90b3]
/usr/local/lib/glusterfs/2.1.0git/xlator/protocol/server.so(protocol_server_interpret+0xbb)[0xb74e438b]
/usr/local/lib/glusterfs/2.1.0git/xlator/protocol/server.so(protocol_server_pollin+0x9c)[0xb74e463c]
/usr/local/lib/glusterfs/2.1.0git/xlator/protocol/server.so(notify+0x7f)[0xb74e46bf]
/usr/local/lib/libglusterfs.so.0(xlator_notify+0x3f)[0xb7f0a4cf]
/usr/local/lib/glusterfs/2.1.0git/transport/socket.so(socket_event_poll_in+0x3d)[0xb64d0aed]
/usr/local/lib/glusterfs/2.1.0git/transport/socket.so(socket_event_handler+0xab)[0xb64d0bab]
/usr/local/lib/libglusterfs.so.0[0xb7f25b1a]
/usr/local/lib/libglusterfs.so.0(event_dispatch+0x21)[0xb7f248e1]
glusterfsd(main+0xbfe)[0x804bc4e]
/lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe5)[0xb7d80775]
glusterfsd[0x8049f11]