Bug 762285 (GLUSTER-553) - server crashed in server_fsync_cbk
Summary: server crashed in server_fsync_cbk
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: GLUSTER-553
Product: GlusterFS
Classification: Community
Component: protocol
Version: mainline
Hardware: All
OS: Linux
high
medium
Target Milestone: ---
Assignee: Vijay Bellur
QA Contact:
URL:
Whiteboard:
: GLUSTER-372 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2010-01-21 10:23 UTC by Raghavendra Bhat
Modified: 2015-12-01 16:45 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed:
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:


Attachments (Terms of Use)

Description Raghavendra Bhat 2010-01-21 08:08:02 UTC
The test was done with ENOSPC, ENOENT and EIO. During those tests one of the servers crashed. With ENOMEM all the servers crashed.

Comment 1 Raghavendra Bhat 2010-01-21 08:09:13 UTC
(In reply to comment #1)
> The test was done with ENOSPC, ENOENT and EIO. During those tests one of the
> servers crashed. With ENOMEM all the servers crashed.

With EIO all the servers crashed.

Comment 2 Raghavendra Bhat 2010-01-21 10:23:18 UTC
All the server processes were crashed due to ENOMEM generated using error-gen. This is the backtrace of the core generated.

Core was generated by `/opt/glusterfs/3.0.1rc1/sbin/glusterfsd -f dist3.vol -l /d/glusterfs/logs/distr'.
Program terminated with signal 11, Segmentation fault.
[New process 10727]
[New process 10726]
[New process 10725]
[New process 10729]
[New process 10728]
[New process 10733]
[New process 10732]
[New process 10730]
[New process 10731]
[New process 10724]
#0  0x00007f2cbc2c0280 in gf_stat_from_stat (gf_stat=0x61fd1c, stat=0x0) at ../../../../libglusterfs/src/protocol.h:87
87              gf_stat->dev         = hton64 (stat->st_dev);
(gdb) bt 
#0  0x00007f2cbc2c0280 in gf_stat_from_stat (gf_stat=0x61fd1c, stat=0x0) at ../../../../libglusterfs/src/protocol.h:87
#1  0x00007f2cbc2c2ccc in server_fsync_cbk (frame=0x7f2cb40009d8, cookie=0x7f2cb4001160, this=0x611ca0, op_ret=0, op_errno=0, 
    prebuf=0x7f2cbdce334c, postbuf=0x0) at server-protocol.c:1857
#2  0x00007f2cbc4e6e7b in iot_fsync_cbk (frame=0x7f2cb4001160, cookie=0x61fbc0, this=0x6113b0, op_ret=0, op_errno=0, 
    prebuf=0x7f2cbdce334c, postbuf=0x0) at io-threads.c:1047
#3  0x00007f2cbdce3424 in default_fsync_cbk (frame=0x61fbc0, cookie=0x621250, this=0x610b10, op_ret=0, op_errno=0, 
    prebuf=0x7f2cbdce334c, postbuf=0x0) at defaults.c:630
#4  0x00007f2cbc90e817 in error_gen_fsync_cbk (frame=0x621250, cookie=0x620e00, this=0x610220, op_ret=0, op_errno=0, 
    prebuf=0x7f2cbb6a5e80, postbuf=0x7f2cbb6a5df0) at error-gen.c:1333
#5  0x00007f2cbcb2844f in posix_fsync (frame=0x620e00, this=0x60f830, fd=0x7f2cb4000e00, datasync=0) at posix.c:2850
#6  0x00007f2cbc90eb0e in error_gen_fsync (frame=0x621250, this=0x610220, fd=0x7f2cb4000e00, flags=0) at error-gen.c:1361
#7  0x00007f2cbdce35bf in default_fsync (frame=0x61fbc0, this=0x610b10, fd=0x7f2cb4000e00, flags=0) at defaults.c:644
#8  0x00007f2cbc4e7011 in iot_fsync_wrapper (frame=0x7f2cb4001160, this=0x6113b0, fd=0x7f2cb4000e00, datasync=0) at io-threads.c:1056
#9  0x00007f2cbdcf836e in call_resume_wind (stub=0x7f2cb40011c0) at call-stub.c:2504
#10 0x00007f2cbdcfe312 in call_resume (stub=0x7f2cb40011c0) at call-stub.c:4304
#11 0x00007f2cbc4ecc8d in iot_worker_ordered (arg=0x6170e0) at io-threads.c:2338
#12 0x00007f2cbd8ac3a0 in start_thread () from /lib64/libpthread.so.0
#13 0x00007f2cbd619bdd in clone () from /lib64/libc.so.6
(gdb) p stat
$1 = (struct stat *) 0x0
(gdb) 


The server log file is attached. LTP tests were being run during the time of the crash.

Comment 3 Anand Avati 2010-01-23 09:47:29 UTC
PATCH: http://patches.gluster.com/patch/2679 in master (error-gen: unwind fsync with proper parameters)

Comment 4 Anand Avati 2010-01-23 16:17:04 UTC
*** Bug 372 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.