Bugzilla (bugzilla.redhat.com) will be under maintenance for infrastructure upgrades and will not be unavailable on July 31st between 12:30 AM - 05:30 AM UTC. We appreciate your understanding and patience. You can follow status.redhat.com for details.
Bug 1203433 - Brick/glusterfsd crash when tried to rm a folder with IO error
Summary: Brick/glusterfsd crash when tried to rm a folder with IO error
Keywords:
Status: CLOSED DUPLICATE of bug 1144315
Alias: None
Product: GlusterFS
Classification: Community
Component: nfs
Version: 3.5.2
Hardware: Unspecified
OS: Unspecified
unspecified
urgent
Target Milestone: ---
Assignee: bugs@gluster.org
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-03-18 19:37 UTC by Peter Auyeung
Modified: 2015-12-22 12:29 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-12-22 12:29:13 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:


Attachments (Terms of Use)
Core file on brick crash on Mar 18 (800.54 KB, application/x-gzip)
2015-03-23 17:06 UTC, Peter Auyeung
no flags Details

Description Peter Auyeung 2015-03-18 19:37:23 UTC
Description of problem:
When tried to rm -rf a folder from an nfs client, a specific brick crashed
the folder was sitting on a non-replicated volume.
We are able to see the folders on all the bricks.

rm over nfs and crash brick is very consistent and reproducible.

We were able to rm that folder over glusterfs mount

Version-Release number of selected component (if applicable):
3.5.2

How reproducible:


Steps to Reproduce:
1. rm -rf <folder>
2. brick crash
3. gluster volume <vol> start force

Actual results:


Expected results:


Additional info:

rm error
http://fpaste.org/199727/70542214/

crash log
http://fpaste.org/199708/14267042/

Comment 1 Peter Auyeung 2015-03-18 19:42:29 UTC
When tried to remove another folder over glusterfs

it crashed a brick too

This time we getting these error:
rm: cannot remove `qa_trunk.old/qa_trunk/externals/shipserv_api/.svn/text-base': Transport endpoint is not connected
rm: cannot remove `qa_trunk.old/qa_trunk/externals/shipserv_api/.svn/prop-base': Transport endpoint is not connected
rm: cannot remove `qa_trunk.old/qa_trunk/externals/shipserv_api/.svn/props': Transport endpoint is not connected
rm: cannot remove `qa_trunk.old/qa_trunk/externals/shipserv_api/.svn/tmp/text-base': Transport endpoint is not connected
rm: cannot remove `qa_trunk.old/qa_trunk/externals/shipserv_api/.svn/tmp/prop-base': Transport endpoint is not connected
rm: cannot remove `qa_trunk.old/qa_trunk/externals/shipserv_api/.svn/tmp/props': Transport endpoint is not connected


same crash pattern

[2015-03-18 19:38:14.507657] W [quota.c:3669:quota_statfs_validate_cbk] 0-sas01-quota: quota context is not present in inode (gfid:00000000-0000-0000-0000-000000000001)
pending frames:
frame : type(0) op(0)

patchset: git://git.gluster.com/glusterfs.git
signal received: 11
time of crash: 2015-03-18 19:38:17configuration details:
argp 1
backtrace 1
dlfcn 1
fdatasync 1
libpthread 1
llistxattr 1
setfsid 1
spinlock 1
epoll.h 1
xattr.h 1
st_atim.tv_nsec 1
package-string: glusterfs 3.5.2
/lib/x86_64-linux-gnu/libc.so.6(+0x36150)[0x7f62349c6150]
/lib/x86_64-linux-gnu/libc.so.6(+0x162761)[0x7f6234af2761]
/usr/lib/x86_64-linux-gnu/glusterfs/3.5.2/xlator/features/marker.so(mq_loc_fill_from_name+0x89)[0x7f622b393c59]
/usr/lib/x86_64-linux-gnu/glusterfs/3.5.2/xlator/features/marker.so(mq_readdir_cbk+0x21f)[0x7f622b39473f]
/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(default_readdir_cbk+0xc2)[0x7f62353b7292]
/usr/lib/x86_64-linux-gnu/glusterfs/3.5.2/xlator/performance/io-threads.so(iot_readdir_cbk+0xc2)[0x7f622b7b3c02]
/usr/lib/x86_64-linux-gnu/glusterfs/3.5.2/xlator/features/access-control.so(posix_acl_readdir_cbk+0xc2)[0x7f622bbe4ca2]
/usr/lib/x86_64-linux-gnu/glusterfs/3.5.2/xlator/storage/posix.so(posix_do_readdir+0x1b8)[0x7f62303688c8]
/usr/lib/x86_64-linux-gnu/glusterfs/3.5.2/xlator/storage/posix.so(posix_readdir+0x13)[0x7f6230368d43]
/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(default_readdir+0x88)[0x7f62353c0c58]
/usr/lib/x86_64-linux-gnu/glusterfs/3.5.2/xlator/features/access-control.so(posix_acl_readdir+0x23c)[0x7f622bbe708c]
/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(default_readdir+0x88)[0x7f62353c0c58]
/usr/lib/x86_64-linux-gnu/glusterfs/3.5.2/xlator/performance/io-threads.so(iot_readdir_wrapper+0x150)[0x7f622b7b7a80]
/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(call_resume+0x1c5)[0x7f62353d6235]
/usr/lib/x86_64-linux-gnu/glusterfs/3.5.2/xlator/performance/io-threads.so(iot_worker+0x146)[0x7f622b7bba66]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x7e9a)[0x7f6234d56e9a]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f6234a838bd]
---------

Comment 2 Peter Auyeung 2015-03-23 17:06:58 UTC
Created attachment 1005506 [details]
Core file on brick crash on Mar 18

Comment 3 Niels de Vos 2015-12-22 12:29:13 UTC
Should be fixed in recent versions.

*** This bug has been marked as a duplicate of bug 1144315 ***


Note You need to log in before you can comment on or make changes to this bug.