Description of problem: For a volume I have set quota on a subdir to 100GB and I was just trying to create data inside that directory. Meanwhile I started lookup `find . | xargs stat` in a loop from another mount point. Although this data creation was I have done rename of files in some other directory and triggered rebalance as well. Infact while data creation as mentioned above was going rename in another directory and rebalance had finished. So, altogether after sometime only data-creation and lookup was going on in parallel and that's also these two operations from different mount-points. Version-Release number of selected component (if applicable): glusterfs-3.6.0.28-1.el6rhs.x86_64 How reproducible: happen to be seen with present exercise Steps to Reproduce: 1. create a volume of type 6x2, start it 2. enable quota 3. mount volume over nfs on clients c1, c2 4. create a dir say "qa1dir/dir1" and provide non-root user ownership to qa1dir and dir1 both. 5. set limit of 100GB on qa1dir/dir1 6. create someother directory in mountpoint and create some files in this directory. 7. start creating data inside qa1dir/dir1 till quota limit is reached. 8. start renaming renaming of files in the directory created in step6. 9. start rebalance. 10. start "find . | xargs stat" in a for loop from client c2. Actual results: there are several bricks that hace coredump. bricks log, pending frames: frame : type(0) op(8) patchset: git://git.gluster.com/glusterfs.git signal received: 11 time of crash: 2014-09-08 01:32:12 configuration details: argp 1 backtrace 1 dlfcn 1 libpthread 1 llistxattr 1 setfsid 1 spinlock 1 epoll.h 1 xattr.h 1 st_atim.tv_nsec 1 package-string: glusterfs 3.6.0.28 /usr/lib64/libglusterfs.so.0(_gf_msg_backtrace_nomem+0xb6)[0x7fd76c2ddf06] /usr/lib64/libglusterfs.so.0(gf_print_trace+0x33f)[0x7fd76c2f859f] /lib64/libc.so.6[0x3c6be326b0] /usr/lib64/libglusterfs.so.0(uuid_unpack+0x0)[0x7fd76c319190] /usr/lib64/libglusterfs.so.0(+0x5ae66)[0x7fd76c318e66] /usr/lib64/libglusterfs.so.0(uuid_utoa+0x37)[0x7fd76c2f5cb7] /usr/lib64/glusterfs/3.6.0.28/xlator/features/quota.so(quota_rename_cbk+0x28e)[0x7fd760bb36be] /usr/lib64/libglusterfs.so.0(default_rename_cbk+0xfd)[0x7fd76c2ec1fd] /usr/lib64/libglusterfs.so.0(+0x4007a)[0x7fd76c2fe07a] /usr/lib64/libglusterfs.so.0(call_resume+0xa0)[0x7fd76c2ff830] /usr/lib64/glusterfs/3.6.0.28/xlator/features/marker.so(marker_rename_done+0x7a)[0x7fd760dd1e4a] /usr/lib64/glusterfs/3.6.0.28/xlator/features/marker.so(marker_rename_release_newp_lock+0x2b4)[0x7fd760dd2414] /usr/lib64/libglusterfs.so.0(default_inodelk_cbk+0xb9)[0x7fd76c2ea769] /usr/lib64/glusterfs/3.6.0.28/xlator/features/locks.so(pl_common_inodelk+0x29f)[0x7fd761613caf] /usr/lib64/glusterfs/3.6.0.28/xlator/features/locks.so(pl_inodelk+0x27)[0x7fd761614537] /usr/lib64/libglusterfs.so.0(default_inodelk_resume+0x14d)[0x7fd76c2e640d] /usr/lib64/libglusterfs.so.0(call_resume+0x2aa)[0x7fd76c2ffa3a] /usr/lib64/glusterfs/3.6.0.28/xlator/performance/io-threads.so(iot_worker+0x158)[0x7fd7613fe348] /lib64/libpthread.so.0[0x3c6c6079d1] /lib64/libc.so.6(clone+0x6d)[0x3c6bee886d] --------- bt of the coredump. #0 uuid_unpack (in=0x8 <Address 0x8 out of bounds>, uu=0x7fd6fb6f54c0) at ../../contrib/uuid/unpack.c:44 #1 0x00007fd76c318e66 in uuid_unparse_x (uu=<value optimized out>, out=0x7fd7181529e0 "1a71b01e-22bc-4b60-a0fc-3beac78cb403", fmt=0x7fd76c340d20 "%08x-%04x-%04x-%02x%02x-%02x%02x%02x%02x%02x%02x") at ../../contrib/uuid/unparse.c:55 #2 0x00007fd76c2f5cb7 in uuid_utoa (uuid=0x8 <Address 0x8 out of bounds>) at common-utils.c:2138 #3 0x00007fd760bb36be in quota_rename_cbk (frame=0x7fd76b14cd64, cookie=<value optimized out>, this=0x26563c0, op_ret=0, op_errno=61, buf=0x7fd76abb8518, preoldparent=0x7fd76abb8668, postoldparent=0x7fd76abb86d8, prenewparent=0x7fd76abb8748, postnewparent=0x7fd76abb87b8, xdata=0x0) at quota.c:1931 #4 0x00007fd76c2ec1fd in default_rename_cbk (frame=0x7fd76b12de38, cookie=<value optimized out>, this=<value optimized out>, op_ret=0, op_errno=61, buf=<value optimized out>, preoldparent=0x7fd76abb8668, postoldparent=0x7fd76abb86d8, prenewparent=0x7fd76abb8748, postnewparent=0x7fd76abb87b8, xdata=0x0) at defaults.c:961 #5 0x00007fd76c2fe07a in call_resume_unwind (stub=0x7fd76abb7f18) at call-stub.c:2604 #6 0x00007fd76c2ff830 in call_resume (stub=0x7fd76abb7f18) at call-stub.c:2843 #7 0x00007fd760dd1e4a in marker_rename_done (frame=0x7fd76b12de38, cookie=<value optimized out>, this=0x2654e50, op_ret=<value optimized out>, op_errno=<value optimized out>, xdata=<value optimized out>) at marker.c:1035 #8 0x00007fd760dd2414 in marker_rename_release_newp_lock (frame=0x7fd76b12de38, cookie=<value optimized out>, this=0x2654e50, op_ret=<value optimized out>, op_errno=<value optimized out>, xdata=<value optimized out>) at marker.c:1101 #9 0x00007fd76c2ea769 in default_inodelk_cbk (frame=0x7fd76b16215c, cookie=<value optimized out>, this=<value optimized out>, op_ret=0, op_errno=0, xdata=<value optimized out>) at defaults.c:1175 #10 0x00007fd761613caf in pl_common_inodelk (frame=0x7fd76b12abd4, this=<value optimized out>, volume=<value optimized out>, inode=<value optimized out>, cmd=7, flock=<value optimized out>, loc=0x7fd76abd0b7c, fd=0x0, xdata=0x0) at inodelk.c:792 #11 0x00007fd761614537 in pl_inodelk (frame=<value optimized out>, this=<value optimized out>, volume=<value optimized out>, loc=<value optimized out>, cmd=<value optimized out>, flock=<value optimized out>, xdata=0x0) at inodelk.c:804 #12 0x00007fd76c2e640d in default_inodelk_resume (frame=0x7fd76b16215c, this=0x2651390, volume=0x7fd71801c530 "dist-rep-marker", loc=0x7fd76abd0b7c, cmd=7, lock=0x7fd76abd0c7c, xdata=0x0) at defaults.c:1577 #13 0x00007fd76c2ffa3a in call_resume_wind (stub=0x7fd76abd0b3c) at call-stub.c:2460 #14 call_resume (stub=0x7fd76abd0b3c) at call-stub.c:2841 #15 0x00007fd7613fe348 in iot_worker (data=0x26830c0) at io-threads.c:214 #16 0x0000003c6c6079d1 in start_thread () from /lib64/libpthread.so.0 #17 0x0000003c6bee886d in clone () from /lib64/libc.so.6 (gdb) quit Expected results: No core dump exepected. Additional info:
Created attachment 935536 [details] coredump
Tried the steps in reproducer and could not hit the bug. Ran twice. 1. create a volume of type 6x2, start it 2. enable quota 3. mount volume over nfs on clients c1, c2 4. create a dir say "qa1dir/dir1" and provide non-root user ownership to qa1dir and dir1 both. 5. set limit of 30GB on qa1dir/dir1 6. create someother directory in mountpoint and create some files in this directory. 7. start creating data inside qa1dir/dir1 till quota limit is reached. 8. start renaming of files in the directory created in step6. 9. start rebalance. 10. start "find . | xargs stat" in a for loop from client c2. Regards, Susant/Venkatesh
As per the https://bugzilla.redhat.com/show_bug.cgi?id=1139473#c7, the verification post-fix is done by Susant/Venkatesh.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHBA-2014-1315.html