Bug 1236672 - quota: brick crashes when create and remove performed in parallel
Summary: quota: brick crashes when create and remove performed in parallel
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: quota
Version: rhgs-3.1
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: RHGS 3.1.1
Assignee: Vijaikumar Mallikarjuna
QA Contact: Anil Shah
URL:
Whiteboard:
Depends On:
Blocks: 1241807 1244109 1244116 1251815
TreeView+ depends on / blocked
 
Reported: 2015-06-29 17:04 UTC by Vijaikumar Mallikarjuna
Modified: 2016-09-17 12:39 UTC (History)
9 users (show)

Fixed In Version: glusterfs-3.7.1-12
Doc Type: Bug Fix
Doc Text:
Previously, brick crashed when create, write, and remove operations were performed in parallel on a quota enabled volume. With this fix, brick does not crash even if create, write, and remove operations are performed parallel.
Clone Of:
: 1244109 (view as bug list)
Environment:
Last Closed: 2015-10-05 07:15:47 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2015:1845 0 normal SHIPPED_LIVE Moderate: Red Hat Gluster Storage 3.1 update 2015-10-05 11:06:22 UTC

Description Vijaikumar Mallikarjuna 2015-06-29 17:04:39 UTC
With below test-case brick crashes in marker

Open 3 different terminal:

In first terminal create files/dirs

mkdir {1..1000}
touch 1/{1..1000}
mkdir 1/d{1..1000}

for i in {1..999}
do
    ii=`expr $i + 1`
    mv $i/* $ii/
done

In second terminal write to file
for i in {1..100}; do dd if=/dev/zero of=f$i bs=1M count=4 oflag=sync; done


In third terminal remove files
rm -rf /mnt/*



Here is the stack trace:
(gdb) bt
#0  list_del (this=0x7f7fe8019980, loc=0x7f7fa80d30d8, ctx=0x7f7fc808f800, 
    contri=0x7f7fc0007cf0, delta=<value optimized out>, 
    remove_xattr=<value optimized out>)
    at ../../../../libglusterfs/src/list.h:76
#1  mq_remove_contri (this=0x7f7fe8019980, loc=0x7f7fa80d30d8, 
    ctx=0x7f7fc808f800, contri=0x7f7fc0007cf0, delta=<value optimized out>, 
    remove_xattr=<value optimized out>) at marker-quota.c:2661
#2  0x00007f7fe719804c in mq_reduce_parent_size_task (opaque=0x7f7fa80d30d0)
    at marker-quota.c:3196
#3  0x00007f7ff759f8a2 in synctask_wrap (old_task=<value optimized out>)
    at syncop.c:376
#4  0x0000003bc7243bf0 in ?? () from /lib64/libc.so.6
#5  0x0000000000000000 in ?? ()


No quota errors in log

Comment 3 senaik 2015-07-06 06:40:14 UTC
Version : glusterfs-3.7.1-7.el6rhs.x86_64

Faced brick crash while running dbench on quota enabled on the volumes and 256 snapshot creation was in progress : 

Core was generated by `/usr/sbin/glusterfsd -s rhs-arch-srv4.lab.eng.blr.redhat.com --volfile-id snapv'.
Program terminated with signal 11, Segmentation fault.
#0  list_del (this=0x7fd6ac017e80, loc=0x7fd610009a38, ctx=0x7fd6840091d0, contri=0x7fd684009fe0, delta=<value optimized out>, remove_xattr=<value optimized out>) at ../../../../libglusterfs/src/list.h:76
76		old->prev->next = old->next;
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.166.el6.x86_64 keyutils-libs-1.4-5.el6.x86_64 krb5-libs-1.10.3-42.el6.x86_64 libacl-2.2.49-6.el6.x86_64 libaio-0.3.107-10.el6.x86_64 libattr-2.4.44-7.el6.x86_64 libcom_err-1.41.12-21.el6.x86_64 libgcc-4.4.7-16.el6.x86_64 libselinux-2.0.94-5.8.el6.x86_64 libuuid-2.17.2-12.18.el6.x86_64 openssl-1.0.1e-39.el6.x86_64 sqlite-3.6.20-1.el6.x86_64 zlib-1.2.3-29.el6.x86_64
(gdb) bt
#0  list_del (this=0x7fd6ac017e80, loc=0x7fd610009a38, ctx=0x7fd6840091d0, contri=0x7fd684009fe0, delta=<value optimized out>, remove_xattr=<value optimized out>) at ../../../../libglusterfs/src/list.h:76
#1  mq_remove_contri (this=0x7fd6ac017e80, loc=0x7fd610009a38, ctx=0x7fd6840091d0, contri=0x7fd684009fe0, delta=<value optimized out>, remove_xattr=<value optimized out>) at marker-quota.c:2634
#2  0x00007fd6aaf82a3c in mq_reduce_parent_size_task (opaque=0x7fd610009a30) at marker-quota.c:3135
#3  0x00007fd6bee271f2 in synctask_wrap (old_task=<value optimized out>) at syncop.c:381
#4  0x00007fd6bd78f8f0 in ?? () from /lib64/libc.so.6
#5  0x0000000000000000 in ?? ()
(gdb)

Comment 8 Anil Shah 2015-08-27 05:16:14 UTC
Not seeing any bricks crash after continuous parallel create and rename operation.

Bug verified on build  glusterfs-3.7.1-12.el7rhgs.x86_64

Comment 10 errata-xmlrpc 2015-10-05 07:15:47 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-1845.html


Note You need to log in before you can comment on or make changes to this bug.