Bug 887098

Summary: gluster mount crashes
Product: [Community] GlusterFS Reporter: rhbugs
Component: access-controlAssignee: vpshastry <vshastry>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: urgent Docs Contact:
Priority: high    
Version: 3.2.5CC: gluster-bugs, nsathyan, rwheeler
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: glusterfs-3.5.0 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 902208 (view as bug list) Environment:
Last Closed: 2014-04-17 11:39:48 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 902208    
Attachments:
Description Flags
Gluster mount crash log none

Description rhbugs 2012-12-14 02:55:36 UTC
Created attachment 663312 [details]
Gluster mount crash log

Description of problem:
Gluster mount crashes when a cp -RPp of directory is done immediately followed by a write to the newly created file.

Version-Release number of selected component (if applicable):


How reproducible:
everytime


Steps to Reproduce:
1. mount -t glusterfs -o defaults,_netdev,acl staging1:/deploy_web /home/deploy/web

2. cd /home/deploy/web
3. mkdir tmp && date > tmp/file
4. cp -Rp tmp createdir && date > createdir/date
-su: createdir/date: Software caused connection abort
  
Actual results:
mount goes away

/dev/xvdf1 on /export/deploy_web type xfs (rw)
staging1:/deploy_web on /home/deploy/web type fuse.glusterfs (rw,allow_other,max_read=131072)

deploy@staging1:~$ df -h
df: `/home/deploy/web': Transport endpoint is not connected
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1       99G  5.4G   89G   6% /
udev            1.9G   12K  1.9G   1% /dev
tmpfs           751M   15M  736M   2% /run
none            5.0M     0  5.0M   0% /run/lock
none            1.9G     0  1.9G   0% /run/shm
/dev/xvdf1       40G  2.1G   38G   6% /export/deploy_web


Expected results:
No crash

Additional info:
Upgraded to 3.3.1 deb from http://ppa.launchpad.net/semiosis/ubuntu-glusterfs-3.3/ubuntu precise main on a diffferent test server.

Still has the same issue

Removing the -p or putting a sleep in seems work around the problem
Works:
cp -R tmp createdir && date > createdir/date 

Works:
cp -Rp tmp createdir && sleep 1 && date > createdir/date

Comment 1 Amar Tumballi 2012-12-14 10:49:55 UTC
just one question, can you see for the testing sake, if 3.4.0qa5 fixes the problem for you?

Comment 2 rhbugs 2012-12-17 05:50:05 UTC
Another update, if the volume is not mounted with acl then the problem does not occur.

Amar, I am attempting to try 3.4.0qa5. Will let you know how it goes.

Comment 3 rhbugs 2012-12-19 08:14:34 UTC
Confirmed it occurs with 3.4.0qa5 also.

Comment 4 Amar Tumballi 2012-12-21 10:10:42 UTC
mount with -oacl, and happens everytime with the steps provided in bug description.

below patch fixes it.

-------------------
diff --git a/xlators/system/posix-acl/src/posix-acl.c b/xlators/system/posix-acl/src/posix-acl.c
index 3928a56..a3570c0 100644
--- a/xlators/system/posix-acl/src/posix-acl.c
+++ b/xlators/system/posix-acl/src/posix-acl.c
@@ -516,7 +516,7 @@ posix_acl_inherit_mode (struct posix_acl *acl, mode_t modein)
         if (mask_ce) {
                 mask_ce->perm &= (mode >> 3) | ~S_IRWXO;
                 mode &= (mask_ce->perm << 3) | ~S_IRWXG;
-        } else {
+        } else if (group_ce) {
                 group_ce->perm &= (mode >> 3) | ~S_IRWXO;
                 mode &= (group_ce->perm << 3) | ~S_IRWXG;
         }
-----------------

Varun, please submit this patch with proper regression test script to review.

Comment 5 Amar Tumballi 2012-12-24 08:44:22 UTC
http://review.gluster.org/4351

Comment 6 Vijay Bellur 2012-12-27 07:03:19 UTC
CHANGE: http://review.gluster.org/4351 (system/posix-acl: prevent NULL pointer dereference of group_ce) merged in master by Vijay Bellur (vbellur)

Comment 7 rhbugs 2013-01-15 03:12:21 UTC
Can this be backported to 3.3 ?

Comment 8 Vijay Bellur 2013-01-22 06:18:24 UTC
CHANGE: http://review.gluster.org/4395 (system/posix-acl: prevent NULL pointer dereference of group_ce) merged in release-3.3 by Anand Avati (avati)

Comment 9 Anand Avati 2013-08-14 22:45:53 UTC
REVIEW: http://review.gluster.org/5631 (tests: make sure to avoid 'grep' in ps output) posted (#1) for review on master by Harshavardhana (harsha)

Comment 10 Anand Avati 2013-08-18 12:08:55 UTC
COMMIT: http://review.gluster.org/5631 committed in master by Vijay Bellur (vbellur) 
------
commit 764c42813df3de9659a1ab7b7356779d2e5d64e5
Author: Harshavardhana <harsha>
Date:   Wed Aug 14 15:44:33 2013 -0700

    tests: make sure to avoid 'grep' in ps output
    
    Change-Id: I48909facd2e3a2dc52a18e44d58c0e0fa2d96ec3
    BUG: 887098
    Signed-off-by: Harshavardhana <harsha>
    Reviewed-on: http://review.gluster.org/5631
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Vijay Bellur <vbellur>

Comment 11 Anand Avati 2013-08-21 01:29:37 UTC
REVIEW: http://review.gluster.org/5672 (tests: Avoid another 'grep' in ps output) posted (#1) for review on master by Harshavardhana (harsha)

Comment 12 Anand Avati 2013-08-21 04:47:11 UTC
COMMIT: http://review.gluster.org/5672 committed in master by Vijay Bellur (vbellur) 
------
commit cc8b149f34b3dc76a4ec094b822816852f6cfd5f
Author: Harshavardhana <harsha>
Date:   Tue Aug 20 18:25:09 2013 -0700

    tests: Avoid another 'grep' in ps output
    
    Previous fix at
    ----------------------------->
    764c42813df3de9659a1ab7b7356779d2e5d64e5
    <-----------------------------
    Was incomplete
    
    Change-Id: I6662168a7af078935a5cbcfae76ec40fc0c112b8
    BUG: 887098
    Signed-off-by: Harshavardhana <harsha>
    Reviewed-on: http://review.gluster.org/5672
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Vijay Bellur <vbellur>

Comment 13 Anand Avati 2014-01-09 06:54:07 UTC
REVIEW: http://review.gluster.org/6671 (tests: fix bug-887098-gmount-crash.t check for mount liveness) posted (#1) for review on master by Krishnan Parthasarathi (kparthas)

Comment 14 Anand Avati 2014-01-09 06:55:03 UTC
REVIEW: http://review.gluster.org/6671 (tests: fix bug-887098-gmount-crash.t check for mount liveness) posted (#2) for review on master by Krishnan Parthasarathi (kparthas)

Comment 15 Niels de Vos 2014-04-17 11:39:48 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.5.0, please reopen this bug report.

glusterfs-3.5.0 has been announced on the Gluster Developers mailinglist [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/6137
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user