Bug 764422 (GLUSTER-2690)

Summary: race between mtab updates of mount and umount
Product: [Community] GlusterFS Reporter: Csaba Henk <csaba>
Component: fuseAssignee: Csaba Henk <csaba>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: medium Docs Contact:
Priority: medium    
Version: mainlineCC: aavati, gluster-bugs, lakshmipathi, saurabh
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: ---
Regression: RTNR Mount Type: ---
Documentation: --- CRM:
Verified Versions: 3.2.3 Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Csaba Henk 2011-04-07 11:35:38 UTC
Following  python script was used to test this scenario:
###
#!/usr/bin/env python

import os, sys, tempfile, select, time

def connect(vol, sleeptime=None):
    d = tempfile.mkdtemp()
    try:
        argv = ["glusterfs", "-LDEBUG", "-l/tmp/gl0.log", '-s', "localhost", 
                 '--volfile-id', vol, '--client-pid=-1', d]
        if os.spawnvp(os.P_WAIT, argv[0], argv):
            raise RuntimeError("command failed: " + " ".join(argv))
        print >> sys.stderr, 'auxiliary glusterfs mount in place'
        os.chdir(d)
        argv = ['umount', '-l', d]
        if sleeptime != None:
	    time.sleep(sleeptime)
        if os.spawnvp(os.P_WAIT, argv[0], argv):
            raise RuntimeError("command failed: " + " ".join(argv))
    finally:
        try:
            os.rmdir(d)
        except:
            print >> sys.stderr, 'stale mount possibly left behind on ' + d
    print >> sys.stderr, 'auxiliary glusterfs mount prepared'

args = sys.argv[1:2]
if len(sys.argv) > 2:
    args.append(float(sys.argv[2])) 
connect(*args)

Comment 1 Csaba Henk 2011-04-07 14:34:44 UTC
For having a hidden mount of glusterfs, we make use of the following technique:

- mount glusterfs on some path
- have the process which is to use the mount to cd into it
- do a lazy umount of the path

This implies that the mount and umount calls for the filesystem follow each other very close -- so close that their mtab edits happen to be racy; if mount wins, then an invalid mtab entry will remain in the final mtab.

That said, purely the closeness of the two events does not imply that there is a race, there is one more thing behind it: we have made mtab update asynchronous (ie. mtab update is done in a separate process which is not waited for by glusterfs). This was to avoid a certain deadlock scenario wrt mtab update; cf. bug 762243.

Comment 2 Csaba Henk 2011-04-11 06:00:02 UTC
*** Bug 2576 has been marked as a duplicate of this bug. ***

Comment 3 Csaba Henk 2011-04-11 06:07:06 UTC
*** Bug 2670 has been marked as a duplicate of this bug. ***

Comment 4 Anand Avati 2011-04-13 01:49:54 UTC
PATCH: http://patches.gluster.com/patch/6805 in master (temporary workaround for mtab update race thingy)

Comment 5 Anand Avati 2011-05-13 17:05:56 UTC
Can you please re-send http://patches.gluster.com/patch/6743 ? Just needs a rebase

Thanks,
Avati

Comment 6 Anand Avati 2011-05-19 19:41:45 UTC
PATCH: http://patches.gluster.com/patch/7183 in master (upon daemonizing, wait on mtab update to terminate in parent)

Comment 7 Anand Avati 2011-05-26 12:55:06 UTC
PATCH: http://patches.gluster.com/patch/7249 in master (Revert "temporary workaround for mtab update race thingy")

Comment 8 Lakshmipathi G 2011-09-15 06:39:29 UTC
tested with 3.2.3 with the given python script and added the script to qa git repo

Comment 9 Junaid 2011-11-08 09:54:39 UTC
*** Bug 3785 has been marked as a duplicate of this bug. ***