Bug 1121920

+++ This bug was initially created as a clone of Bug #986916 +++

Description of problem:
=========================
In a distribute-replicate volume, when directories with same names are created and deleted continuously on fuse and nfs mount points, after certain time the mount points hang. 

Refer to bug: 922792

Version-Release number of selected component (if applicable):
==============================================================
root@rhs-client11 [Jul-22-2013-16:00:29] >rpm -qa | grep glusterfs-server
glusterfs-server-3.4.0.12rhs.beta5-2.el6rhs.x86_64

root@rhs-client11 [Jul-22-2013-16:00:40] >gluster --version
glusterfs 3.4.0.12rhs.beta5 built on Jul 18 2013 07:00:39

How reproducible:

test_bug_922792.sh
===================
#!/bin/bash

dir=$(dirname $(readlink -f $0))
echo 'Script in '$dir
while :
do
        mkdir -p foo$1/bar/gee
        mkdir -p foo$1/bar/gne
        mkdir -p foo$1/lna/gme
        rm -rf foo$1
done

Steps to Reproduce:
===================
1. Create a distribute-replicate volume ( 6 x 2 ). 4 storage nodes . 3 bricks on each storage node. 

2. Start the volume.

3. Create 2 fuse and 2 nfs mounts on each RHEL5.9 and RHEL6.4 clients. 

4. From all the mount points execute: "test_bug_922792.sh" 

Actual results:
===============
After sometime, fuse and nfs mount hangs. 

Expected results:
================
Fuse and nfs mount shouldn't hang.

--- Additional comment from RHEL Product and Program Management on 2013-07-22 08:44:37 EDT ---

Since this issue was entered in bugzilla, the release flag has been
set to ? to ensure that it is properly evaluated for this release.

--- Additional comment from  on 2013-07-22 08:54:39 EDT ---

On one of the client (wingo) the mkdir and rm -rf process is in D state:
====================================================
[root@wingo ~]# ps axl | awk '$10 ~ /D/'
0     0  3946     1  18   0  65468   720 reques D    ?          0:00 mkdir -p foo/bar/gee
0     0  4608     1  18   0  65468   720 reques D    ?          0:00 mkdir -p foo/lna/gme
0     0  4616     1  15   0  58944   680 -      D    ?          0:00 rm -rf foo

[root@wingo ~]# hostname
wingo.lab.eng.blr.redhat.com

[root@wingo ~]# mount | grep gluster
glusterfs#rhs-client11:/dis_rep_vol2 on /mnt/gm1 type fuse (rw,default_permissions,allow_other,max_read=131072)
glusterfs#rhs-client11:/dis_rep_vol2 on /mnt/gm2 type fuse (rw,default_permissions,allow_other,max_read=131072)

[root@wingo ~]# mount | grep nfs
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
rhs-client11:/dis_rep_vol2 on /mnt/nfsm1 type nfs (rw,addr=10.70.36.35)
rhs-client11:/dis_rep_vol2 on /mnt/nfsm2 type nfs (rw,addr=10.70.36.35)


CLient2
===============
root@darrel [Jul-22-2013-18:20:28] >hostname
darrel.lab.eng.blr.redhat.com

root@darrel [Jul-22-2013-18:20:32] >mount | grep gluster
rhs-client11:/dis_rep_vol2 on /mnt/gm1 type fuse.glusterfs (rw,default_permissions,allow_other,max_read=131072)
rhs-client11:/dis_rep_vol2 on /mnt/gm2 type fuse.glusterfs (rw,default_permissions,allow_other,max_read=131072)

root@darrel [Jul-22-2013-18:20:38] >mount | grep nfs
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
rhs-client11:/dis_rep_vol2 on /mnt/nfsm1 type nfs (rw,addr=10.70.36.35)
rhs-client11:/dis_rep_vol2 on /mnt/nfsm2 type nfs (rw,addr=10.70.36.35)


Volume Information:
=================
root@rhs-client11 [Jul-22-2013-17:56:33] >gluster v status
Status of volume: dis_rep_vol2
Gluster process						Port	Online	Pid
------------------------------------------------------------------------------
Brick rhs-client11:/rhs/brick1/b0			49152	Y	17093
Brick rhs-client12:/rhs/brick1/b1			49152	Y	31379
Brick rhs-client11:/rhs/brick1/b2			49153	Y	17105
Brick rhs-client12:/rhs/brick1/b3			49153	Y	31391
Brick rhs-client11:/rhs/brick1/b4			49154	Y	17117
Brick rhs-client12:/rhs/brick1/b5			49154	Y	31403
Brick rhs-client13:/rhs/brick1/b6			49152	Y	23157
Brick rhs-client14:/rhs/brick1/b7			49152	Y	22336
Brick rhs-client13:/rhs/brick1/b8			49153	Y	23169
Brick rhs-client14:/rhs/brick1/b9			49153	Y	22348
Brick rhs-client13:/rhs/brick1/b10			49154	Y	23181
Brick rhs-client14:/rhs/brick1/b11			49154	Y	22360
NFS Server on localhost					2049	Y	17132
Self-heal Daemon on localhost				N/A	Y	17137
NFS Server on rhs-client14				2049	Y	22375
Self-heal Daemon on rhs-client14			N/A	Y	22379
NFS Server on rhs-client12				2049	Y	31418
Self-heal Daemon on rhs-client12			N/A	Y	31422
NFS Server on rhs-client13				2049	Y	23195
Self-heal Daemon on rhs-client13			N/A	Y	23200
 
There are no active volume tasks
root@rhs-client11 [Jul-22-2013-17:56:45] >gluster v info
 
Volume Name: dis_rep_vol2
Type: Distributed-Replicate
Volume ID: dfc73c32-4b48-46b3-8430-16b6449889c9
Status: Started
Number of Bricks: 6 x 2 = 12
Transport-type: tcp
Bricks:
Brick1: rhs-client11:/rhs/brick1/b0
Brick2: rhs-client12:/rhs/brick1/b1
Brick3: rhs-client11:/rhs/brick1/b2
Brick4: rhs-client12:/rhs/brick1/b3
Brick5: rhs-client11:/rhs/brick1/b4
Brick6: rhs-client12:/rhs/brick1/b5
Brick7: rhs-client13:/rhs/brick1/b6
Brick8: rhs-client14:/rhs/brick1/b7
Brick9: rhs-client13:/rhs/brick1/b8
Brick10: rhs-client14:/rhs/brick1/b9
Brick11: rhs-client13:/rhs/brick1/b10
Brick12: rhs-client14:/rhs/brick1/b11