Bug 1282424

Description of problem:
=================
Deletion of directory from multiple clients failing with Directory not empty

Version-Release number of selected component (if applicable):
==============
glusterfs-server-3.7.5-6


How reproducible:


Steps to Reproduce:
============
1. Create distributed replica volume and attach four hot tier bricks 
2. mount it on client using fuse and do untar the linux kernel (tar -xvf linux-3.19.tar.gz)
4. After some time delete  linux-3.19 directory but failing with 

[root@dht-rhs-23 linux]# pwd
/mnt/disrep_tier/linux
[root@dht-rhs-23 linux]# ls 
linux-3.19  linux-3.19.tar.gz  new
[root@dht-rhs-23 linux]# rm -rf linux-3.19
rm: cannot remove ‘linux-3.19/Documentation/DocBook/media’: Directory not empty
rm: cannot remove ‘linux-3.19/arch/tile/include’: Directory not empty
rm: cannot remove ‘linux-3.19/arch/um’: Directory not empty
[root@dht-rhs-23 linux]# 

Actual results:


Expected results:
===========
rm -rf should delete all files 


Additional info:
==============
[root@rhs-client19 ~]# gluster vol info disrep_tier
 
Volume Name: disrep_tier
Type: Tier
Volume ID: ea4bd2c2-efd3-4d25-bbc1-8f6d9c75dafc
Status: Started
Number of Bricks: 8
Transport-type: tcp
Hot Tier :
Hot Tier Type : Distributed-Replicate
Number of Bricks: 2 x 2 = 4
Brick1: rhs-client19.lab.eng.blr.redhat.com:/rhs/brick5/tier
Brick2: rhs-client18.lab.eng.blr.redhat.com:/rhs/brick5/tier
Brick3: rhs-client19.lab.eng.blr.redhat.com:/rhs/brick6/tier
Brick4: rhs-client18.lab.eng.blr.redhat.com:/rhs/brick6/tier
Cold Tier:
Cold Tier Type : Distributed-Replicate
Number of Bricks: 2 x 2 = 4
Brick5: rhs-client18.lab.eng.blr.redhat.com:/rhs/brick7/disrep_teri
Brick6: rhs-client19.lab.eng.blr.redhat.com:/rhs/brick7/disrep_teri
Brick7: rhs-client18.lab.eng.blr.redhat.com:/rhs/brick6/disrep_teri
Brick8: rhs-client19.lab.eng.blr.redhat.com:/rhs/brick6/disrep_teri
Options Reconfigured:
performance.readdir-ahead: on
features.ctr-enabled: on


Cold tier
=====
[root@rhs-client18 media]# pwd
/rhs/brick7/disrep_teri/linux/linux-3.19/Documentation/DocBook/media
[root@rhs-client18 media]# ls -lrth 
total 4.0K
d---------. 2 root root 6 Nov 16 16:58 v4l

Hot Tier 
=======
[root@rhs-client18 DocBook]# cd /rhs/brick5/tier/linux/linux-3.19/Documentation/DocBook/media
[root@rhs-client18 media]# ls -lrth
total 4.0K
d---------. 2 root root 6 Nov 16 16:58 v4l