Bug 1332069

Description of problem:
when attach tier operation was performed while operations such as - rm -rf, new file creation, directory creations from ganesha v4 mount points were in progress, IO hung for more than 4-5 minutes. 

Volume Name: ironman
Type: Tier
Volume ID: 6796ffce-cee2-4a54-8817-555843083e69
Status: Started
Number of Bricks: 10
Transport-type: tcp
Hot Tier :
Hot Tier Type : Distributed-Replicate
Number of Bricks: 2 x 2 = 4
Brick1: 10.70.47.156:/bricks/brick3/im1
Brick2: 10.70.47.156:/bricks/brick2/im1
Brick3: 10.70.47.156:/bricks/brick1/im1
Brick4: 10.70.47.156:/bricks/brick0/im1
Cold Tier:
Cold Tier Type : Disperse
Number of Bricks: 1 x (4 + 2) = 6
Brick5: 10.70.47.192:/bricks/brick0/im1
Brick6: 10.70.47.160:/bricks/brick0/im1
Brick7: 10.70.47.178:/bricks/brick0/im1
Brick8: 10.70.47.192:/bricks/brick1/im1
Brick9: 10.70.47.160:/bricks/brick1/im1
Brick10: 10.70.47.178:/bricks/brick1/im1
Options Reconfigured:
cluster.tier-mode: cache
features.ctr-enabled: on
features.quota-deem-statfs: on
features.inode-quota: on
features.quota: on
ganesha.enable: on
features.cache-invalidation: on
nfs.disable: on
performance.readdir-ahead: on
nfs-ganesha: enable
cluster.enable-shared-storage: enable


Version-Release number of selected component (if applicable):
glusterfs-server-3.7.9-2.el7rhgs.x86_64

How reproducible:
1/1

Steps to Reproduce:
1. create a disperse volume
2. enable quota and set limits, enable ganesha
2. mount the vol using nfs ganesha v4 protocol from 5 different mount points and perform linux untar form one mount point
3. create 100 dirs with 5 files within them
4. create 100 files & rename files to a different name
5. move all dirs created in step 3 within a new directory
6. perform rm -rf on the dir of linux untar
7. create files and directories
8. create a large file so file is always open for writes
9. while 6,7,8 is in progress perform attach tier

Actual results:
attach tier succeeded, however IO from all mount points hung for more than 4 minutes, after which IO operations continued. However, IO hang for this long is not acceptable

Expected results:
attach tier should succeed and no disruption to IO process should be seen

Additional info:
sosreport, statedump shall be attached. Marking the component for now as tiering.