Description of problem:
=======================
after adding brick start rebalance and start removing data from mount using rm -rf. 
For few files and Directories it failed with error 'No such file or directory'.


Version-Release number of selected component (if applicable):
=============================================================
3.4.0.59rhs-1.2.toyota.hotfix.el6rhs.x86_64


How reproducible:
=================
always

Steps to Reproduce:
===================
1. create and mount Dist-rep(18x2) volume 
2. create files and Directories on it.
3. add 12 new bricks in that volume.
4. start rebalance with start force option
5. from mount point start deleting data using rm -rf


[root@localhost in]# cd linux-2.6.32.63/
[root@localhost linux-2.6.32.63]# ls -l
total 4
drwxrwxr-x 42 root root 4096 Aug 11  2014 drivers
drwxrwxr-x  3 root root   17 Aug 11  2014 include
drwxrwxr-x  3 root root   20 Aug 11  2014 samples
drwxrwxr-x  3 root root   20 Aug 11  2014 scripts
drwxrwxr-x  3 root root   22 Aug 11  2014 security
drwxrwxr-x  3 root root   19 Aug 11  2014 sound
drwxrwxr-x  3 root root   17 Aug 11  2014 tools
drwxrwxr-x  3 root root   16 Aug 11  2014 virt
[root@localhost linux-2.6.32.63]# rm -rf *
rm: cannot remove `drivers': No such file or directory
rm: cannot remove `include/acpi/platform': No such file or directory
rm: cannot remove `tools/perf/util/include/asm': No such file or directory
[root@localhost linux-2.6.32.63]# ls
drivers  include  tools


Actual results:
===============
File/Directory deletion is failing with error 'No such file or Directory'


Expected results:
================
rm -rf should not fail with such error if all sub-volumes were up.


Additional info:
================

log snippet :-
[2014-08-11 04:45:10.771372] W [dht-selfheal.c:427:dht_selfheal_dir_mkdir_cbk] 0-mio-dht: selfhealing directory <gfid:cff3a67d-ee90-435f-aaf9-5e811a145fa9>/platform failed: No such file or directory
[2014-08-11 04:45:10.773968] W [dht-selfheal.c:427:dht_selfheal_dir_mkdir_cbk] 0-mio-dht: selfhealing directory <gfid:cff3a67d-ee90-435f-aaf9-5e811a145fa9>/platform failed: No such file or directory
[2014-08-11 04:45:10.775313] W [dht-selfheal.c:427:dht_selfheal_dir_mkdir_cbk] 0-mio-dht: selfhealing directory <gfid:cff3a67d-ee90-435f-aaf9-5e811a145fa9>/platform failed: No such file or directory
[2014-08-11 04:45:10.775472] W [dht-selfheal.c:427:dht_selfheal_dir_mkdir_cbk] 0-mio-dht: selfhealing directory <gfid:cff3a67d-ee90-435f-aaf9-5e811a145fa9>/platform failed: No such file or directory
[2014-08-11 04:45:10.776247] W [dht-selfheal.c:427:dht_selfheal_dir_mkdir_cbk] 0-mio-dht: selfhealing directory <gfid:cff3a67d-ee90-435f-aaf9-5e811a145fa9>/platform failed: No such file or directory
[2014-08-11 04:45:10.777131] W [dht-selfheal.c:427:dht_selfheal_dir_mkdir_cbk] 0-mio-dht: selfhealing directory <gfid:cff3a67d-ee90-435f-aaf9-5e811a145fa9>/platform failed: No such file or directory
[2014-08-11 04:45:10.852869] W [client-rpc-fops.c:2523:client3_3_opendir_cbk] 0-mio-client-41: remote operation failed: No such file or directory. Path: <gfid:cff3a67d-ee90-435f-aaf9-5e811a145fa9>/platform (9112ea8b-64e4-4051-8104-30860b0fbad5)
[2014-08-11 04:45:10.852931] W [client-rpc-fops.c:2523:client3_3_opendir_cbk] 0-mio-client-40: remote operation failed: No such file or directory. Path: <gfid:cff3a67d-ee90-435f-aaf9-5e811a145fa9>/platform (9112ea8b-64e4-4051-8104-30860b0fbad5)
[2014-08-11 04:45:10.860235] W [client-rpc-fops.c:2523:client3_3_opendir_cbk] 0-mio-client-37: remote operation failed: No such file or directory. Path: <gfid:cff3a67d-ee90-435f-aaf9-5e811a145fa9>/platform (9112ea8b-64e4-4051-8104-30860b0fbad5)
[2014-08-11 04:45:10.860565] W [client-rpc-fops.c:2523:client3_3_opendir_cbk] 0-mio-client-36: remote operation failed: No such file or directory. Path: <gfid:cff3a67d-ee90-435f-aaf9-5e811a145fa9>/platform (9112ea8b-64e4-4051-8104-30860b0fbad5)
[2014-08-11 04:45:10.860722] W [client-rpc-fops.c:2523:client3_3_opendir_cbk] 0-mio-client-38: remote operation failed: No such file or directory. Path: <gfid:cff3a67d-ee90-435f-aaf9-5e811a145fa9>/platform (9112ea8b-64e4-4051-8104-30860b0fbad5)
[2014-08-11 04:45:10.860757] W [client-rpc-fops.c:2523:client3_3_opendir_cbk] 0-mio-client-39: remote operation failed: No such file or directory. Path: <gfid:cff3a67d-ee90-435f-aaf9-5e811a145fa9>/platform (9112ea8b-64e4-4051-8104-30860b0fbad5)
[2014-08-11 04:45:10.860790] W [client-rpc-fops.c:2523:client3_3_opendir_cbk] 0-mio-client-42: remote operation failed: No such file or directory. Path: <gfid:cff3a67d-ee90-435f-aaf9-5e811a145fa9>/platform (9112ea8b-64e4-4051-8104-30860b0fbad5)
[2014-08-11 04:45:10.860818] W [client-rpc-fops.c:2523:client3_3_opendir_cbk] 0-mio-client-43: remote operation failed: No such file or directory. Path: <gfid:cff3a67d-ee90-435f-aaf9-5e811a145fa9>/platform (9112ea8b-64e4-4051-8104-30860b0fbad5)
[2014-08-11 04:45:10.860850] W [client-rpc-fops.c:2523:client3_3_opendir_cbk] 0-mio-client-44: remote operation failed: No such file or directory. Path: <gfid:cff3a67d-ee90-435f-aaf9-5e811a145fa9>/platform (9112ea8b-64e4-4051-8104-30860b0fbad5)
[2014-08-11 04:45:10.860878] W [client-rpc-fops.c:2523:client3_3_opendir_cbk] 0-mio-client-45: remote operation failed: No such file or directory. Path: <gfid:cff3a67d-ee90-435f-aaf9-5e811a145fa9>/platform (9112ea8b-64e4-4051-8104-30860b0fbad5)
[2014-08-11 04:45:10.860910] W [client-rpc-fops.c:2523:client3_3_opendir_cbk] 0-mio-client-46: remote operation failed: No such file or directory. Path: <gfid:cff3a67d-ee90-435f-aaf9-5e811a145fa9>/platform (9112ea8b-64e4-4051-8104-30860b0fbad5)
...
...
[2014-08-11 04:45:11.510973] W [client-rpc-fops.c:2523:client3_3_opendir_cbk] 0-mio-client-37: remote operation failed: No such file or directory. Path: <gfid:cff3a67d-ee90-435f-aaf9-5e811a145fa9>/platform/x86 (8d20420c-d6bf-45d9-98cc-a4fe0d18d256)
[2014-08-11 04:45:11.511417] W [client-rpc-fops.c:2523:client3_3_opendir_cbk] 0-mio-client-36: remote operation failed: No such file or directory. Path: <gfid:cff3a67d-ee90-435f-aaf9-5e811a145fa9>/platform/x86 (8d20420c-d6bf-45d9-98cc-a4fe0d18d256)
[2014-08-11 04:45:11.511730] W [client-rpc-fops.c:2523:client3_3_opendir_cbk] 0-mio-client-38: remote operation failed: No such file or directory. Path: <gfid:cff3a67d-ee90-435f-aaf9-5e811a145fa9>/platform/x86 (8d20420c-d6bf-45d9-98cc-a4fe0d18d256)
[2014-08-11 04:45:11.511769] W [client-rpc-fops.c:2523:client3_3_opendir_cbk] 0-mio-client-39: remote operation failed: No such file or directory. Path: <gfid:cff3a67d-ee90-435f-aaf9-5e811a145fa9>/platform/x86 (8d20420c-d6bf-45d9-98cc-a4fe0d18d256)
[2014-08-11 04:45:11.511812] W [client-rpc-fops.c:2523:client3_3_opendir_cbk] 0-mio-client-40: remote operation failed: No such file or directory. Path: <gfid:cff3a67d-ee90-435f-aaf9-5e811a145fa9>/platform/x86 (8d20420c-d6bf-45d9-98cc-a4fe0d18d256)
[2014-08-11 04:45:11.511859] W [client-rpc-fops.c:2523:client3_3_opendir_cbk] 0-mio-client-41: remote operation failed: No such file or directory. Path: <gfid:cff3a67d-ee90-435f-aaf9-5e811a145fa9>/platform/x86 (8d20420c-d6bf-45d9-98cc-a4fe0d18d256)
[2014-08-11 04:45:11.511892] W [client-rpc-fops.c:2523:client3_3_opendir_cbk] 0-mio-client-43: remote operation failed: No such file or directory. Path: <gfid:cff3a67d-ee90-435f-aaf9-5e811a145fa9>/platform/x86 (8d20420c-d6bf-45d9-98cc-a4fe0d18d256)
[2014-08-11 04:45:11.511919] W [client-rpc-fops.c:2523:client3_3_opendir_cbk] 0-mio-client-42: remote operation failed: No such file or directory. Path: <gfid:cff3a67d-ee90-435f-aaf9-5e811a145fa9>/platform/x86 (8d20420c-d6bf-45d9-98cc-a4fe0d18d256)
[2014-08-11 04:45:11.511967] W [client-rpc-fops.c:2523:client3_3_opendir_cbk] 0-mio-client-44: remote operation failed: No such file or directory. Path: <gfid:cff3a67d-ee90-435f-aaf9-5e811a145fa9>/platform/x86 (8d20420c-d6bf-45d9-98cc-a4fe0d18d256)
[2014-08-11 04:45:11.512006] W [client-rpc-fops.c:2523:client3_3_opendir_cbk] 0-mio-client-45: remote operation failed: No such file or directory. Path: <gfid:cff3a67d-ee90-435f-aaf9-5e811a145fa9>/platform/x86 (8d20420c-d6bf-45d9-98cc-a4fe0d18d256)
[2014-08-11 04:45:11.512039] W [client-rpc-fops.c:2523:client3_3_opendir_cbk] 0-mio-client-46: remote operation failed: No such file or directory. Path: <gfid:cff3a67d-ee90-435f-aaf9-5e811a145fa9>/platform/x86 (8d20420c-d6bf-45d9-98cc-a4fe0d18d256)
[2014-08-11 04:45:11.512081] W [client-rpc-fops.c:2523:client3_3_opendir_cbk] 0-mio-client-47: remote operation failed: No such file or directory. Path: <gfid:cff3a67d-ee90-435f-aaf9-5e811a145fa9>/platform/x86 (8d20420c-d6bf-45d9-98cc-a4fe0d18d256)
[2014-08-11 04:45:11.585285] W [nfs3.c:3514:nfs3svc_rmdir_cbk] 0-nfs: ef918778: <gfid:cff3a67d-ee90-435f-aaf9-5e811a145fa9>/platform/x86 => -1 (No such file or directory)
...