Bug 810076

+++ This bug was initially created as a clone of Bug #809083 +++

Description of problem:
In geo-replication setup with three slaves, while running running iozone on 5 clients simultaneously, one of the iozone failed. 

the client log said . 
[2012-03-30 00:57:00.049633] I [client.c:136:client_register_grace_timer] 0-master-client-0: Registering a grace timer
[2012-03-30 00:57:00.049687] I [client.c:136:client_register_grace_timer] 0-master-client-1: Registering a grace timer
[2012-03-30 00:57:04.036226] W [client.c:2078:client_rpc_notify] 0-master-client-0: Cancelling the grace timer
[2012-03-30 00:57:04.036425] I [client-handshake.c:1632:select_server_supported_programs] 0-master-client-0: Using Program GlusterFS 3git, Num (1298437), Version (330)
[2012-03-30 00:57:04.036865] I [client-handshake.c:1429:client_setvolume_cbk] 0-master-client-0: Connected to 172.17.251.54:24010, attached to remote volume '/root/bricks/m1'.
[2012-03-30 00:57:04.036900] I [client-handshake.c:1441:client_setvolume_cbk] 0-master-client-0: Server and Client lk-version numbers are not same, reopening the fds
[2012-03-30 00:57:04.037853] I [client-handshake.c:456:client_set_lk_version_cbk] 0-master-client-0: Server lk version = 1
[2012-03-30 00:57:04.042650] W [client.c:2078:client_rpc_notify] 0-master-client-1: Cancelling the grace timer
[2012-03-30 00:57:04.042848] I [client-handshake.c:1632:select_server_supported_programs] 0-master-client-1: Using Program GlusterFS 3git, Num (1298437), Version (330)
[2012-03-30 00:57:04.043226] I [client-handshake.c:1429:client_setvolume_cbk] 0-master-client-1: Connected to 172.17.251.54:24011, attached to remote volume '/root/bricks/m2'.
[2012-03-30 00:57:04.043259] I [client-handshake.c:1441:client_setvolume_cbk] 0-master-client-1: Server and Client lk-version numbers are not same, reopening the fds
[2012-03-30 00:57:04.051401] I [fuse-bridge.c:4081:fuse_graph_setup] 0-fuse: switched to graph 0
[2012-03-30 00:57:04.051600] I [client-handshake.c:456:client_set_lk_version_cbk] 0-master-client-1: Server lk version = 1
[2012-03-30 00:57:04.051942] I [fuse-bridge.c:3358:fuse_init] 0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.13 kernel 7.13
[2012-04-01 23:47:00.675108] W [client3_1-fops.c:419:client3_1_open_cbk] 0-master-client-0: remote operation failed: No such file or directory. Path: /iozone.tmp
[2012-04-01 23:47:00.695656] W [client3_1-fops.c:1054:client3_1_getxattr_cbk] 0-master-client-0: remote operation failed: No such file or directory. Path: (null)
[2012-04-01 23:47:00.706529] W [client3_1-fops.c:2607:client3_1_lookup_cbk] 0-master-client-1: remote operation failed: Invalid argument. Path: /iozone.tmp
[2012-04-01 23:47:00.706779] W [client3_1-fops.c:2607:client3_1_lookup_cbk] 0-master-client-0: remote operation failed: Invalid argument. Path: /iozone.tmp
[2012-04-01 23:47:00.706815] I [dht-layout.c:600:dht_layout_normalize] 0-master-dht: found anomalies in /iozone.tmp. holes=1 overlaps=0
[2012-04-01 23:47:00.706867] W [fuse-bridge.c:694:fuse_fd_cbk] 0-glusterfs-fuse: 99: OPEN() /iozone.tmp => -1 (No such file or directory)
[2012-04-01 23:47:15.170440] W [client3_1-fops.c:2607:client3_1_lookup_cbk] 0-master-client-0: remote operation failed: Stale NFS file handle. Path: /iozone.tmp
 

How reproducible: not often 


Steps to Reproduce:
1.Create a geo-rep setup with more slaves .
2.Run iozone on many clients simultaneously(iozone -a -f <filename>)