Bug 962089

Description of problem:
4 storage node,1 fuse client a lot of small files write.
with the time elaspe,the write perfmance decrease.
bricks on zfsonlinux.

volume info:
Type: Distributed-Replicate
Volume ID: e5a8bda4-05b7-43a2-ad31-8dc2935c1a62
Status: Started
Number of Bricks: 2 x 2 = 4
Transport-type: tcp
Bricks:
Brick1: 192.168.1.1:/mnt/zpool/zfs
Brick2: 192.168.1.3:/mnt/zpool/zfs
Brick3: 192.168.1.5:/mnt/zpool/zfs
Brick4: 192.168.1.7:/mnt/zpool/zfs
.seal heal on.

the ip 1.4,bricks log mass log such as 
[2013-05-11 23:40:32.415823] I [server3_1-fops.c:1538:server_open_cbk] 0-CCstorage-server: 3185144: OPEN (null) (--) ==> -1 (No such file or directory)
[2013-05-11 23:40:32.580106] I [server3_1-fops.c:252:server_inodelk_cbk] 0-CCstorage-server: 3185146: INODELK (null) (--) ==> -1 (No such file or directory)
[2013-05-11 23:40:32.580808] I [server3_1-fops.c:1538:server_open_cbk] 0-CCstorage-server: 3185147: OPEN (null) (--) ==> -1 (No such file or directory)
[2013-05-11 23:40:32.711864] I [server3_1-fops.c:252:server_inodelk_cbk] 0-CCstorage-server: 3185149: INODELK (null) (--) ==> -1 (No such file or directory)
.
other ip 1.1/1.3/1.5/1.7 bricks log :
occaslly disconnect 7,1,3,5,and then connect them.
[2013-05-12 00:12:55.485252] I [server.c:703:server_rpc_notify] 0-CCstorage-server: disconnecting connectionfrom CCstorage-7-10060-2013/05/07-16:45:06:802222-CCstorage-client-2-0
[2013-05-12 00:12:55.485356] I [server-helpers.c:741:server_connection_put] 0-CCstorage-server: Shutting down connection CCstorage-7-10060-2013/05/07-16:45:06:802222-CCstorage-client-2-0
[2013-05-12 00:12:55.485404] I [server-helpers.c:629:server_connection_destroy] 0-CCstorage-server: destroyed connection of CCstorage-7-10060-2013/05/07-16:45:06:802222-CCstorage-client-2-0
[2013-05-12 00:12:56.531380] I [glusterfsd-mgmt.c:65:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2013-05-12 00:12:56.545271] I [server.c:703:server_rpc_notify] 0-CCstorage-server: disconnecting connectionfrom CCstorage-1-17013-2013/05/07-16:42:57:205102-CCstorage-client-2-0
[2013-05-12 00:12:56.545332] I [server-helpers.c:741:server_connection_put] 0-CCstorage-server: Shutting down connection CCstorage-1-17013-2013/05/07-16:42:57:205102-CCstorage-client-2-0
[2013-05-12 00:12:56.545414] I [server-helpers.c:629:server_connection_destroy] 0-CCstorage-server: destroyed connection of CCstorage-1-17013-2013/05/07-16:42:57:205102-CCstorage-client-2-0
[2013-05-12 00:12:56.545708] I [server.c:703:server_rpc_notify] 0-CCstorage-server: disconnecting connectionfrom CCstorage-5-1031-2013/05/07-16:41:22:683925-CCstorage-client-2-0
[2013-05-12 00:12:56.545757] I [server-helpers.c:741:server_connection_put] 0-CCstorage-server: Shutting down connection CCstorage-5-1031-2013/05/07-16:41:22:683925-CCstorage-client-2-0
[2013-05-12 00:12:56.545794] I [server-helpers.c:629:server_connection_destroy] 0-CCstorage-server: destroyed connection of CCstorage-5-1031-2013/05/07-16:41:22:683925-CCstorage-client-2-0
[2013-05-12 00:12:56.549487] I [server.c:703:server_rpc_notify] 0-CCstorage-server: disconnecting connectionfrom CCstorage-3-7393-2013/05/07-16:41:17:535761-CCstorage-client-2-0
[2013-05-12 00:12:56.549533] I [server-helpers.c:741:server_connection_put] 0-CCstorage-server: Shutting down connection CCstorage-3-7393-2013/05/07-16:41:17:535761-CCstorage-client-2-0
[2013-05-12 00:12:56.549575] I [server-helpers.c:629:server_connection_destroy] 0-CCstorage-server: destroyed connection of CCstorage-3-7393-2013/05/07-16:41:17:535761-CCstorage-client-2-0
[2013-05-12 00:12:57.563223] I [glusterfsd-mgmt.c:65:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2013-05-12 00:12:57.565056] I [posix-aio.c:515:posix_aio_off] 0-CCstorage-posix: Linux AIO not available at build-time. Continuing with synchronous IO.
.
On the native client ,ls -la mount point slow than normal .time/strace ls -la mount point ,find fstat hang some time,then return.real time abnormal..the user time and sys time normal.

If   gluster volume heal CCstorage info,hang..,so we can not know the status of the bricke..and then some gluster cli cannot use,you should wait some time.

Version-Release number of selected component (if applicable):

3.3.1 source install
How reproducible:
durable write a lot of small files.

Steps to Reproduce:
1.
2.
3.
  
Actual results:
write perfmance decrease..

Expected results:

write perfmance keep constant.
Additional info: