Created attachment 573026 [details]
Attaching NFS Server Log

Description of problem:
whenever a brick goes offline, the client logs the following message numerous times. There isn't any failures because of these errors. 

[2012-03-27 18:36:13.622031] W [client3_1-fops.c:1773:client3_1_fxattrop_cbk] 0-dstore-client-0: remote operation failed: Transport endpoint is not connected
[2012-03-27 18:36:13.622083] E [rpc-clnt.c:382:saved_frames_unwind] (-->/usr/local/lib/libgfrpc.so.0(rpc_clnt_notify+0x13c) [0x7fc52fffdc02] (-->/usr/local/lib/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x155) [0x7fc52fffd121] (-->/usr/local/lib/libgfrpc.so.0(saved_frames_destroy+0x1f) [0x7fc52fffcb67]))) 0-dstore-client-0: forced unwinding frame type(GlusterFS 3.1) op(FXATTROP(34)) called at 2012-03-27 18:36:13.440330 (xid=0x31428x)
[2012-03-27 18:36:13.622108] W [client3_1-fops.c:1773:client3_1_fxattrop_cbk] 0-dstore-client-0: remote operation failed: Transport endpoint is not connected
[2012-03-27 18:36:13.622135] I [client.c:136:client_register_grace_timer] 0-dstore-client-0: Registering a grace timer
[2012-03-27 18:36:13.622155] I [client.c:2096:client_rpc_notify] 0-dstore-client-0: disconnected
[2012-03-27 18:36:13.622238] W [client3_1-fops.c:1773:client3_1_fxattrop_cbk] 0-dstore-client-0: remote operation failed: Transport endpoint is not connected
[2012-03-27 18:36:13.669832] E [iobuf.c:660:iobuf_unref] (-->/usr/local/lib/glusterfs/3git/xlator/protocol/client.so(client_fxattrop+0x17e) [0x7fc52bd21ea7] (-->/usr/local/lib/glusterfs/3git/xlator/protocol/client.so(client3_1_fxattrop+0x574) [0x7fc52bd4211a] (-->/usr/local/lib/glusterfs/3git/xlator/protocol/client.so(client_submit_request+0x678) [0x7fc52bd1be60]))) 0-iobuf: invalid argument: iobuf


Version-Release number of selected component (if applicable):
mainline

How reproducible:
often

Steps to Reproduce:
1.Create a Distribute-Replicate Volume
2.Start the Volume
3.Bring down one of the brick of the volume