Description of problem: While rewriting a large file and concurrently doing graph change, the dd write process exited with an EBADF


Version-Release number of selected component (if applicable): Upstream


How reproducible: Conistently


Steps to Reproduce:
1. dd if=/dev/urandom of=/mnt/gluster/ddfile bs=64k count=10240
2. while true; do gluster volume set test2 performance.write-behind off; sleep 1; gluster volume set test2 performance.write-behind on; sleep 1; done
3.
  
Actual results:

# dd if=/dev/urandom of=/mnt/gluster/ddfile bs=64k count=10240
10240+0 records in
10240+0 records out
671088640 bytes (671 MB) copied, 122.63 s, 5.5 MB/s

# dd if=/dev/urandom of=/mnt/gluster/ddfile bs=64k count=10240
dd: writing `ddfile': Bad file descriptor
dd: closing output file `ddfile': Bad file descripto




Expected results: Rewrites shouldn't continue without errors


Additional info:

Volume info:
Volume Name: test2
Type: Distribute
Volume ID: cc2b574e-ffae-44a3-94cd-1eaa9559738f
Status: Started
Number of Bricks: 1
Transport-type: tcp
Bricks:
Brick1: shortwing:/falcon/d1



Client log-


[2012-04-11 18:02:24.199482] I [client-handshake.c:1429:client_setvolume_cbk] 63-test2-client-0: Connected to 127.0.1.1:24010, attached to re
mote volume '/falcon/d1'.
[2012-04-11 18:02:24.199500] I [client-handshake.c:1441:client_setvolume_cbk] 63-test2-client-0: Server and Client lk-version numbers are not
 same, reopening the fds
[2012-04-11 18:02:24.214536] I [fuse-bridge.c:4119:fuse_graph_setup] 0-fuse: switched to graph 63
[2012-04-11 18:02:24.214651] I [client-handshake.c:456:client_set_lk_version_cbk] 63-test2-client-0: Server lk version = 1
[2012-04-11 18:02:24.652734] W [fd.c:807:__fd_ctx_set] (-->/usr/local/lib/glusterfs/3git/xlator/protocol/client.so(client3_1_open_cbk+0x2b4) 
[0x7f0cd3515c80] (-->/usr/local/lib/glusterfs/3git/xlator/protocol/client.so(this_fd_set_ctx+0x20b) [0x7f0cd3512bb1] (-->/usr/local/lib/libgl
usterfs.so.0(fd_ctx_set+0xb5) [0x7f0cd8018852]))) 63-: 0xa97054c test2-client-0
[2012-04-11 18:02:24.652799] W [client-helpers.c:120:this_fd_set_ctx] 63-test2-client-0:  (64756944-a8ca-4066-902d-d835252ea969): failed to s
et remote fd
[2012-04-11 18:02:24.652871] W [fd.c:807:__fd_ctx_set] (-->/usr/local/lib/glusterfs/3git/xlator/protocol/client.so(client3_1_open_cbk+0x696) 
[0x7f0cd3516062] (-->/usr/local/lib/glusterfs/3git/xlator/performance/read-ahead.so(ra_open_cbk+0x289) [0x7f0cd30dd857] (-->/usr/local/lib/li
bglusterfs.so.0(fd_ctx_set+0xb5) [0x7f0cd8018852]))) 63-: 0xa97054c test2-read-ahead
[2012-04-11 18:02:24.652897] W [read-ahead.c:103:ra_open_cbk] 63-test2-read-ahead: cannot set read-ahead context information in fd (0xa97054c
)
[2012-04-11 18:02:24.652943] W [fuse-bridge.c:3624:fuse_migrate_fd] 0-glusterfs-fuse: open on gfid (64756944-a8ca-4066-902d-d835252ea969) fai
led (Cannot allocate memory)
[2012-04-11 18:02:24.653325] I [client.c:2160:notify] 62-test2-client-0: current graph is no longer active, destroying rpc_client 
[2012-04-11 18:02:24.653437] W [fuse-resolve.c:360:fuse_resolve_fd] 0-fuse-resolve: migration of fd (0xa97054c) did not complete, failing fop with EBADF
[2012-04-11 18:02:24.653593] I [client.c:136:client_register_grace_timer] 62-test2-client-0: Registering a grace timer
[2012-04-11 18:02:24.653619] I [client.c:2099:client_rpc_notify] 62-test2-client-0: disconnected
[2012-04-11 18:02:24.688571] W [fuse-resolve.c:360:fuse_resolve_fd] 0-fuse-resolve: migration of fd (0xa97054c) did not complete, failing fop with EBADF
[2012-04-11 18:02:25.199183] W [client.c:112:client_grace_timeout] 55-test2-client-0: client grace timer expired, updating the lk-version to 2
[2012-04-11 18:02:25.203251] I [io-cache.c:1558:check_cache_size_ok] 65-test2-quick-read: Max cache size is 3888902144
[2012-04-11 18:02:25.203316] I [io-cache.c:1558:check_cache_size_ok] 65-test2-io-cache: Max cache size is 3888902144
[2012-04-11 18:02:25.204249] I [client.c:2151:notify] 65-test2-client-0: parent translators are ready, attempting connect on transport