Description of problem:

We were running a simultaneous streaming direct I/O test case on two nodes, kool
and salem.  The dd ran on salem, but kool didn't make any progress.  After
killing the dd on salem, the dd on kool remained stuck.  Here is the relevant
output from debugfs.

Kool (node 4):
Resource ffff81005faec0c0 Name (len=24) "       2             40a"  
Local Copy, Master is node 5
Granted Queue
Conversion Queue
01550021 PR (CW) Master:     02dc0022
Waiting Queue


Salem (node 5):
Resource ffff81006628ac80 Name (len=24) "       2             40a"  
Master Copy
Granted Queue
02960031 CW
Conversion Queue
02dc0022 NL (CW) Remote:   4 01550021
Waiting Queue

Test case:
1. create file which is 1/2 of memory.

dd if=/dev/urandom bs=1024k count=1024 of=largefile

2. on both nodes, read the file until the I/O speed stabalizes.

for i in `seq 10`; do dd if=largefile of=/dev/null bs=4k; done

3. on both nodes, read the file with direct I/O
for i in `seq 10`; do dd if=largefile of=/dev/null bs=4k iflag=direct; done


Version-Release number of selected component (if applicable):
2.6.21-rc6 + gfs from CVS HEAD

How reproducible:


Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info: