Bug 998024

Description of problem:
nfsd sometimes grants delegations too soon following conflicting open requests

Version-Release number of selected component (if applicable):
N/A

How reproducible:
Easily reproducible.

Steps to Reproduce:
Client 1:

1. open a file and leave it open (ensures we already have some state
established on the server when it's deciding whether to grant us a
delegation)
# vi /mnt/file1

2. switch to another terminal window and run the following
# touch /mnt/file2
# while :; do cat /mnt/file2; done
Client 1 gets a read delegation on in the OPEN reply for file2 in the
above command.

Client 2:
# date >> /mnt/file2

Actual results:
The command on client 2 never finishes.  What the server returns
NFS4ERR_DELAY to client 2 and recalls the delegation from client 1.
Client 1 returns the delegation to the server and immediately sends a
new OPEN, for which the server grants it another read delegation.  This
happens before client 2 "retries" the OPEN, so it's just going to wind
up getting NFS4ERR_DELAY again.

Expected results:
Client 2 should be able to open the file for writing after returning from its delay before Client 1 is granted another read delegation.

Additional info:
nfsd does currently track track the fact that there was a conflicting open via the nfs4_file->fi_had_conflict field... the problem is that the nfs4_file goes away when the reference count goes to zero.