Created attachment 1479868 [details]
dmesg output (things go south around 3982.800208)

Description of problem:

We have 100 or so Linux machines running Scientific Linux 7.4 with various kernels:

3.10.0-693.5.2.el7
3.10.0-514.21.1.el7

mounting CIFS shares on a BlueArc/Hitachi NAS.  We occasionally see "stuck files", meaning files that show up in a directory listing but can't be removed, read, written to, or accessed in any way.  The server reports there is still an open handle on the file, but there are no processes still accessing the file on the client.  The only way to resolve the problem is to have an administrator to force close the file on the server or to reboot the Linux client machine (unmounting all CIFS shares does not resolve it either).

After some investigation, it seems that when an application with an open CIFS file is interrupted with Ctrl-C, sometimes the kernel is not sending a close request to the server on that file handle.  If you then try to delete it, it never gets deleted (because the server still sees an open handle) and the server comes back with a STATUS_DELETE_PENDING error on any further attempts to access,  create, or delete a file with the same path.

The following loop:

while true; do rm -f TMP; sleep 0.01s ; date > TMP; done

creates this sequence of SMB operations on the network, according to tcpdump:

...
# rm -f TMP;
Create Request (asks for delete-on-close)
Create Response (gets file handle FH1)
Close Request
Close Response (closes file handle FH1, file should be deleted)
# date > TMP;
Create Request (asks to create/overwrite file)
Create Response (gets file handle FH2)
  GetInfo Request (SMB2_FILE_INTERNAL_INFO)
  GetInfo Response
  SetInfo Request (SMB2_FILE_ENDOFFILE_INFO)
  SetInfo Response
  Create Request (gets file handle FH3)
  Create Response
    SetInfo Request (SMB2_FILE_BASIC_INFO)
    SetInfo Response
  Close Request
  Close Response (closes file handle FH3)
  Write Request
  Write Response
Close Request
Close Response (closes file handle FH2)
... <repeat>

Now I hit Ctrl-C on the above loop.  In this case, it seems to have interrupted the rm (unlink) command, resulting in:

Create Request (asks for delete-on-close)
Create Response (gets file handle FH4)

but no Close request/response afterwards.  So this unlink is never completed.  Then when I restart the loop above, I see this first, I'm not sure why (cifs_revalidate_dentry_attr, maybe?):

Create Request (open if it exists)
Create Response
  GetInfo Request (SMB2_FILE_ALL_INFO)
  GetInfo Response (succeeds)
Close Request
Close Response

And then the actual rm/unlink:

Create Request (asks for delete-on-close)
Create Response
Close Request
Close Response

after which every operation on the file results in an error STATUS_DELETE_PENDING:

Create Request
Create Response (Error: STATUS_DELETE_PENDING)
<repeat ad infinitum>

So, it seems interrupting an unlink call can cause a file handle to stay open until manually forced at the server or until the client machine is rebooted.

How reproducible:

Pretty consistent

Steps to Reproduce:

1. mount a CIFS file system and cd to a directory under that mount

2. Run the following and hit Ctrl-C during it:
while true; do rm -f TMP ; sleep 0.01s ; date > TMP; done

3. Repeat 2. until you get the error:
bash: TMP: No such file or directory


Actual results:

File is in a stuck state -- server still thinks someone has it open.

Expected results:

File should be released and able to be accessed normally.

Additional info: