From Bugzilla Helper: User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.6) Gecko/20050317 Firefox/1.0.2 Description of problem: Certain workloads that remove lots of files run slowly on Linux clients using NFSv4 and with delegation enabled on the server. By examining a network trace, we saw that the Linux NFSv4 client does not send a DELEG_RETURN before deleting a file when there is an outstanding delegation on that file. Instead, the server delays the remove operation, does a call back, and the client then does the DELEG_RETURN. The client redrives the REMOVE request a second later. Customers using NFSv4 against an NFSv4 server with delegation enabled will see terrible performance on workloads that remove lots of files after reading or writing to them. Version-Release number of selected component (if applicable): How reproducible: Always Steps to Reproduce: 1. Enable delegation on an NFSv4 server 2. Mount one of the server's exports from Linux client using NFSv4 3. Try a kernel build on that share Actual Results: Every on-the-wire remove operation results in an NFS4ERR_DELAY and a server call back. The client takes a second to recover and redrive the remove operation. Expected Results: The client should do a DELEG_RETURN before removing a file that has been granted a delegation. Additional info: There is a policy question about just when the client should return a delegation -- just after closing the file, or should it wait as long as possible (until the file's cached pages are reclaimed or the file itself is deleted by the client). In addition we don't want to add a second round trip by doing the DELEG_RETURN in a separate compound, so some cleverness will be needed to make nfs4_proc_remove check if it needs to do the return and build the DELEG_RETURN op into the same compound as the REMOVE. Trond knows about this issue and is working on a solution for 2.6 kernels.
This is a problem for Fedora Core kernels as well.
Created attachment 143462 [details] Return NFSv4 delegation during file removal
Created attachment 143464 [details] Return NFSv4 delegation during silly rename
Created attachment 143467 [details] Return NFSv4 delegation during rename
Attached three patches from mainline that implement delegation return during file removal.
This request was evaluated by Red Hat Product Management for inclusion in a Red Hat Enterprise Linux maintenance release. Product Management has requested further review of this request by Red Hat Engineering, for potential inclusion in a Red Hat Enterprise Linux Update release for currently deployed products. This request is not yet committed for inclusion in an Update release.
QE ack for 4.5.
committed in stream U5 build 42.40. A test kernel with this patch is available from http://people.redhat.com/~jbaron/rhel4/
An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on the solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHBA-2007-0304.html