Bug 815436 - NFS client hangs sporadically
NFS client hangs sporadically
Status: CLOSED INSUFFICIENT_DATA
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: kernel (Show other bugs)
6.2
x86_64 Linux
unspecified Severity medium
: rc
: ---
Assigned To: Jeff Layton
Red Hat Kernel QE team
NFS server NFSD
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-04-23 11:33 EDT by Mark Nipper
Modified: 2013-01-08 07:51 EST (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-01-08 07:51:18 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
sysrq-trigger output (51.38 KB, application/gzip)
2012-04-23 11:33 EDT, Mark Nipper
no flags Details
rpc_debug output (2.51 KB, application/gzip)
2012-04-23 11:34 EDT, Mark Nipper
no flags Details

  None (edit)
Description Mark Nipper 2012-04-23 11:33:52 EDT
Created attachment 579609 [details]
sysrq-trigger output

Description of problem:
We have a RHEL 6.2 client mounting a RHEL 5.8 server.  Only mounting a couple of fairly idle mounts from the server, but the client is hanging on those mounts and load shoots up as more processes are stuck waiting for it to come back.  This appears to be happening with the NFSv3 client which is somewhat unexpected.  Rebooting the client fixes everything temporarily for a few days usually.

$ cat /proc/fs/nfsfs/servers 
NV SERVER   PORT USE HOSTNAME
v3 805330bc  801   2 file.laits.utexas.edu
$ mount | grep file.laits
file.laits.utexas.edu:/mnt/lvol1/home on /mnt/home type nfs (rw,nosuid,addr=128.83.48.188)
file.laits.utexas.edu:/mnt/media on /mnt/media type nfs (ro,noexec,nosuid,addr=128.83.48.188)
$ w | grep load
 10:32:12 up 17:37,  2 users,  load average: 299.13, 294.86, 286.31

Version-Release number of selected component (if applicable):
Client is running 2.6.32-220.7.1.el6.x86_64 with nfs-utils-1.2.3-15.el6.x86_64.

How reproducible:
It only seems to take a few days usually from the last reboot to get into this state.  But I'm not sure what exactly is triggering it.

Steps to Reproduce:
1. hard to say exactly
  
Actual results:
NFS client hangs.

Expected results:
NFS client shouldn't hang.

Additional info:
I'm attaching the output from "echo t > /proc/sysrq-trigger" and "echo 0 > /proc/sys/sunrpc/rpc_debug".
Comment 1 Mark Nipper 2012-04-23 11:34:23 EDT
Created attachment 579610 [details]
rpc_debug output
Comment 3 RHEL Product and Program Management 2012-05-03 01:23:46 EDT
Since RHEL 6.3 External Beta has begun, and this bug remains
unresolved, it has been rejected as it is not proposed as
exception or blocker.

Red Hat invites you to ask your support representative to
propose this request, if appropriate and relevant, in the
next release of Red Hat Enterprise Linux.
Comment 4 J. Bruce Fields 2012-05-21 16:58:23 EDT
Next time there's a hang:
- Is there any network traffic between the client and server?  (Check this with something like "wireshark -Slk -f'host <yourclient> && host file.laits.utexas.edu'".  If there is something going on, stop the capture and save it (in the default "libpcap" format), and we'd like to see it.)
- What state are the server threads in?  (You could log into the server and do the same sort of syrq-trigger dump.)
- Are all the server daemons running?  (mountd, statd, lockd).
Comment 5 Tore H. Larsen 2012-07-27 22:22:54 EDT
cc
Comment 6 Jeff Layton 2013-01-08 07:51:18 EST
No response to Bruce's query in quite some time. I'm going to go ahead and close this with a resolution of INSUFFICIENT_DATA. Please reopen if this is still reproducible.

Note You need to log in before you can comment on or make changes to this bug.