Red Hat Bugzilla – Bug 212398
NFS mounts freeze
Last modified: 2007-11-16 20:14:54 EST
From Bugzilla Helper:
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; InfoPath.1)
Description of problem:
We have a RHEL client that is mounting some shares from a NAS device via NFS (version 3). The client has been running various flavors of RHEL 3 for a couple of years now without issue. We recently upgraded to RHEL4 (u4) with all applicable patches and now we see the NFS mount freeze up after a period of time (usually overnight). If we kill the processes trying to access the mounts, unmount, and remount the shares, everything is fine until the next occurence. The only error message we see in /var/log/messages is:
kernel: RPC: error 512 connecting to server xxx.xxx.xxx.xxx
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1.Mount an NFS share using the default options plus intr.
2.Have some process access the share over a period of time.
After a period of time, data on the mounts cannot be accessed.
This seems similar to a problem that was discussed in the NFSv4 mailing list even though we are not trying to use NFSv4. The discussion referenced bugzilla bug 194793 but I was unable to get access to that particular bug id when I logged in to bugzilla.
Also, I am running the EL.SMP version of the kernel (2 CPUS).
Our remaining clients which are still running RHEL3 (update 6) are not experiencing this issue even though they are mounting the same shares and running the same processes.
This sort of thing works for me. I do it all of the time.
Perhaps a stacktrace of all of the processes in the system, when
the hangs occurs could be attached, please?
Same Problem here. A long tcpdump reveiled outgoing TCP Connects from NFS Client
Port 664, but no answers from the server.
Research pointed to an IPMI Card see http://lkml.org/lkml/2006/10/10/239
You can fix this using sysctl
root@host:~> sysctl -w sunrpc.min_resvport=665
to make it permanent add the following line to /etc/sysctl.conf
sunrpc.min_resvport = 665
Changing the default setting for xprt_min_resvport does seem like a good
thing to do. Changing the default would also match the RHEL-5 kernel as
well as upstream.
Created attachment 143922 [details]
Mike, can you tell if this is the problem that you are seeing?
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release. Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products. This request is not yet committed for inclusion in an Update
(In reply to comment #2)
> Same Problem here. A long tcpdump reveiled outgoing TCP Connects from NFS Client
> Port 664, but no answers from the server.
> Research pointed to an IPMI Card see http://lkml.org/lkml/2006/10/10/239
The IPMI Board swallows all packages directed to Port 664 (used by ipmi for RPC
calls). All Server answers will be hidden from the Client OS. Problem hast been
fixed in linux Kernels starting with 2.6.19 by raising the minimum RPC port used
by NFS to Port 665.
QE ack for RHEL4.5.
Posted to rhkernel-list.
committed in stream U5 build 45. A test kernel with this patch is available from
looks like the fix for this one is in -48, there is nothing in sysctl.conf on
this system that would be setting min_resvport.
[root@dhcp58-247 sunrpc]# cat min_resvport
[root@dhcp58-247 sunrpc]# cat max_resvport
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.