Red Hat Bugzilla – Bug 505535
mount of from UDP only server fails
Last modified: 2009-10-27 03:18:34 EDT
Description of problem:
I just preupgrade'd from F10 to F11. Automounter wasn't able to mount all the necessary exports after the upgrade. When I tried one of the failing ones by hand with mount I got the error message:
# mount -t nfs server:/path/to/export /mnt
mount.nfs: an incorrect mount option was specified
After some head-scratching I figured out that this server was UDP only so this works:
# mount -t nfs -o proto=udp server:/path/to/export /mnt
It seems like that mount.nfs no longer falls back to UDP if TCP fails.
Version-Release number of selected component (if applicable):
I believe this problems is fixed by nfs-utils-1.2.0-3 which
is currently being tested.
If your are interested in helping with this testing please uses
the following to download the new package, then install the
package via the 'rpm -Uhv <nfs-utils>' command...
Finally then please record the results...
That is the version I'm using and it doesn't work (see above). That was the first thing I tried :-)
yeah I see that...
What is the output of 'rpcinfo -p <server>'?
Our IT guys have "fixed" the server now to allow TCP too. So I guess this will not be helpful:
# rpcinfo -p <server>
program vers proto port service
100003 3 tcp 2049 nfs
100003 2 tcp 2049 nfs
100024 1 tcp 4047 status
100024 1 udp 4047 status
100011 1 udp 4049 rquotad
100021 4 tcp 4045 nlockmgr
100021 3 tcp 4045 nlockmgr
100021 1 tcp 4045 nlockmgr
100021 4 udp 4045 nlockmgr
100021 3 udp 4045 nlockmgr
100021 1 udp 4045 nlockmgr
100005 3 tcp 4046 mountd
100005 2 tcp 4046 mountd
100005 1 tcp 4046 mountd
100005 3 udp 4046 mountd
100005 2 udp 4046 mountd
100005 1 udp 4046 mountd
100003 3 udp 2049 nfs
100003 2 udp 2049 nfs
100000 2 tcp 111 portmapper
100000 2 udp 111 portmapper
Well I guess that a good and bad thing... Its a good thing because you
really really really want to using TCP over UPD but it eliminated
the configuration what duplicated the bug...
In my testing I set RPCNFSDARGS="--no-tcp" in /etc/sysconfig/nfs
and then started the server... The mount from the client (w/out the
proto=udp) worked... So I wonder what the difference is in my
configuration and in yours...
Could please ping your IT guy and find out what they did to make
everything listen on UDP only?
Please note: https://bugzilla.redhat.com/show_bug.cgi?id=505018
It seems with that bz the person did an 'yum update' and the
problem went away... just an FYI...
I think I see the problem...
On any server turn off all tcp traffic to nfs, rpc.mountd
and portmapper/rpcbind by setting the following two variables
RPCNFSDARGS="-N 4 --no-tcp"
and then turn off tcp traffice to portmapper/rpcbind with:
iptables -A INPUT -p tcp -i eth0 --dport 111 -j REJECT
Then from an F-11 client try doing a NFS mount. The mount
will hang unless the 'proto=udp' option is set.
I will take a look at this.
I can reproduce this just by blocking the portmapper's UDP ports. There isn't any way that mount transport/version negotiation can work if the portmapper can't be contacted from the client. And I don't see any logic in user space or the kernel that will retry portmapper requests on a different transport.
Created attachment 347607 [details]
NFS: fix return code of nfs_mount when rpcbind fails
This fixes the renegotiation for me on 2.6.30-rc8 with text-based mounts. There is still a long delay before the mount succeeds, to be addressed elsewhere.
Just FYI: the server in question was a NetApps NFS filer. The configuration option to "fix" it was
options nfs.tcp.enable on
That option disables/enables *all* TCP NFS traffic completely.
On the client side it doesn't matter if a firewall was active or not.
(In reply to comment #5)
> Could please ping your IT guy and find out what they did to make
> everything listen on UDP only?
Just being a Red Hat Linux 9 NFS server is sufficient. (don't ask... :-))
Anyway... mounts did work fine up until at least Fedora 8 (with all updates applied), and I think even up to Fedora 10 (vanilla) - but not sure. Something was modified which breaks UDP-only detection (which was without long delay too).
I suggest following up on this bug in either #505018 _or_ here, but at the moment both bugs are just duplicates of each other.
Correction: 505018 might be something different, at least for the original poster.
(In reply to comment #11)
> Just FYI: the server in question was a NetApps NFS filer. The configuration
> option to "fix" it was
> options nfs.tcp.enable on
> That option disables/enables *all* TCP NFS traffic completely.
> On the client side it doesn't matter if a firewall was active or not.
I was able to try this with a filer (DOT 7.3.1, I think) last week, and it didn't seem to affect the rpcbind transport settings. I wonder if there were other services running on that filer that forced rpcbind to continue to listen on TCP.
It's worth mentioning that you have to disable nfsv4 on the filer as well, since it requires NFS/TCP support.
Created attachment 349124 [details]
roll-up of changes to address failure to mount UDP-only server
This attachment is a set of patches rolled into a single patch that attempts to address mount.nfs failures when mounting a UDP-only server. The patch is against upstream nfs-utils-1.2.0, but will probably apply to Fedora 11's nfs-utils.
This patch attempts to make text-based NFS mounts behave more like legacy NFS mounts by trying the rpcbind queries first before sending the request to the kernel. This avoids a number of kernel issues that will need to be addressed over the long term in the kernel.
There are likely still some rough edges, but I'd be interested if others would try this out.
nfs-utils-1.2.0-6.fc11 has been submitted as an update for Fedora 11.
nfs-utils-1.2.0-6.fc11 has been pushed to the Fedora 11 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
su -c 'yum --enablerepo=updates-testing update nfs-utils'. You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F11/FEDORA-2009-10643
nfs-utils-1.2.0-6.fc11 has been pushed to the Fedora 11 stable repository. If problems still persist, please make note of it in this bug report.