Bug 505535 - mount of from UDP only server fails
mount of from UDP only server fails
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: nfs-utils (Show other bugs)
11
All Linux
low Severity medium
: ---
: ---
Assigned To: Steve Dickson
Fedora Extras Quality Assurance
:
Depends On:
Blocks: 516998
  Show dependency treegraph
 
Reported: 2009-06-12 05:47 EDT by Stefan Becker
Modified: 2009-10-27 03:18 EDT (History)
3 users (show)

See Also:
Fixed In Version: 1.2.0-6.fc11
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2009-10-27 03:18:34 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)
NFS: fix return code of nfs_mount when rpcbind fails (946 bytes, patch)
2009-06-12 12:29 EDT, Chuck Lever
no flags Details | Diff
roll-up of changes to address failure to mount UDP-only server (34.73 KB, patch)
2009-06-23 13:57 EDT, Chuck Lever
no flags Details | Diff

  None (edit)
Description Stefan Becker 2009-06-12 05:47:26 EDT
Description of problem:

I just preupgrade'd from F10 to F11. Automounter wasn't able to mount all the necessary exports after the upgrade. When I tried one of the failing ones by hand with mount I got the error message:

  # mount -t nfs server:/path/to/export /mnt
mount.nfs: an incorrect mount option was specified

After some head-scratching I figured out that this server was UDP only so this works:

  # mount -t nfs -o proto=udp server:/path/to/export /mnt

It seems like that mount.nfs no longer falls back to UDP if TCP fails.

Version-Release number of selected component (if applicable):
kernel-PAE-2.6.29.4-167.fc11.i686
nfs-utils-1.2.0-3.fc11.i586

How reproducible:
always
Comment 1 Steve Dickson 2009-06-12 08:58:14 EDT
I believe this problems is fixed by nfs-utils-1.2.0-3 which
is currently being tested. 

If your are interested in helping with this testing please uses 
the following to download the new package, then install the 
package via the 'rpm -Uhv <nfs-utils>' command... 

https://admin.fedoraproject.org/updates/nfs-utils-1.2.0-3.fc11

Finally then please record the results...
Comment 2 Stefan Becker 2009-06-12 09:05:24 EDT
That is the version I'm using and it doesn't work (see above). That was the first thing I tried :-)
Comment 3 Steve Dickson 2009-06-12 09:42:39 EDT
yeah I see that... 

What is the output of 'rpcinfo -p <server>'?
Comment 4 Stefan Becker 2009-06-12 09:47:52 EDT
Our IT guys have "fixed" the server now to allow TCP too. So I guess this will not be helpful:

# rpcinfo -p <server>
   program vers proto   port  service
    100003    3   tcp   2049  nfs
    100003    2   tcp   2049  nfs
    100024    1   tcp   4047  status
    100024    1   udp   4047  status
    100011    1   udp   4049  rquotad
    100021    4   tcp   4045  nlockmgr
    100021    3   tcp   4045  nlockmgr
    100021    1   tcp   4045  nlockmgr
    100021    4   udp   4045  nlockmgr
    100021    3   udp   4045  nlockmgr
    100021    1   udp   4045  nlockmgr
    100005    3   tcp   4046  mountd
    100005    2   tcp   4046  mountd
    100005    1   tcp   4046  mountd
    100005    3   udp   4046  mountd
    100005    2   udp   4046  mountd
    100005    1   udp   4046  mountd
    100003    3   udp   2049  nfs
    100003    2   udp   2049  nfs
    100000    2   tcp    111  portmapper
    100000    2   udp    111  portmapper
Comment 5 Steve Dickson 2009-06-12 10:01:05 EDT
Well I guess that a good and bad thing... Its a good thing because you
really really really want to using TCP over UPD but it eliminated
the configuration what duplicated the bug... 

In my testing I set  RPCNFSDARGS="--no-tcp" in /etc/sysconfig/nfs
and then started the server... The mount from the client (w/out the 
proto=udp) worked... So I wonder what the difference is in my
configuration and in yours... 

Could please ping your IT guy and find out what they did to make
everything listen on UDP only?
Comment 6 Steve Dickson 2009-06-12 10:08:53 EDT
Please note: https://bugzilla.redhat.com/show_bug.cgi?id=505018

It seems with that bz the person did an 'yum update' and the
problem went away... just an FYI...
Comment 7 Steve Dickson 2009-06-12 10:38:23 EDT
I think I see the problem... 

On any server turn off all tcp traffic to nfs, rpc.mountd
and portmapper/rpcbind by setting the following two variables
in /etc/sysconfig/nfs:

    RPCNFSDARGS="-N 4 --no-tcp"
    RPCMOUNTDOPTS="--no-tcp"

and then turn off tcp traffice to portmapper/rpcbind with:

iptables -A INPUT -p tcp -i eth0 --dport 111 -j REJECT

Then from an F-11 client try doing a NFS mount. The mount
will hang unless the 'proto=udp' option is set.
Comment 8 Chuck Lever 2009-06-12 11:01:12 EDT
I will take a look at this.
Comment 9 Chuck Lever 2009-06-12 11:54:22 EDT
I can reproduce this just by blocking the portmapper's UDP ports.  There isn't any way that mount transport/version negotiation can work if the portmapper can't be contacted from the client.  And I don't see any logic in user space or the kernel that will retry portmapper requests on a different transport.
Comment 10 Chuck Lever 2009-06-12 12:29:59 EDT
Created attachment 347607 [details]
NFS: fix return code of nfs_mount when rpcbind fails

This fixes the renegotiation for me on 2.6.30-rc8 with text-based mounts.  There is still a long delay before the mount succeeds, to be addressed elsewhere.
Comment 11 Stefan Becker 2009-06-15 04:20:00 EDT
Just FYI: the server in question was a NetApps NFS filer. The configuration option to "fix" it was 

    options nfs.tcp.enable on

That option disables/enables *all* TCP NFS traffic completely.

On the client side it doesn't matter if a firewall was active or not.
Comment 12 Daniel Roesen 2009-06-16 05:50:49 EDT
(In reply to comment #5)
> Could please ping your IT guy and find out what they did to make
> everything listen on UDP only?  

Just being a Red Hat Linux 9 NFS server is sufficient. (don't ask... :-))

Anyway... mounts did work fine up until at least Fedora 8 (with all updates applied), and I think even up to Fedora 10 (vanilla) - but not sure. Something was modified which breaks UDP-only detection (which was without long delay too).

I suggest following up on this bug in either #505018 _or_ here, but at the moment both bugs are just duplicates of each other.
Comment 13 Daniel Roesen 2009-06-16 05:51:42 EDT
Correction: 505018 might be something different, at least for the original poster.
Comment 14 Chuck Lever 2009-06-23 13:38:56 EDT
(In reply to comment #11)
> Just FYI: the server in question was a NetApps NFS filer. The configuration
> option to "fix" it was 
> 
>     options nfs.tcp.enable on
> 
> That option disables/enables *all* TCP NFS traffic completely.
> 
> On the client side it doesn't matter if a firewall was active or not.  

I was able to try this with a filer (DOT 7.3.1, I think) last week, and it didn't seem to affect the rpcbind transport settings.  I wonder if there were other services running on that filer that forced rpcbind to continue to listen on TCP.

It's worth mentioning that you have to disable nfsv4 on the filer as well, since it requires NFS/TCP support.
Comment 15 Chuck Lever 2009-06-23 13:57:11 EDT
Created attachment 349124 [details]
roll-up of changes to address failure to mount UDP-only server

This attachment is a set of patches rolled into a single patch that attempts to address mount.nfs failures when mounting a UDP-only server.  The patch is against upstream nfs-utils-1.2.0, but will probably apply to Fedora 11's nfs-utils.

This patch attempts to make text-based NFS mounts behave more like legacy NFS mounts by trying the rpcbind queries first before sending the request to the kernel.  This avoids a number of kernel issues that will need to be addressed over the long term in the kernel.

There are likely still some rough edges, but I'd be interested if others would try this out.
Comment 16 Fedora Update System 2009-10-19 13:25:24 EDT
nfs-utils-1.2.0-6.fc11 has been submitted as an update for Fedora 11.
http://admin.fedoraproject.org/updates/nfs-utils-1.2.0-6.fc11
Comment 17 Fedora Update System 2009-10-20 20:46:05 EDT
nfs-utils-1.2.0-6.fc11 has been pushed to the Fedora 11 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update nfs-utils'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F11/FEDORA-2009-10643
Comment 18 Fedora Update System 2009-10-27 03:18:05 EDT
nfs-utils-1.2.0-6.fc11 has been pushed to the Fedora 11 stable repository.  If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.