This service will be undergoing maintenance at 00:00 UTC, 2016-08-01. It is expected to last about 1 hours
Bug 214684 - Can't specify per-map options
Can't specify per-map options
Status: CLOSED CURRENTRELEASE
Product: Fedora
Classification: Fedora
Component: nfs-utils (Show other bugs)
6
All Linux
medium Severity medium
: ---
: ---
Assigned To: Ian Kent
Ben Levenson
:
Depends On:
Blocks: 238652
  Show dependency treegraph
 
Reported: 2006-11-08 16:03 EST by Joe Sunday
Modified: 2007-11-30 17:11 EST (History)
4 users (show)

See Also:
Fixed In Version: autofs-5.0.1-0.rc3.29
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2007-10-07 21:44:53 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)
Fix tokenizer to distinguish between global option and dn string (1.19 KB, patch)
2006-11-09 02:48 EST, Ian Kent
no flags Details | Diff
Use TCP then UDP when probing mountd (512 bytes, patch)
2006-12-19 02:32 EST, Ian Kent
no flags Details | Diff

  None (edit)
Description Joe Sunday 2006-11-08 16:03:25 EST
Description of problem:
The manpage for auto.master(5) says:
       The format of a master map entry is:

       mount-point [map-type[,format]:]map [options]
(...)
       options
              Any remaining command line arguments without leading dashes  (-)
              are  taken  as  options  (-o)  to mount.  Arguments with leading
              dashes are considered options for the maps.

An entry of the form:
/u      /etc/auto.u   intr,hard,rsize=32768,wsize=32768,nfsvers=3,timeo=600,tcp

Gives me this in /var/log/messages when it fails to start:
Nov  8 15:48:18 sunday-fc6 automount[7698]: Starting automounter version
5.0.1-0.rc2.17, master map auto.master
Nov  8 15:48:18 sunday-fc6 automount[7698]: master_error: syntax error while
parsing map.
Nov  8 15:48:18 sunday-fc6 automount[7698]: master_notify: syntax error in map
near [ bogus option ]
Nov  8 15:48:18 sunday-fc6 automount[7698]: master_read_master: no mounts in table

If I remove the third argument, it works fine (except not passing my options on)
This works in RHEL 3.

Version-Release number of selected component (if applicable):
5.0.1-0.rc2.17

How reproducible:
100%

Expected results:
It should accept the options I've supplied for that map and pass them through to
mount.
Comment 1 Ian Kent 2006-11-08 21:28:05 EST
(In reply to comment #0)
> Expected results:
> It should accept the options I've supplied for that map and pass them through to
> mount.

Indeed it should.
I'll check.

Ian

Comment 2 Ian Kent 2006-11-09 02:48:11 EST
Created attachment 140750 [details]
Fix tokenizer to distinguish between global option and dn string

Please try this patch.
I don't think you'll need to but if it doesn't apply
to 0.rc2.17 get 0.rc2.21 from Rawhide. If 0.rc2.22
appears on in FC updates it is the same as Rawhide
0.rc2.21.
Comment 3 Joe Sunday 2006-11-09 10:20:42 EST
I applied it to 0.rc2.22 that was in FC updates this morning. It starts up now,
but it ignores the options supplied. This is my auto.master:
/u      /etc/auto.u     rsize=32768,wsize=32768,nfsvers=3,timeo=600,tcp

This is a test auto.u:
sunday  -rw,intr filer:/vol/users/sunday

And this is what I see in ps when I try to access /u/sunday:
root      2931  2919  0 10:14 ?        00:00:00 /bin/mount -t nfs -s -o rw,intr
filer:/vol/users/sunday /u/sunday
root      2932  2931  0 10:14 ?        00:00:00 /sbin/mount.nfs
filer:/vol/users/sunday /u/sunday -s -o rw,intr

(I can only do a TCP mount from here, and without mount getting -o tcp it hangs)
Comment 4 Ian Kent 2006-11-09 11:56:26 EST
(In reply to comment #3)
> I applied it to 0.rc2.22 that was in FC updates this morning. It starts up now,
> but it ignores the options supplied. This is my auto.master:
> /u      /etc/auto.u     rsize=32768,wsize=32768,nfsvers=3,timeo=600,tcp
> 
> This is a test auto.u:
> sunday  -rw,intr filer:/vol/users/sunday
> 
> And this is what I see in ps when I try to access /u/sunday:
> root      2931  2919  0 10:14 ?        00:00:00 /bin/mount -t nfs -s -o rw,intr
> filer:/vol/users/sunday /u/sunday
> root      2932  2931  0 10:14 ?        00:00:00 /sbin/mount.nfs
> filer:/vol/users/sunday /u/sunday -s -o rw,intr
> 
> (I can only do a TCP mount from here, and without mount getting -o tcp it hangs)

That is the way it works now.
It has changed.

Now, options specified for a mount point in the map over-ride
the global options as in Solaris.

Is it possible to either add the extra needed options to the
map entry or remove the map entry options?

Ian
Comment 5 Joe Sunday 2006-11-09 13:33:29 EST
The real mount map I need is coming from NIS that I don't control, so I can't
remove the options from there or add to them. Whoever does control that map put
rw,intr on it and it seems to work across all the OS's we use, but we then tweak
the rest on a per-host basis depending on the particular circumstances.

This makes upgrading a lab that depends on the old behavior or adding new
machines into an existing NIS environment difficult at best. Is there a reason
merging the options was removed?

I can work around it with a shell script to mangle the NIS output, but this
seems like a regression that really is going to catch people in large lab
environments.
Comment 6 Ian Kent 2006-11-09 13:45:55 EST
(In reply to comment #5)
> The real mount map I need is coming from NIS that I don't control, so I can't
> remove the options from there or add to them. Whoever does control that map put
> rw,intr on it and it seems to work across all the OS's we use, but we then tweak
> the rest on a per-host basis depending on the particular circumstances.
> 
> This makes upgrading a lab that depends on the old behavior or adding new
> machines into an existing NIS environment difficult at best. Is there a reason
> merging the options was removed?
> 
> I can work around it with a shell script to mangle the NIS output, but this
> seems like a regression that really is going to catch people in large lab
> environments.

This was discussed at length and the decision was to implement
as close to the Solaris behaviour as possible.

However, in your case I'm still a bit puzzled as to the actual
problem? You mentioned TCP at one point but that is the default,
you use nfsver=3 which is also tried first by mount(8). So is
the problem the rsize and wsize, I'm not sure what the default
is for them in FC6. Perhaps they need to change and perhaps the
defaults I mention above aren't happening? They should be as
the original decision was predicated on this.

Ian
Comment 7 Ian Kent 2006-11-09 13:55:43 EST
(In reply to comment #3)
> (I can only do a TCP mount from here, and without mount getting -o tcp it hangs)

Indeed, this deserves more investigation.
I'll have a look but can you check whether this is mount(8)
hanging or automount itself.

To do this look for a mount subprocess, whose parent is
automount, when it hangs. If there is one it's possibly
mount probing UDP before TCP. Otherwise, maybe I need to
smarten up the autofs code in this respect.

Ian
Comment 8 Joe Sunday 2006-11-09 14:01:33 EST
> This was discussed at length and the decision was to implement
> as close to the Solaris behaviour as possible.

Except you already had people depending on the way Linux did things. Solaris
automount is still a different beast, and now while it's close to the way that
works, it's not close to the way it's worked for years on Linux. At least a
option to keep the old behavior as a command line switch or prefixing the option
list with something would have been nice.

> You mentioned TCP at one point but that is the default,
> you use nfsver=3 which is also tried first by mount(8)

nfs(5) disagrees with you on both counts (and the ps output above indicates
automount isn't trying to override those):

Three different versions of the NFS protocol are supported by the Linux
NFS client: NFS version 2, NFS version 3, and NFS version 4.  To  mount
via  NFS version 2, use the nfs file system type and specify nfsvers=2.
Version 2 is the default protocol version for the nfs file system  type
when  nfsvers= is not specified on the mount command.  To mount via NFS
version 3, use the nfs file system  type  and  specify  nfsvers=3.  

       tcp            Mount  the NFS filesystem using the TCP protocol instead
                      of the default UDP protocol.  Many NFS servers only sup-
                      port UDP.

       udp            Mount  the  NFS filesystem using the UDP protocol.  This
                      is the default.
Comment 9 Joe Sunday 2006-11-09 14:03:26 EST
(In reply to comment #7)
> (In reply to comment #3)
> > (I can only do a TCP mount from here, and without mount getting -o tcp it hangs)
> 
> Indeed, this deserves more investigation.
> I'll have a look but can you check whether this is mount(8)
> hanging or automount itself.
> 
> To do this look for a mount subprocess, whose parent is
> automount, when it hangs. If there is one it's possibly
> mount probing UDP before TCP. Otherwise, maybe I need to
> smarten up the autofs code in this respect.

It's mount. There's already a ps output in comment 3 that shows exactly that.

Comment 10 Jeffrey Moyer 2006-11-09 14:16:09 EST
(In reply to comment #8)

> > You mentioned TCP at one point but that is the default,
> > you use nfsver=3 which is also tried first by mount(8)
> 
> nfs(5) disagrees with you on both counts (and the ps output above indicates
> automount isn't trying to override those):

Then the docs need updating.  Mount will try mounts in the order:
version 3 tcp
version 3 udp
version 2 tcp
version 2 udp
Comment 11 Joe Sunday 2006-11-09 14:17:32 EST
(In reply to comment #9)

> It's mount. There's already a ps output in comment 3 that shows exactly that.

Almost exactly that. The pid of automount is 2919 (the box is still running) and
that was the only mount in the ps list at the time.
Comment 12 Ian Kent 2006-11-09 14:33:10 EST
(In reply to comment #11)
> (In reply to comment #9)
> 
> > It's mount. There's already a ps output in comment 3 that shows exactly that.
> 
> Almost exactly that. The pid of automount is 2919 (the box is still running) and
> that was the only mount in the ps list at the time.

It does look like it, yes.

But from comment #10 it looks like we do need to look at
mount's behavior. Perhaps something previous, like a portmap
query, is using UDP before or in addition to TCP. I seem to
remember thinking that there were no calls to the RPC NULL
procedure before some of the queries made by mount. A sure
way to hang an app if the protocol is blocked by a router
or something.

The point here is that this type of hang can affect other
situations not only only yours.

Ian
Comment 13 Joe Sunday 2006-11-09 14:34:03 EST
(In reply to comment #10)

> > nfs(5) disagrees with you on both counts (and the ps output above indicates
> > automount isn't trying to override those):
> 
> Then the docs need updating.  Mount will try mounts in the order:
> version 3 tcp
> version 3 udp
> version 2 tcp
> version 2 udp

Well, then -o tcp is setting something than just tcp:
# mount filer:/vol/users/sunday /u/sunday
mount: mount to NFS server 'filer' failed: timed out (retrying).
^C

# mount filer:/vol/users/sunday /u/sunday -o tcp
#
Comment 14 Ian Kent 2006-11-09 14:39:48 EST
(In reply to comment #13)
> (In reply to comment #10)
> 
> > > nfs(5) disagrees with you on both counts (and the ps output above indicates
> > > automount isn't trying to override those):
> > 
> > Then the docs need updating.  Mount will try mounts in the order:
> > version 3 tcp
> > version 3 udp
> > version 2 tcp
> > version 2 udp
> 
> Well, then -o tcp is setting something than just tcp:
> # mount filer:/vol/users/sunday /u/sunday
> mount: mount to NFS server 'filer' failed: timed out (retrying).
> ^C
> 
> # mount filer:/vol/users/sunday /u/sunday -o tcp
> #

It does heaps of work probing, trying to check server
availability and version etc.

Check the number of open ports after a mount in an
environment that has no restrictings. You'll see about
5 tcp ports in TIME_WAIT and of course that doesn't show
what may have been done with UDP.

Ian
Comment 15 Joe Sunday 2006-11-09 15:02:18 EST
(In reply to comment #14)

> > Well, then -o tcp is setting something than just tcp:
> > # mount filer:/vol/users/sunday /u/sunday
> > mount: mount to NFS server 'filer' failed: timed out (retrying).
> > ^C
> > 
> > # mount filer:/vol/users/sunday /u/sunday -o tcp
> > #
> 
> It does heaps of work probing, trying to check server
> availability and version etc.
> 
> Check the number of open ports after a mount in an
> environment that has no restrictings. You'll see about
> 5 tcp ports in TIME_WAIT and of course that doesn't show
> what may have been done with UDP.

Then the statement that tcp and nfsver=3 is default isn't exactly correct if it
isn't identical behavior to what I get if I explicitly set it. Auto-detect
doesn't do much good if it hangs before it tries the default.

It works if I explicitly set -o tcp, and it hangs otherwise.

FWIW, I just looked at one of our solaris 9 boxes, it has an identical auto.u as
above, and the line in auto_master looks like:
/u              /etc/auto.u             -nobrowse,noquota

Solaris does default correctly to tcp v3, and does it without hanging. Also, the
option names are different across platforms, so we can't put them in NIS. (But
we don't need any on Solaris other than rw,intr)

The end result is still the same, although we're wandering a bit. I used to be
able to set the options we needed to make Linux mount correctly on top of the
ones specified globally in NIS. I can't do that anymore, and the defaults don't
work.
Comment 16 Ian Kent 2006-11-09 22:37:02 EST
(In reply to comment #15)
> 
> Solaris does default correctly to tcp v3, and does it without hanging. Also, the
> option names are different across platforms, so we can't put them in NIS. (But
> we don't need any on Solaris other than rw,intr)
> 
> The end result is still the same, although we're wandering a bit. I used to be
> able to set the options we needed to make Linux mount correctly on top of the
> ones specified globally in NIS. I can't do that anymore, and the defaults don't
> work.

Maybe you missed my point somehow since this is what I've been
saying.

I think mount(8) is not working properly in this case and that
is what autofs uses to perform it's mounts.

And I've suggested I think I may know what the problem is.

I'll investigate further as soon as I get some time.

Ian
Comment 17 Ian Kent 2006-11-09 22:43:11 EST
(In reply to comment #15)
> 
> Solaris does default correctly to tcp v3, and does it without hanging. Also, the
> option names are different across platforms, so we can't put them in NIS. (But
> we don't need any on Solaris other than rw,intr)

That's not the case any more for just about all the mount
options. Again a result of interoperability work.

Ian
Comment 18 Ian Kent 2006-12-19 02:13:06 EST
(In reply to comment #10)
> (In reply to comment #8)
> 
> > > You mentioned TCP at one point but that is the default,
> > > you use nfsver=3 which is also tried first by mount(8)
> > 
> > nfs(5) disagrees with you on both counts (and the ps output above indicates
> > automount isn't trying to override those):
> 
> Then the docs need updating.  Mount will try mounts in the order:
> version 3 tcp
> version 3 udp
> version 2 tcp
> version 2 udp

Yes. I see this in nfsmount.c.

I also see that nfsmount.c probes UDP then TCP when looking
for mountd, oops!

I'd change it but there is probably a historic reason for
doing this, however if we're going to use TCP for the NFS
mount it would make sense to use TCP for the mountd protocol
as well.

Jeff, any history your'e aware of here?

Ian
Comment 19 Ian Kent 2006-12-19 02:32:48 EST
Created attachment 143980 [details]
Use TCP then UDP when probing mountd

Following on from comment #18.

This patch changes nfsmount to use the same protocol
order when probing the mountd service as it uses when
it probes the nfs service.

Could you try this patch for nfs-utils and see if it
resolves the problem you reported.

Ian
Comment 20 Jeffrey Moyer 2006-12-19 08:04:43 EST
(In reply to comment #18)
> (In reply to comment #10)
> > (In reply to comment #8)
> > 
> > > > You mentioned TCP at one point but that is the default,
> > > > you use nfsver=3 which is also tried first by mount(8)
> > > 
> > > nfs(5) disagrees with you on both counts (and the ps output above indicates
> > > automount isn't trying to override those):
> > 
> > Then the docs need updating.  Mount will try mounts in the order:
> > version 3 tcp
> > version 3 udp
> > version 2 tcp
> > version 2 udp
> 
> Yes. I see this in nfsmount.c.
> 
> I also see that nfsmount.c probes UDP then TCP when looking
> for mountd, oops!
> 
> I'd change it but there is probably a historic reason for
> doing this, however if we're going to use TCP for the NFS
> mount it would make sense to use TCP for the mountd protocol
> as well.
> 
> Jeff, any history your'e aware of here?

Not that I know of.  CC-ing steved in hopes that he can tell us.  I think he is
the one who introduced that code in the first place.
Comment 21 Joe Sunday 2007-01-03 10:23:32 EST
(In reply to comment #19)

> Could you try this patch for nfs-utils and see if it
> resolves the problem you reported.

I haven't had a chance to take down my workstation to test this on.

I'll try to check it today. Thanks.
Comment 22 Joe Sunday 2007-01-04 19:34:12 EST
That patch appears to work. It now mounts without hanging.
Comment 23 Ian Kent 2007-02-14 01:09:47 EST
Hi Karl,

Could you check this bug please?

Sorry for the delay passing this on but it looks like
we need to change the default protocol to match the
requested protocol when probing. This can be a problem
when UDP is blocked for some reason, but TCP is not, as
is the case here.

Ian
Comment 25 Steve Dickson 2007-03-06 08:33:56 EST
Sorry for the delayed response....

wrt Comment #18

> I also see that nfsmount.c probes UDP then TCP when 
> looking for mountd, oops!
This is true, UDP probes to mountd are used instead
of TCP to save the reserver port space. 

> any history your'e aware of here?

This history... So we can weather the mounting
storms when when a machine boots and does a ton
of autofs mounts are done, I use UDP probes 
to mountd instead of TCP, so the connection
do not go into TIMEWAIT, which in turn case
an reserver port to be usable for a while.

With this approach, I was able to get over
1000 mounts happening simultaneously... which
is a good thing... imho...

So I'm a bit reluctant to apply Ian's patch
since it would be taking a step backward wrt
to mount storms... 

Unfortunately, this bug has number of different part,
so what exactly is the problem this probing 
approach is causing?
Comment 26 Joe Sunday 2007-03-06 09:56:26 EST
In a nut shell:
I can no longer specify additional options in my auto.master file to override those coming in from NIS 
(which I don't own). One of these that's needed in my environment is 'tcp' as we don't do NFS over UDP 
for various reasons (of which I'm not the responsible party, so changing that isn't an option)

Mount now uses tcp by default anyway, so it doesn't matter there, and I can do the mount by hand 
without specifying 'tcp'. However, you're still probing with UDP, even if mount is going to use TCP. In an 
environment that doesn't allow NFS over UDP, this causes the automounter to hang for 2 minutes and 
then fail the mount.

Alternatively, if I flip it so UDP is allowed and TCP isn't, then your probe will suceed and the mount itself 
will hang.

Therefore, if mount is defaulting to tcp, the automount probe should match. Or, the old behaviour of 
allowing the union of machine-local options in auto.master and NIS options in the mount map itself 
should be restored so I can force TCP on the mounts that need it.
Comment 27 Ian Kent 2007-03-06 10:10:16 EST
(In reply to comment #26)
> Therefore, if mount is defaulting to tcp, the automount probe should match.
Or, the old behaviour of 
> allowing the union of machine-local options in auto.master and NIS options in
the mount map itself 
> should be restored so I can force TCP on the mounts that need it.

Yes, that always did seem like a sensible way to do things but it
makes autofs different from the Solaris behaviour so we would need
to do it differently. Perhaps I should add a command option or
additional configuration option to override this for backward
compatibility. I probably should have considered this before
but I was focused on the new version. Now things have settled
down so let me think about it for a little while.

Ian
Comment 28 Steve Dickson 2007-03-06 10:14:59 EST
Explicitly specifying the -o tcp should stop the UDP
probes... or is this not an option?
Comment 29 Joe Sunday 2007-03-06 10:24:49 EST
Steve,

FC6 changed the way automount works so that I can no longer specify additional options in the local 
auto.master file in addition to those from the NIS mount map (which I don't control). Now local options 
only apply if none are specified in the map itself, and get ignored if any exist in NIS. Therefore I can't 
explicitly specify -o tcp anymore. I could in previous versions.

Regardless, it's bad practice to probe one protocol and then mount using the other.. Your probe is 
worthless, because you're not actually probing for the export mount is going to use anyway. Any sort of 
environment where one proto is allowed and one isn't is going to have issues with the mismatch.

If you really want to do udp probes by default, then mount should default to udp as well.
Comment 30 Steve Dickson 2007-03-06 11:04:20 EST
> Therefore I can't explicitly specify -o tcp anymore. I 
> could in previous versions.
Ah... didn't realize this... And this is a feature?? ;-) 

> Your probe is worthless, because you're not actually probing for 
> the export mount is going to use anyway. 
Well, its not worthless if the remote mountd or portmapper or
the actual nfs server is down... There are very long (unchangeable)
timeouts in the glibc code that cause unreasonable hangs
when those daemons are down... 

> Any sort of environment where one proto is allowed and 
> one isn't is going to have issues with the mismatch.
True.. thats why -o tcp does not do UDP probes.
I guess I never image that ability would be taken away...
Comment 31 Ian Kent 2007-03-06 11:45:51 EST
(In reply to comment #30)
> > Therefore I can't explicitly specify -o tcp anymore. I 
> > could in previous versions.
> Ah... didn't realize this... And this is a feature?? ;-) 

Not really, it was decided upon as a requirement, Solaris
like behaviour whereever possible, for the new version but
I didn't add anything for backward compatibility for this
particular setting. In hindsight, my bad.

Ian
Comment 32 Steve Dickson 2007-03-12 13:16:05 EDT
So where are we going with this bug... 

Does it make sense for -o tcp mount to still do
udp probs (which would make bz 212269 happy) or
are -o tcp going to be reenabled in the near 
future in autofs land? or both?
Comment 33 Ian Kent 2007-03-12 13:36:07 EDT
(In reply to comment #32)
> So where are we going with this bug... 
> 
> Does it make sense for -o tcp mount to still do
> udp probs (which would make bz 212269 happy) or
> are -o tcp going to be reenabled in the near 
> future in autofs land? or both?

I haven't got to it yet but I'm going to add a config
option to allow customers to revert to the original
mount option behaviour of accumulating options. In hind
sight I should have at least provided that for backward
compatibility from the beginning.

Ian



Comment 34 Ian Kent 2007-04-19 03:54:48 EDT
(In reply to comment #33)
> I haven't got to it yet but I'm going to add a config
> option to allow customers to revert to the original
> mount option behaviour of accumulating options. In hind
> sight I should have at least provided that for backward
> compatibility from the beginning.

autofs-5.0.1-0.rc3.29 is present in updates/testing.
This update provides the ability to append the options
given in the master map (instead of overriding them).
In addition a command line option (-O <option string>)
has been added to allow the specification of a global
options string so that multi-vendor sites can avoid
adding incompatible options to centraly administered
maps.

Please try this package update out and let me know if
it resolves this problem.

Ian
Comment 35 Joe Sunday 2007-05-01 13:56:22 EDT
That update seems to resolve my issue. Mount options are merged from the central NIS map as well as the 
auto.master file.
Comment 36 Ian Kent 2007-10-07 21:44:53 EDT
As per comment #35 this has been fixed some time ago.

Note You need to log in before you can comment on or make changes to this bug.