Bug 358091 - automount accesses udp and portmap even in NFS4 environment
automount accesses udp and portmap even in NFS4 environment
Status: CLOSED WONTFIX
Product: Fedora
Classification: Fedora
Component: autofs (Show other bugs)
7
i386 Linux
low Severity low
: ---
: ---
Assigned To: Ian Kent
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2007-10-30 08:23 EDT by Jukka Lehtonen
Modified: 2008-06-16 22:45 EDT (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2008-06-16 22:45:54 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)
debug log (30.01 KB, text/plain)
2007-10-31 09:30 EDT, Jukka Lehtonen
no flags Details

  None (edit)
Description Jukka Lehtonen 2007-10-30 08:23:07 EDT
Description of problem:
autofs (automount) seems to depend on nfs3 and udp.  During installation of NFS
server, (CentOS 5, x86_64) I did enable only the NFSv4.  Its pseudofilesystem
is rooted (fsid=0) at /foo/bar.  Some directories (/foo/bar/RPMS) are in the
same volume, others (/foo/bar/home) bind-mounted from another volume and
explicitly exported.

Clients (all Fedora 7) use autofs with maps in LDAP and thus mount
fstype=nfs4, and explicitly via tcp.  What they ask from server is of form
'server:/RPMS'.

To make sure no client mounts with nfs3, /etc/hosts.allow prevents all but
localhost from accessing the portmap service.  And not trusting that,
iptables was set to DROP access to portmapper's port, and UDP/2049.

Manual mount was fine, but autofs had timeouts.  It turned out that automount
does access udp/2049 on the server and that port must be open (although not
listened on), and that tcp/111 (portmap) has to REJECT, in order to avoid
timeouts.

So problem (1):
automount does attempt to access tcp/portmap and udp/nfs although mount options
instruct to use nfs4 (ie no portmap directly) and tcp for nfs.

Fedora installation offers NFS-option.  However, it attempts only NFS3 mount,
so NFS4-server won't do.  That is a separate problem.  However, it inspired to
enable NFS3 temporarily:

Disabled /etc/hosts.allow restriction and allowed traffic to tcp/111.  NFS3 now
worked for F7 installations.

But, problem (2):
- nfs4 clients now produce on the server logs:
mountd: refused unmount request from client.domain for /RPMS (/): not exported

Mount of '/RPMS' from server is fine, files on /foo/bar/RPMS are shown.
Unmount succeeds as well, but server logs an error, which must be result
of client automount talking to server portmap, which should not occur on
nfs v4 dialogue.


Version-Release number of selected component (if applicable):
client F7: autofs-5.0.1-27.i386
server CentOS5: kernel-2.6.18-8.1.14.el5.x86_64

How reproducible:
Every time.

Steps to Reproduce:
1. Export subdirectory as fsid=0
2. Use autofs to mount nfs4 via tcp
3.
  
Actual results:
error message and traffic via ports unrelated to nfs4

Expected results:
all client-server traffic via tcp/2049
Comment 1 Ian Kent 2007-10-30 10:57:46 EDT
(In reply to comment #0)
> Description of problem:
> autofs (automount) seems to depend on nfs3 and udp.  During installation of NFS
> server, (CentOS 5, x86_64) I did enable only the NFSv4.  Its pseudofilesystem
> is rooted (fsid=0) at /foo/bar.  Some directories (/foo/bar/RPMS) are in the
> same volume, others (/foo/bar/home) bind-mounted from another volume and
> explicitly exported.

I'm not entirely clear on your problem, so we may need to
go further with the description.

But see below.

> 
> So problem (1):
> automount does attempt to access tcp/portmap and udp/nfs although mount options
> instruct to use nfs4 (ie no portmap directly) and tcp for nfs.

I'm not sure what you mean by this.
NFS4 can use portmap like any other NFS version.
The NFS4 requirement is that it "should" be able to work
without consulting portmap.

Since there is no way for autofs to know if the NFS4 server
is listening on the standard NFS port automount must consult
the portmap service and does so unless the port=<number>
options is given in the mount entry. This option will also
prevent the mount from being bind mounted.

I would need to check to see what mount(8) does in this
case, it's been a while since this issue came up. I do
seem to remember that, unless the proto=<protocol> mount
option is specified, it will also probe portmap, but I'm
not sure about that.

> 
> Fedora installation offers NFS-option.  However, it attempts only NFS3 mount,
> so NFS4-server won't do.  That is a separate problem.  However, it inspired to
> enable NFS3 temporarily:
> 
> Disabled /etc/hosts.allow restriction and allowed traffic to tcp/111.  NFS3 now
> worked for F7 installations.
> 
> But, problem (2):
> - nfs4 clients now produce on the server logs:
> mountd: refused unmount request from client.domain for /RPMS (/): not exported
> 
> Mount of '/RPMS' from server is fine, files on /foo/bar/RPMS are shown.
> Unmount succeeds as well, but server logs an error, which must be result
> of client automount talking to server portmap, which should not occur on
> nfs v4 dialogue.

automount doesn't consult portmap during the umount
operation but mount may be doing so. The problem here
may the result of autofs using incorrect path information
when the umount is done, I can't tell from the description
above.

To analyze this situation, from the viewpoint of autofs,
we would need an example of the autofs maps you're using,
an autofs debug log of the problem happening, and an
example of the exports from the server.

See http://people.redhat.com/jmoyer for more information
about what we need.

Ian
Comment 2 Jukka Lehtonen 2007-10-31 09:30:42 EDT
Created attachment 244551 [details]
debug log
Comment 3 Jukka Lehtonen 2007-10-31 09:56:24 EDT
OK, to rule out things, repeated with local maps.

Client (Fedora7):
d# rpm -q autofs
autofs-5.0.1-27

d# uname -r
2.6.22.7-85.fc7

d# grep -v "#" /etc/auto.master
/misc   /etc/auto.misc --timeout=30 --debug

d# grep -v "#" /etc/auto.misc
RPMS -fstype=nfs4,ro,hard,intr,nodev,nosuid,proto=tcp fs2:/RPMS

d# grep auto /etc/nsswitch.conf
automount:  files

d# grep -v "#" /etc/sysconfig/autofs
TIMEOUT=300
BROWSE_MODE="no"


Server (CentOS 5):
fs2# uname -r
2.6.18-8.1.14.el5

fs2# cat /etc/exports
/local/export          
*(rw,fsid=0,insecure,no_subtree_check,sync,anonuid=65534,anongid=65534)

fs2# /var/log/messages
-----
Oct 31 13:31:48 fs2 mountd[2452]: refused unmount request from d.foo for /RPMS
(/): not exported
Oct 31 13:35:34 fs2 mountd[2452]: refused unmount request from d.foo for /RPMS
(/): not exported
Oct 31 13:38:08 fs2 mountd[2452]: refused unmount request from d.foo for /RPMS
(/): not exported
Oct 31 13:41:57 fs2 -- MARK --
Oct 31 13:46:57 fs2 -- MARK --
Oct 31 13:51:57 fs2 -- MARK --
-----


Repeated five times:
- autofs start
- ls /misc/RPMS
- wait for unmount
- autofs stop

The produced debug is in my previous comment.
The five attempts differed on the server iptables setting.

1. around 13:31
ACCEPT     udp dpt:2049 
ACCEPT     tcp dpt:111
-produces warning on server
-10 packets to tcp/111, 1 packet to udp/2049

2. around 13:35
REJECT     udp dpt:2049 
ACCEPT     tcp dpt:111
-produces warning on server
-10 packets to tcp/111, 1 packet to udp/2049

3. around 13:38
DROP       udp dpt:2049 
ACCEPT     tcp dpt:111
-produces warning on server
-10 packets to tcp/111, 1 packet to udp/2049

4. around 13:41
DROP       udp dpt:2049 
REJECT     tcp dpt:111 reject-with icmp-port-unreachable 
-3 packets to tcp/111, 1 packet to udp/2049

5. around 13:44
DROP       udp dpt:2049 
DROP       tcp dpt:111
-14 packets to tcp/111, 1 packet to udp/2049
-timeout of about 6 minutes on unmount, as noticeable from the debug


'ls /misc/RPMS', ie mount has noticeable (second of two) delay, if udp/2049 is
not ACCEPT on server

On server:
- On mount, one new packet arrives to both tcp/111 and udp/2049.
- On unmount, only the tcp/111 receives packets (two in state NEW), except if
tcp/111 is DROP, when client sends 13 packets.


Thus, there are IMO issues:
1. On mount, udp/2049 is accessed even if mount will use tcp.  Minor delay, ie
not a real problem.
2. On unmount, tcp/111 that DROPs packets causes untolerable timeout.
3. On unmount, if server does listen to tcp/111, it will receive unmount request
for "non-exported path".  (The scenario that I have not tested is to export '/'
as fsid=0, where both nfs3 and nfs4 clients will request same path from the server.
Comment 4 Ian Kent 2007-10-31 10:58:20 EDT
(In reply to comment #3)
> OK, to rule out things, repeated with local maps.

Fine structured testing you've done here.
That's great.

> 
> Client (Fedora7):
> d# rpm -q autofs
> autofs-5.0.1-27
> 
> d# uname -r
> 2.6.22.7-85.fc7
> 
> d# grep -v "#" /etc/auto.master
> /misc   /etc/auto.misc --timeout=30 --debug
> 
> d# grep -v "#" /etc/auto.misc
> RPMS -fstype=nfs4,ro,hard,intr,nodev,nosuid,proto=tcp fs2:/RPMS

I'm not sure what we can do about this really because,
as I mentioned previously, if port=<the nfs4 server port>
isn't given as an option then autofs will ask portmap
what port the service is on in order to send ping to
the NULL procedure. 

But, now that I think about it, this might not end up
being a problem. I have another request to not probe
the server when the mount entry isn't multi-host
mount, like these. So lets deal with the other bits
and I'll port that patch and see how it goes for you.

> 
> d# grep auto /etc/nsswitch.conf
> automount:  files
> 
> d# grep -v "#" /etc/sysconfig/autofs
> TIMEOUT=300
> BROWSE_MODE="no"
> 
> 
> Server (CentOS 5):
> fs2# uname -r
> 2.6.18-8.1.14.el5
> 
> fs2# cat /etc/exports
> /local/export          
> *(rw,fsid=0,insecure,no_subtree_check,sync,anonuid=65534,anongid=65534)
> 
> fs2# /var/log/messages
> -----
> Oct 31 13:31:48 fs2 mountd[2452]: refused unmount request from d.foo for /RPMS
> (/): not exported
> Oct 31 13:35:34 fs2 mountd[2452]: refused unmount request from d.foo for /RPMS
> (/): not exported
> Oct 31 13:38:08 fs2 mountd[2452]: refused unmount request from d.foo for /RPMS
> (/): not exported
> Oct 31 13:41:57 fs2 -- MARK --
> Oct 31 13:46:57 fs2 -- MARK --
> Oct 31 13:51:57 fs2 -- MARK --
> -----
> 
> 
> Repeated five times:
> - autofs start
> - ls /misc/RPMS
> - wait for unmount
> - autofs stop
> 
> The produced debug is in my previous comment.
> The five attempts differed on the server iptables setting.
> 
> 1. around 13:31
> ACCEPT     udp dpt:2049 
> ACCEPT     tcp dpt:111
> -produces warning on server
> -10 packets to tcp/111, 1 packet to udp/2049
> 
> 2. around 13:35
> REJECT     udp dpt:2049 
> ACCEPT     tcp dpt:111
> -produces warning on server
> -10 packets to tcp/111, 1 packet to udp/2049
> 
> 3. around 13:38
> DROP       udp dpt:2049 
> ACCEPT     tcp dpt:111
> -produces warning on server
> -10 packets to tcp/111, 1 packet to udp/2049
> 
> 4. around 13:41
> DROP       udp dpt:2049 
> REJECT     tcp dpt:111 reject-with icmp-port-unreachable 
> -3 packets to tcp/111, 1 packet to udp/2049
> 
> 5. around 13:44
> DROP       udp dpt:2049 
> DROP       tcp dpt:111
> -14 packets to tcp/111, 1 packet to udp/2049
> -timeout of about 6 minutes on unmount, as noticeable from the debug
> 
> 
> 'ls /misc/RPMS', ie mount has noticeable (second of two) delay, if udp/2049 is
> not ACCEPT on server

Yes, I think the patch I mentioned will help with this,
assuming mount is doing what it's supposed to.

> 
> On server:
> - On mount, one new packet arrives to both tcp/111 and udp/2049.
> - On unmount, only the tcp/111 receives packets (two in state NEW), except if
> tcp/111 is DROP, when client sends 13 packets.

The umount has to me umount(8) because autofs definitely
doesn't do any RPC requests for a umount, it just calls
umount. We'll need to consult the nfs-utils maintainer 
about this. But first let's sort out the autofs bits.

> 
> 
> Thus, there are IMO issues:
> 1. On mount, udp/2049 is accessed even if mount will use tcp.  Minor delay, ie
> not a real problem.

Yes, but I don't like it so we need to fix it.

> 2. On unmount, tcp/111 that DROPs packets causes untolerable timeout.
> 3. On unmount, if server does listen to tcp/111, it will receive unmount request
> for "non-exported path".  (The scenario that I have not tested is to export '/'
> as fsid=0, where both nfs3 and nfs4 clients will request same path from the
server.

Yes, what I need to work out here is if autofs is, in fact,
issuing a umount for the correct path. I'll have a look at
the log you posted and see if I can work it out.

Ian
Comment 5 Jukka Lehtonen 2007-11-02 07:44:22 EDT
(In reply to comment #4)
> I'm not sure what we can do about this really because,
> as I mentioned previously, if port=<the nfs4 server port>
> isn't given as an option then autofs will ask portmap
> what port the service is on in order to send ping to
> the NULL procedure. 
Silly me, forgot to test that option.  Yes, adding port=<foo>
does remove the one ping to portmap port from the mount event.
It does not prevent the unmount from sending two pings to
portmap. (Or many if using DROP firewall target.)
Comment 6 Ian Kent 2007-11-02 09:56:14 EDT
(In reply to comment #5)
> (In reply to comment #4)
> > I'm not sure what we can do about this really because,
> > as I mentioned previously, if port=<the nfs4 server port>
> > isn't given as an option then autofs will ask portmap
> > what port the service is on in order to send ping to
> > the NULL procedure. 
> Silly me, forgot to test that option.  Yes, adding port=<foo>
> does remove the one ping to portmap port from the mount event.
> It does not prevent the unmount from sending two pings to
> portmap. (Or many if using DROP firewall target.)

And you can't use "proto=" either.
I'll have a quick look at umount soon as I get a chance.

Ian
Comment 7 Jukka Lehtonen 2007-11-02 10:18:19 EDT
Remembered to try manual mount/umount for comparison:

root d:/mnt > mount -v -t nfs4 -o proto=tcp,port=2049,ro,hard,intr,nodev,nosuid
fs2:/RPMS test
mount: pinging: prog 100003 vers 4 prot tcp port 2049

root d:/mnt > umount -v test
mount: Unable to connect to 1.2.3.4:111, errno 111 (Connection refused)
fs2:/RPMS umounted
Could not find /mnt/test in mtab


The 'mount' does not probe udp/2049, like the automounting does.
The 'umount' does probe the tcp/111 (twice, which my server did REJECT, hence
the error on client side), so there the autofs is not to blame.

And when the server allows access to portmap,
server says:
Nov  2 16:16:02 fs2 mountd[2452]: refused unmount request from d.foo for /RPMS
(/): not exported

while client says:
root d:/mnt > umount -v test
mount: trying 1.2.3.4 prog 100005 vers 3 prot tcp port 2053
fs2:/RPMS umounted
Could not find /mnt/test in mtab
Comment 8 Bug Zapper 2008-05-14 10:55:36 EDT
This message is a reminder that Fedora 7 is nearing the end of life. Approximately 30 (thirty) days from now Fedora will stop maintaining and issuing updates for Fedora 7. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as WONTFIX if it remains open with a Fedora 'version' of '7'.

Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version prior to Fedora 7's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that we may not be able to fix it before Fedora 7 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora please change the 'version' of this bug. If you are unable to change the version, please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. If possible, it is recommended that you try the newest available Fedora distribution to see if your bug still exists.

Please read the Release Notes for the newest Fedora distribution to make sure it will meet your needs:
http://docs.fedoraproject.org/release-notes/

The process we are following is described here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 9 Bug Zapper 2008-06-16 22:45:52 EDT
Fedora 7 changed to end-of-life (EOL) status on June 13, 2008. 
Fedora 7 is no longer maintained, which means that it will not 
receive any further security or bug fix updates. As a result we 
are closing this bug. 

If you can reproduce this bug against a currently maintained version 
of Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.

Note You need to log in before you can comment on or make changes to this bug.