+++ This bug was initially created as a clone of Bug #1970478 +++

Description of problem:
I use the autofs' /net function for mounting NFS servers. Until recently I used FC33 and never had any problems. Recently I upgraded to FC34 and the function is broken.

The initial mount when entering the directory still works. Interestingly, all directories returned by showmount are mounted right away. After a timeout of 5 minutes the directories should be unmounted and mounted again when re-entering. But not all directories are unmounted cleanly, although they are not in use. If this condition has occurred, then no further mounts will be performed from now on. Instead, an error occurs. The service must be stopped, the directories must be unmounted manually and the service must be restarted.

Version-Release number of selected component (if applicable):
autofs-5.1.7-13.fc34.src.rpm

How reproducible:
Always since upgraded to FC34 as descrived above.

Steps to Reproduce:
1. activate /net line in auto.master
2. restart service
3. cd /net/server/exported_directory_1
4. ls
5. cd /
6. wait > 300 seconds timeout
7. cd /net/server/exported_directory_2
8. error 'No such file or directory'

Actual results:
error 'No such file or directory'
directory does not get mounted

Expected results:
directory gets mounted just like under FC33


Additional info:
I'm not sure if this is similiar to #837649, that has been reported for FC17.

The timeout does not seem to work cleanly either. After 300 seconds all directories are still mounted. 'fuser' on these directories does not return any process.

--- Additional comment from  on 2021-06-10 15:29:28 UTC ---



--- Additional comment from Ian Kent on 2021-06-11 14:39:58 UTC ---

(In reply to redhat from comment #1)
> Created attachment 1789924 [details]
> output from /proc/mounts in failed state

Right, I'll look into this.
This may be a problem with the order of exports from the server.
I suspect I've fixed this already recently but I'll need to check.

Can you post what showmount -e <server> returns please.

--- Additional comment from  on 2021-06-11 14:56:19 UTC ---

Hi Ian,

as requested the output of showmount

$ LC_ALL=c showmount -e manicminer
Export list for manicminer:
/var/lib/tftpboot/desinfect2020_x64 192.168.22.0/255.255.255.0
/var/lib/tftpboot/desinfect2020_x86 192.168.22.0/255.255.255.0
/home2/spiele                       192.168.22.81
/home2/media                        192.168.22.81
/home2/fedora                       192.168.22.81
/home/spiele/archiv                 192.168.22.81
/home/backup/archiv                 192.168.22.81
/home2/backup                       192.168.22.81,192.168.22.79
/home/backup                        192.168.22.79
/home/erik                          192.168.22.81,192.168.22.79
/home2/dvb/dbox                     192.168.22.81,192.168.22.73

manicminer is still running FC33.

--- Additional comment from Ian Kent on 2021-06-13 06:20:45 UTC ---

(In reply to redhat from comment #3)
> Hi Ian,
> 
> as requested the output of showmount
> 
> $ LC_ALL=c showmount -e manicminer
> Export list for manicminer:
> /var/lib/tftpboot/desinfect2020_x64 192.168.22.0/255.255.255.0
> /var/lib/tftpboot/desinfect2020_x86 192.168.22.0/255.255.255.0
> /home2/spiele                       192.168.22.81
> /home2/media                        192.168.22.81
> /home2/fedora                       192.168.22.81
> /home/spiele/archiv                 192.168.22.81
> /home/backup/archiv                 192.168.22.81
> /home2/backup                       192.168.22.81,192.168.22.79
> /home/backup                        192.168.22.79
> /home/erik                          192.168.22.81,192.168.22.79
> /home2/dvb/dbox                     192.168.22.81,192.168.22.73

Yeah, I think that order will trigger the problem.
I'll apply the changes as soon as I can, and we'll see.

> 
> manicminer is still running FC33.

Makes no difference.

Ian

--- Additional comment from Fedora Update System on 2021-06-15 00:47:28 UTC ---

FEDORA-2021-cb47f6132e has been submitted as an update to Fedora 34. https://bodhi.fedoraproject.org/updates/FEDORA-2021-cb47f6132e

--- Additional comment from  on 2021-06-15 15:14:38 UTC ---

Hi Ian,

(In reply to Fedora Update System from comment #5)
> FEDORA-2021-cb47f6132e has been submitted as an update to Fedora 34.
> https://bodhi.fedoraproject.org/updates/FEDORA-2021-cb47f6132e

I have tried it out. Now I can no longer access /net/manicminer/home/backup/archive.

Here is the output of /proc/mounts

-hosts /net autofs rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,indirect,pipe_ino=41745 0 0
-hosts /net/manicminer/home/backup autofs rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,pipe_ino=41745 0 0
-hosts /net/manicminer/home/erik autofs rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,pipe_ino=41745 0 0
-hosts /net/manicminer/home/spiele/archiv autofs rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,pipe_ino=41745 0 0
-hosts /net/manicminer/home2/backup autofs rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,pipe_ino=41745 0 0
-hosts /net/manicminer/home2/dvb/dbox autofs rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,pipe_ino=41745 0 0
-hosts /net/manicminer/home2/fedora autofs rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,pipe_ino=41745 0 0
-hosts /net/manicminer/home2/media autofs rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,pipe_ino=41745 0 0
-hosts /net/manicminer/home2/spiele autofs rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,pipe_ino=41745 0 0
-hosts /net/manicminer/var/lib/tftpboot/desinfect2020_x64 autofs rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,pipe_ino=41745 0 0
-hosts /net/manicminer/var/lib/tftpboot/desinfect2020_x86 autofs rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,pipe_ino=41745 0 0
manicminer:/home2/backup /net/manicminer/home2/backup nfs4 rw,nosuid,nodev,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.22.81,local_lock=none,addr=192.168.22.1 0 0

I believe that the client should only mount directories that are also exported for it. Please look again at the output of showmount above.

The client has the IP address 192.168.22.81. /home/backup is not directly exported for it but /home/backup/archive. This still worked cleanly with FC33.

Is there any other information I can provide to support you?

--- Additional comment from Fedora Update System on 2021-06-16 02:01:58 UTC ---

FEDORA-2021-cb47f6132e has been pushed to the Fedora 34 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-cb47f6132e`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-cb47f6132e

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

--- Additional comment from Ian Kent on 2021-06-16 02:51:18 UTC ---

(In reply to redhat from comment #6)
> Hi Ian,
> 
> (In reply to Fedora Update System from comment #5)
> > FEDORA-2021-cb47f6132e has been submitted as an update to Fedora 34.
> > https://bodhi.fedoraproject.org/updates/FEDORA-2021-cb47f6132e
> 
> I have tried it out. Now I can no longer access
> /net/manicminer/home/backup/archive.

Ok, I'll need to investigate further.

The code for this functionality has been completely re-written and
I've found and fixed few problems so far.

The reason it was re-written is that the original code was hard to
understand and performed really, really badly for servers with a
large number of exports.

> 
> Here is the output of /proc/mounts
> 
> -hosts /net autofs
> rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,indirect,
> pipe_ino=41745 0 0
> -hosts /net/manicminer/home/backup autofs
> rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,
> pipe_ino=41745 0 0
> -hosts /net/manicminer/home/erik autofs
> rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,
> pipe_ino=41745 0 0
> -hosts /net/manicminer/home/spiele/archiv autofs
> rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,
> pipe_ino=41745 0 0
> -hosts /net/manicminer/home2/backup autofs
> rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,
> pipe_ino=41745 0 0
> -hosts /net/manicminer/home2/dvb/dbox autofs
> rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,
> pipe_ino=41745 0 0
> -hosts /net/manicminer/home2/fedora autofs
> rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,
> pipe_ino=41745 0 0
> -hosts /net/manicminer/home2/media autofs
> rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,
> pipe_ino=41745 0 0
> -hosts /net/manicminer/home2/spiele autofs
> rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,
> pipe_ino=41745 0 0
> -hosts /net/manicminer/var/lib/tftpboot/desinfect2020_x64 autofs
> rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,
> pipe_ino=41745 0 0
> -hosts /net/manicminer/var/lib/tftpboot/desinfect2020_x86 autofs
> rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,
> pipe_ino=41745 0 0
> manicminer:/home2/backup /net/manicminer/home2/backup nfs4
> rw,nosuid,nodev,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,
> hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.22.81,
> local_lock=none,addr=192.168.22.1 0 0
> 
> I believe that the client should only mount directories that are also
> exported for it. Please look again at the output of showmount above.

Perhaps, but the code to get the exports was the same generated code
that showmount originally used, I'm pretty sure it didn't exclude
exports based on those ip addresses and I didn't previously exclude
them either, I would wait until the NFS mount attempt and ignore
the failure. I did have to re-implement the code to get the exports
as the generated code was crashing due to stack overflow for a large
number of exports so maybe I missed something there.

So I'm not sure what's happening here, I'll need to check how the
old code functioned.

> 
> The client has the IP address 192.168.22.81. /home/backup is not directly
> exported for it but /home/backup/archive. This still worked cleanly with
> FC33.
> 
> Is there any other information I can provide to support you?

Probably not, I'll need to setup a test and use the old code.
If there's anything else I need I'll ask.

Do you have access to an older autofs rpm that works for you
or do I need find one?

--- Additional comment from Ian Kent on 2021-06-16 02:52:27 UTC ---

btw, I will still push out that update when it's time since it has
some additional fixes.

--- Additional comment from Ian Kent on 2021-06-16 03:58:27 UTC ---

(In reply to Ian Kent from comment #8)
> (In reply to redhat from comment #6)
> > Hi Ian,
> > 
> > (In reply to Fedora Update System from comment #5)
> > > FEDORA-2021-cb47f6132e has been submitted as an update to Fedora 34.
> > > https://bodhi.fedoraproject.org/updates/FEDORA-2021-cb47f6132e
> > 
> > I have tried it out. Now I can no longer access
> > /net/manicminer/home/backup/archive.
> 
> Ok, I'll need to investigate further.

Ok, so there's a couple of unexpected things going on here.

First though, there is a bug, the mount failure of what would be
/home/backup for you isn't being ignored, I'll need to fix that.

But there was something else unexpected I saw, nfs v4 mounts don't
appear to be honoring those IP restrictions. So the exported dir
/home/backup will get mounted and if it doesn't have an existing
sub directory "archive" the autofs trigger mount won't be mounted
so the export won't be accessible.

If the mounts are done using nfs v3 the mount will not succeed and
on the earlier autofs versions that mount fail will be ignored and
the autofstrigger mount would still be mounted and the export would
be able to be mounted.

I'll fix that, what's called nonstrict, mount handling but there
may be other surprises due to nfs v4 behaving differently. Maybe
my setup is using different versions or a different os (fedora)
and you see different behavior, I don't know.

Ian

--- Additional comment from Ian Kent on 2021-06-16 05:00:20 UTC ---

Can you give this scracth build a try please:
https://koji.fedoraproject.org/koji/taskinfo?taskID=70208041

It corrects that nonstrict mount fail handling I mentioned and
should behave the same as earlier versions.

--- Additional comment from  on 2021-06-16 09:11:48 UTC ---

Hi Ian,

(In reply to Ian Kent from comment #11)
> Can you give this scracth build a try please:
> https://koji.fedoraproject.org/koji/taskinfo?taskID=70208041
> 
> It corrects that nonstrict mount fail handling I mentioned and
> should behave the same as earlier versions.

As far as I have been able to check with tests so far, the version behaves as expected with regard to the reported error.

However, I have noticed another unpleasantness. The timeout for unmounting is set in /etc/autofs.conf with 'dismount_interval = 300', i.e. 5 minutes. In practice, it is more like 10 minutes, without me having measured it with a stopwatch.

Does the output 'timeo=600' in /proc/mounts indicate this or is this a different timeout?

manicminer:/home/erik /net/manicminer/home/erik nfs4 rw,nosuid,nodev,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.22.81,local_lock=none,addr=192.168.22.1 0 0

--- Additional comment from Ian Kent on 2021-06-16 10:40:33 UTC ---

(In reply to redhat from comment #12)
> Hi Ian,
> 
> (In reply to Ian Kent from comment #11)
> > Can you give this scracth build a try please:
> > https://koji.fedoraproject.org/koji/taskinfo?taskID=70208041
> > 
> > It corrects that nonstrict mount fail handling I mentioned and
> > should behave the same as earlier versions.
> 
> As far as I have been able to check with tests so far, the version behaves
> as expected with regard to the reported error.

Ok, so I should push another update with this change?

> 
> However, I have noticed another unpleasantness. The timeout for unmounting
> is set in /etc/autofs.conf with 'dismount_interval = 300', i.e. 5 minutes.
> In practice, it is more like 10 minutes, without me having measured it with
> a stopwatch.

That config option is in the "[ amd ]" section what your looking for is
the timeout config option in the "[ autofs ]" section.

However, installed default config value is the same in both cases.
It should expire mounts in about timeout + timeout/4 seconds but I have
noticed lately that's been more like timeout + timeout/2 but I haven't
looked into it yet.

You can always adjust the timeout option to your liking.

> 
> Does the output 'timeo=600' in /proc/mounts indicate this or is this a
> different timeout?

IIRC that's the NFS retry timeout and is in tenths of a second, there's
also retrans. See the nfs(5) man page. These relate to NFS mounting not
automount expire, so it won't help.

Ian

--- Additional comment from  on 2021-06-16 10:56:44 UTC ---

(In reply to Ian Kent from comment #13)
> > > Can you give this scracth build a try please:
> > > https://koji.fedoraproject.org/koji/taskinfo?taskID=70208041
> > > 
> > > It corrects that nonstrict mount fail handling I mentioned and
> > > should behave the same as earlier versions.
> > 
> > As far as I have been able to check with tests so far, the version behaves
> > as expected with regard to the reported error.
> 
> Ok, so I should push another update with this change?

Why do you think another update would be necessary? As far as I have tested so far, I see no need, because the package autofs-5.1.7-16.fc34.x86_64.rpm is good.

--- Additional comment from Ian Kent on 2021-06-17 04:01:46 UTC ---

(In reply to redhat from comment #14)
> (In reply to Ian Kent from comment #13)
> > > > Can you give this scracth build a try please:
> > > > https://koji.fedoraproject.org/koji/taskinfo?taskID=70208041
> > > > 
> > > > It corrects that nonstrict mount fail handling I mentioned and
> > > > should behave the same as earlier versions.
> > > 
> > > As far as I have been able to check with tests so far, the version behaves
> > > as expected with regard to the reported error.
> > 
> > Ok, so I should push another update with this change?
> 
> Why do you think another update would be necessary? As far as I have tested
> so far, I see no need, because the package autofs-5.1.7-16.fc34.x86_64.rpm
> is good.

That's a test build, I was just making sure it solved your problem
before submitting it as an update.

I'll go ahead and do that.