Bug 1970478 - /net mount being not cleanly mounted and unmounted
Summary: /net mount being not cleanly mounted and unmounted
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: autofs
Version: 34
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: Ian Kent
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: 1973025 1973892
TreeView+ depends on / blocked
 
Reported: 2021-06-10 14:26 UTC by redhat
Modified: 2021-06-28 01:30 UTC (History)
1 user (show)

Fixed In Version: autofs-5.1.7-17.fc34
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1973025 (view as bug list)
Environment:
Last Closed: 2021-06-28 01:30:54 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
output from /proc/mounts just after ´ls /net/manicminer/home2' (2.75 KB, text/plain)
2021-06-10 14:26 UTC, redhat
no flags Details
output from /proc/mounts in failed state (245 bytes, text/plain)
2021-06-10 15:29 UTC, redhat
no flags Details

Description redhat 2021-06-10 14:26:15 UTC
Created attachment 1789871 [details]
output from /proc/mounts just after ´ls /net/manicminer/home2'

Description of problem:
I use the autofs' /net function for mounting NFS servers. Until recently I used FC33 and never had any problems. Recently I upgraded to FC34 and the function is broken.

The initial mount when entering the directory still works. Interestingly, all directories returned by showmount are mounted right away. After a timeout of 5 minutes the directories should be unmounted and mounted again when re-entering. But not all directories are unmounted cleanly, although they are not in use. If this condition has occurred, then no further mounts will be performed from now on. Instead, an error occurs. The service must be stopped, the directories must be unmounted manually and the service must be restarted.

Version-Release number of selected component (if applicable):
autofs-5.1.7-13.fc34.src.rpm

How reproducible:
Always since upgraded to FC34 as descrived above.

Steps to Reproduce:
1. activate /net line in auto.master
2. restart service
3. cd /net/server/exported_directory_1
4. ls
5. cd /
6. wait > 300 seconds timeout
7. cd /net/server/exported_directory_2
8. error 'No such file or directory'

Actual results:
error 'No such file or directory'
directory does not get mounted

Expected results:
directory gets mounted just like under FC33


Additional info:
I'm not sure if this is similiar to #837649, that has been reported for FC17.

The timeout does not seem to work cleanly either. After 300 seconds all directories are still mounted. 'fuser' on these directories does not return any process.

Comment 1 redhat 2021-06-10 15:29:28 UTC
Created attachment 1789924 [details]
output from /proc/mounts in failed state

Comment 2 Ian Kent 2021-06-11 14:39:58 UTC
(In reply to redhat from comment #1)
> Created attachment 1789924 [details]
> output from /proc/mounts in failed state

Right, I'll look into this.
This may be a problem with the order of exports from the server.
I suspect I've fixed this already recently but I'll need to check.

Can you post what showmount -e <server> returns please.

Comment 3 redhat 2021-06-11 14:56:19 UTC
Hi Ian,

as requested the output of showmount

$ LC_ALL=c showmount -e manicminer
Export list for manicminer:
/var/lib/tftpboot/desinfect2020_x64 192.168.22.0/255.255.255.0
/var/lib/tftpboot/desinfect2020_x86 192.168.22.0/255.255.255.0
/home2/spiele                       192.168.22.81
/home2/media                        192.168.22.81
/home2/fedora                       192.168.22.81
/home/spiele/archiv                 192.168.22.81
/home/backup/archiv                 192.168.22.81
/home2/backup                       192.168.22.81,192.168.22.79
/home/backup                        192.168.22.79
/home/erik                          192.168.22.81,192.168.22.79
/home2/dvb/dbox                     192.168.22.81,192.168.22.73

manicminer is still running FC33.

Comment 4 Ian Kent 2021-06-13 06:20:45 UTC
(In reply to redhat from comment #3)
> Hi Ian,
> 
> as requested the output of showmount
> 
> $ LC_ALL=c showmount -e manicminer
> Export list for manicminer:
> /var/lib/tftpboot/desinfect2020_x64 192.168.22.0/255.255.255.0
> /var/lib/tftpboot/desinfect2020_x86 192.168.22.0/255.255.255.0
> /home2/spiele                       192.168.22.81
> /home2/media                        192.168.22.81
> /home2/fedora                       192.168.22.81
> /home/spiele/archiv                 192.168.22.81
> /home/backup/archiv                 192.168.22.81
> /home2/backup                       192.168.22.81,192.168.22.79
> /home/backup                        192.168.22.79
> /home/erik                          192.168.22.81,192.168.22.79
> /home2/dvb/dbox                     192.168.22.81,192.168.22.73

Yeah, I think that order will trigger the problem.
I'll apply the changes as soon as I can, and we'll see.

> 
> manicminer is still running FC33.

Makes no difference.

Ian

Comment 5 Fedora Update System 2021-06-15 00:47:28 UTC
FEDORA-2021-cb47f6132e has been submitted as an update to Fedora 34. https://bodhi.fedoraproject.org/updates/FEDORA-2021-cb47f6132e

Comment 6 redhat 2021-06-15 15:14:38 UTC
Hi Ian,

(In reply to Fedora Update System from comment #5)
> FEDORA-2021-cb47f6132e has been submitted as an update to Fedora 34.
> https://bodhi.fedoraproject.org/updates/FEDORA-2021-cb47f6132e

I have tried it out. Now I can no longer access /net/manicminer/home/backup/archive.

Here is the output of /proc/mounts

-hosts /net autofs rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,indirect,pipe_ino=41745 0 0
-hosts /net/manicminer/home/backup autofs rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,pipe_ino=41745 0 0
-hosts /net/manicminer/home/erik autofs rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,pipe_ino=41745 0 0
-hosts /net/manicminer/home/spiele/archiv autofs rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,pipe_ino=41745 0 0
-hosts /net/manicminer/home2/backup autofs rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,pipe_ino=41745 0 0
-hosts /net/manicminer/home2/dvb/dbox autofs rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,pipe_ino=41745 0 0
-hosts /net/manicminer/home2/fedora autofs rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,pipe_ino=41745 0 0
-hosts /net/manicminer/home2/media autofs rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,pipe_ino=41745 0 0
-hosts /net/manicminer/home2/spiele autofs rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,pipe_ino=41745 0 0
-hosts /net/manicminer/var/lib/tftpboot/desinfect2020_x64 autofs rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,pipe_ino=41745 0 0
-hosts /net/manicminer/var/lib/tftpboot/desinfect2020_x86 autofs rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,pipe_ino=41745 0 0
manicminer:/home2/backup /net/manicminer/home2/backup nfs4 rw,nosuid,nodev,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.22.81,local_lock=none,addr=192.168.22.1 0 0

I believe that the client should only mount directories that are also exported for it. Please look again at the output of showmount above.

The client has the IP address 192.168.22.81. /home/backup is not directly exported for it but /home/backup/archive. This still worked cleanly with FC33.

Is there any other information I can provide to support you?

Comment 7 Fedora Update System 2021-06-16 02:01:58 UTC
FEDORA-2021-cb47f6132e has been pushed to the Fedora 34 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-cb47f6132e`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-cb47f6132e

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 8 Ian Kent 2021-06-16 02:51:18 UTC
(In reply to redhat from comment #6)
> Hi Ian,
> 
> (In reply to Fedora Update System from comment #5)
> > FEDORA-2021-cb47f6132e has been submitted as an update to Fedora 34.
> > https://bodhi.fedoraproject.org/updates/FEDORA-2021-cb47f6132e
> 
> I have tried it out. Now I can no longer access
> /net/manicminer/home/backup/archive.

Ok, I'll need to investigate further.

The code for this functionality has been completely re-written and
I've found and fixed few problems so far.

The reason it was re-written is that the original code was hard to
understand and performed really, really badly for servers with a
large number of exports.

> 
> Here is the output of /proc/mounts
> 
> -hosts /net autofs
> rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,indirect,
> pipe_ino=41745 0 0
> -hosts /net/manicminer/home/backup autofs
> rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,
> pipe_ino=41745 0 0
> -hosts /net/manicminer/home/erik autofs
> rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,
> pipe_ino=41745 0 0
> -hosts /net/manicminer/home/spiele/archiv autofs
> rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,
> pipe_ino=41745 0 0
> -hosts /net/manicminer/home2/backup autofs
> rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,
> pipe_ino=41745 0 0
> -hosts /net/manicminer/home2/dvb/dbox autofs
> rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,
> pipe_ino=41745 0 0
> -hosts /net/manicminer/home2/fedora autofs
> rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,
> pipe_ino=41745 0 0
> -hosts /net/manicminer/home2/media autofs
> rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,
> pipe_ino=41745 0 0
> -hosts /net/manicminer/home2/spiele autofs
> rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,
> pipe_ino=41745 0 0
> -hosts /net/manicminer/var/lib/tftpboot/desinfect2020_x64 autofs
> rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,
> pipe_ino=41745 0 0
> -hosts /net/manicminer/var/lib/tftpboot/desinfect2020_x86 autofs
> rw,relatime,fd=11,pgrp=5411,timeout=300,minproto=5,maxproto=5,offset,
> pipe_ino=41745 0 0
> manicminer:/home2/backup /net/manicminer/home2/backup nfs4
> rw,nosuid,nodev,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,
> hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.22.81,
> local_lock=none,addr=192.168.22.1 0 0
> 
> I believe that the client should only mount directories that are also
> exported for it. Please look again at the output of showmount above.

Perhaps, but the code to get the exports was the same generated code
that showmount originally used, I'm pretty sure it didn't exclude
exports based on those ip addresses and I didn't previously exclude
them either, I would wait until the NFS mount attempt and ignore
the failure. I did have to re-implement the code to get the exports
as the generated code was crashing due to stack overflow for a large
number of exports so maybe I missed something there.

So I'm not sure what's happening here, I'll need to check how the
old code functioned.

> 
> The client has the IP address 192.168.22.81. /home/backup is not directly
> exported for it but /home/backup/archive. This still worked cleanly with
> FC33.
> 
> Is there any other information I can provide to support you?

Probably not, I'll need to setup a test and use the old code.
If there's anything else I need I'll ask.

Do you have access to an older autofs rpm that works for you
or do I need find one?

Comment 9 Ian Kent 2021-06-16 02:52:27 UTC
btw, I will still push out that update when it's time since it has
some additional fixes.

Comment 10 Ian Kent 2021-06-16 03:58:27 UTC
(In reply to Ian Kent from comment #8)
> (In reply to redhat from comment #6)
> > Hi Ian,
> > 
> > (In reply to Fedora Update System from comment #5)
> > > FEDORA-2021-cb47f6132e has been submitted as an update to Fedora 34.
> > > https://bodhi.fedoraproject.org/updates/FEDORA-2021-cb47f6132e
> > 
> > I have tried it out. Now I can no longer access
> > /net/manicminer/home/backup/archive.
> 
> Ok, I'll need to investigate further.

Ok, so there's a couple of unexpected things going on here.

First though, there is a bug, the mount failure of what would be
/home/backup for you isn't being ignored, I'll need to fix that.

But there was something else unexpected I saw, nfs v4 mounts don't
appear to be honoring those IP restrictions. So the exported dir
/home/backup will get mounted and if it doesn't have an existing
sub directory "archive" the autofs trigger mount won't be mounted
so the export won't be accessible.

If the mounts are done using nfs v3 the mount will not succeed and
on the earlier autofs versions that mount fail will be ignored and
the autofstrigger mount would still be mounted and the export would
be able to be mounted.

I'll fix that, what's called nonstrict, mount handling but there
may be other surprises due to nfs v4 behaving differently. Maybe
my setup is using different versions or a different os (fedora)
and you see different behavior, I don't know.

Ian

Comment 11 Ian Kent 2021-06-16 05:00:20 UTC
Can you give this scracth build a try please:
https://koji.fedoraproject.org/koji/taskinfo?taskID=70208041

It corrects that nonstrict mount fail handling I mentioned and
should behave the same as earlier versions.

Comment 12 redhat 2021-06-16 09:11:48 UTC
Hi Ian,

(In reply to Ian Kent from comment #11)
> Can you give this scracth build a try please:
> https://koji.fedoraproject.org/koji/taskinfo?taskID=70208041
> 
> It corrects that nonstrict mount fail handling I mentioned and
> should behave the same as earlier versions.

As far as I have been able to check with tests so far, the version behaves as expected with regard to the reported error.

However, I have noticed another unpleasantness. The timeout for unmounting is set in /etc/autofs.conf with 'dismount_interval = 300', i.e. 5 minutes. In practice, it is more like 10 minutes, without me having measured it with a stopwatch.

Does the output 'timeo=600' in /proc/mounts indicate this or is this a different timeout?

manicminer:/home/erik /net/manicminer/home/erik nfs4 rw,nosuid,nodev,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.22.81,local_lock=none,addr=192.168.22.1 0 0

Comment 13 Ian Kent 2021-06-16 10:40:33 UTC
(In reply to redhat from comment #12)
> Hi Ian,
> 
> (In reply to Ian Kent from comment #11)
> > Can you give this scracth build a try please:
> > https://koji.fedoraproject.org/koji/taskinfo?taskID=70208041
> > 
> > It corrects that nonstrict mount fail handling I mentioned and
> > should behave the same as earlier versions.
> 
> As far as I have been able to check with tests so far, the version behaves
> as expected with regard to the reported error.

Ok, so I should push another update with this change?

> 
> However, I have noticed another unpleasantness. The timeout for unmounting
> is set in /etc/autofs.conf with 'dismount_interval = 300', i.e. 5 minutes.
> In practice, it is more like 10 minutes, without me having measured it with
> a stopwatch.

That config option is in the "[ amd ]" section what your looking for is
the timeout config option in the "[ autofs ]" section.

However, installed default config value is the same in both cases.
It should expire mounts in about timeout + timeout/4 seconds but I have
noticed lately that's been more like timeout + timeout/2 but I haven't
looked into it yet.

You can always adjust the timeout option to your liking.

> 
> Does the output 'timeo=600' in /proc/mounts indicate this or is this a
> different timeout?

IIRC that's the NFS retry timeout and is in tenths of a second, there's
also retrans. See the nfs(5) man page. These relate to NFS mounting not
automount expire, so it won't help.

Ian

Comment 14 redhat 2021-06-16 10:56:44 UTC
(In reply to Ian Kent from comment #13)
> > > Can you give this scracth build a try please:
> > > https://koji.fedoraproject.org/koji/taskinfo?taskID=70208041
> > > 
> > > It corrects that nonstrict mount fail handling I mentioned and
> > > should behave the same as earlier versions.
> > 
> > As far as I have been able to check with tests so far, the version behaves
> > as expected with regard to the reported error.
> 
> Ok, so I should push another update with this change?

Why do you think another update would be necessary? As far as I have tested so far, I see no need, because the package autofs-5.1.7-16.fc34.x86_64.rpm is good.

Comment 15 Ian Kent 2021-06-17 04:01:46 UTC
(In reply to redhat from comment #14)
> (In reply to Ian Kent from comment #13)
> > > > Can you give this scracth build a try please:
> > > > https://koji.fedoraproject.org/koji/taskinfo?taskID=70208041
> > > > 
> > > > It corrects that nonstrict mount fail handling I mentioned and
> > > > should behave the same as earlier versions.
> > > 
> > > As far as I have been able to check with tests so far, the version behaves
> > > as expected with regard to the reported error.
> > 
> > Ok, so I should push another update with this change?
> 
> Why do you think another update would be necessary? As far as I have tested
> so far, I see no need, because the package autofs-5.1.7-16.fc34.x86_64.rpm
> is good.

That's a test build, I was just making sure it solved your problem
before submitting it as an update.

I'll go ahead and do that.

Comment 16 Fedora Update System 2021-06-17 07:03:47 UTC
FEDORA-2021-5637adb694 has been submitted as an update to Fedora 34. https://bodhi.fedoraproject.org/updates/FEDORA-2021-5637adb694

Comment 17 Fedora Update System 2021-06-18 01:21:36 UTC
FEDORA-2021-5637adb694 has been pushed to the Fedora 34 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-5637adb694`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-5637adb694

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 18 Fedora Update System 2021-06-20 01:25:11 UTC
FEDORA-2021-12a538321a has been pushed to the Fedora 34 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-12a538321a`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-12a538321a

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 19 Fedora Update System 2021-06-28 01:30:54 UTC
FEDORA-2021-12a538321a has been pushed to the Fedora 34 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.