Bug 1049017 - Regression: autofs mounts hang if maps are reloaded while the mount is expiring
Summary: Regression: autofs mounts hang if maps are reloaded while the mount is expiring
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: autofs
Version: 5.10
Hardware: All
OS: Linux
high
high
Target Milestone: rc
: ---
Assignee: Ian Kent
QA Contact: JianHong Yin
URL:
Whiteboard:
Depends On:
Blocks: 1144746
TreeView+ depends on / blocked
 
Reported: 2014-01-06 19:33 UTC by David Halliwell
Modified: 2019-08-15 03:44 UTC (History)
9 users (show)

Fixed In Version: autofs-5.0.1-0.rc2.184.el5
Doc Type: Bug Fix
Doc Text:
In a previous version, a check for mounted file systems was removed from autofs mount control if a miscellaneous device was not used. However, a subsequent update introduced a mount export function that requires this check. As a consequence, autofs mounts sometimes became unresponsive when re-reading the mount map. This update fixes the bug and autofs mounts no longer hang in the scenario described.
Clone Of:
: 1144746 (view as bug list)
Environment:
Last Closed: 2014-09-16 00:17:23 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
autofs debug messages of issue reproduced (126.18 KB, text/plain)
2014-01-29 22:40 UTC, Kyle Squizzato
no flags Details
Patch - check for existing offset mount before mounting (1.10 KB, patch)
2014-01-30 08:59 UTC, Ian Kent
no flags Details | Diff


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2014:1240 0 normal SHIPPED_LIVE autofs bug fix update 2014-09-16 04:14:15 UTC

Description David Halliwell 2014-01-06 19:33:26 UTC
Description of problem:
-----------------------
If autofs maps are reloaded while a /net (indirect) mount is in the process of expiring, the mount will hang blocking all further IO.

Version-Release number of selected component (if applicable):
-------------------------------------------------------------
Affected version:
autofs-5.0.1-0.rc2.183.el5.x86_64

Based on my testing, this does not occur with the previous version:
autofs-5.0.1-0.rc2.177.el5.x86_64

How reproducible:
-----------------
Reproducible with the steps below.

Steps to Reproduce:
-------------------
#autofs version:  autofs-5.0.1-0.rc2.183.el5.x86_64

# Edit /etc/sysconfig/autofs
Set debugging:  OPTIONS="-d"
Set timeout:    DEFAULT_TIMEOUT=30

# Start autofs
/etc/init.d/autofs start

# Mount the remote filesystem using /net
ls -l /net/nfsserver/exportedfs/dir

# Wait for ~30 secs until the NFS filesystem is expired/unmounted
watch -n1 mount

# The indirect mount is still present at this point
cat /proc/mounts

# Reload the maps (do this quickly before the indirect mount expires)
/etc/init.d/autofs reload

# Now wait a while for the indirect (autofs) mount to expire, then try to access the remote filesystem again
ls -l /net/nfsserver/exportedfs/dir

Actual results:
---------------
All further IO with the mountpoint /net/nfsserver/exportedfs/dir hangs.

If you dig through the debug messages, you can see the following steps occur:

1/  The offset (indirect) mounts for each exported filesystem are created in /net/nfsserver
2/  The NFS (direct) mount is created at the appropriate offset
3/  This is timed out by autofs after ~30secs
4/ The NFS (direct) filesystem is unmounted

5/ At this stage we tell autofs to reload the maps.
* For some reason, this causes the offset (indirect) directories to be mounted again *

6/ The expiry continues, but the offset directories are now in use again so the expiry fails

Expected results:
-----------------
The mountpoint should not hang.
On older versions of autofs, step 5/ does not seem to cause the offset directories to be mounted again.

Comment 3 David Halliwell 2014-01-27 15:45:33 UTC
This bug does not occur if the misc device is enabled.  So in the config file, set:
USE_MISC_DEVICE=no

We have a support case open for this bug.  Support services have managed to recreate it, we're now trying to push for a fix.

Comment 6 Ian Kent 2014-01-27 22:50:44 UTC
(In reply to David Halliwell from comment #3)
> This bug does not occur if the misc device is enabled.  So in the config
> file, set:
> USE_MISC_DEVICE=no

Not terribly relevant since the misc device should always
be used.

Comment 9 Kyle Squizzato 2014-01-29 22:40:07 UTC
Created attachment 857283 [details]
autofs debug messages of issue reproduced

Comment 10 Ian Kent 2014-01-30 00:43:32 UTC
(In reply to Kyle Squizzato from comment #9)
> Created attachment 857283 [details]
> autofs debug messages of issue reproduced

Thanks for that Kyle.
The log is what I needed but I'm still not sure what is going
on, I'll have a look.

Comment 13 Ian Kent 2014-01-30 08:03:36 UTC
(In reply to Ian Kent from comment #10)
> (In reply to Kyle Squizzato from comment #9)
> > Created attachment 857283 [details]
> > autofs debug messages of issue reproduced
> 
> Thanks for that Kyle.
> The log is what I needed but I'm still not sure what is going
> on, I'll have a look.

This log is rather interesting.

I can see where the second mount is done and I'm slowly
remembering what I did with the change that lead to this.

But the change that allows the second mount to happen was
done a much longer time ago.

Looking at it I can't see why I commented out the mounted
check ...... it's certainly needed by the later change.

But more puzzling is the hang.
In theory it shouls mount on top of the second mount leaving
us none the wiser that we have a bug ...... puzzling.

Ian

Comment 14 Ian Kent 2014-01-30 08:59:17 UTC
Created attachment 857392 [details]
Patch - check for existing offset mount before mounting

We can check if this resolves the problem.
Looks like it might.

Comment 15 Ian Kent 2014-01-30 11:16:03 UTC
A package with the above patch is available for testing at:
http://people.redhat.com/~ikent/autofs-5.0.1-0.rc2.183.bz1049017.1

Please check if this resolves the problem and report back.

Comment 18 David Halliwell 2014-01-31 14:36:42 UTC
(In reply to Ian Kent from comment #15)
> A package with the above patch is available for testing at:
> http://people.redhat.com/~ikent/autofs-5.0.1-0.rc2.183.bz1049017.1
> 
> Please check if this resolves the problem and report back.

Thanks, I can confirm that this patch seems to resolve the problem for us.

Comment 31 errata-xmlrpc 2014-09-16 00:17:23 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-1240.html


Note You need to log in before you can comment on or make changes to this bug.