Bug 211206

Summary: kernel 2.6.18-1.2200.fc5 breaks autofs on NFS
Product: [Fedora] Fedora Reporter: Robin Humble <humble+fedora>
Component: kernelAssignee: Ian Kent <ikent>
Status: CLOSED ERRATA QA Contact: Brian Brock <bbrock>
Severity: high Docs Contact:
Priority: medium    
Version: 5CC: davej, dhowells, jmoyer, staubach, steved, tomek, wtogami
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2006-11-12 05:52:15 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 216178    
Attachments:
Description Flags
Handle incorrectly null dentries created by autofs when permission was denied
none
autofs-4.1.4-29 patch to deal with changed semantics of mkdir none

Description Robin Humble 2006-10-17 20:51:24 UTC
Description of problem:
autofs doesn't work with 2.6.18-1.2200.fc5 when the mountpoint for the
automounted dirs is an NFS directory. it works fine in 2.6.17-1.2187_FC5 and in
all previous kernels.
this looks like a screwup in FS-Cache or how NFS and autofs uses it to me.

Version-Release number of selected component (if applicable):
kernel-2.6.18-1.2200.fc5
autofs-4.1.4-29

How reproducible:

always: if autofs is chkconfig'd on at boot then it always fails.

mostly: if autofs is chkconfig'd off and then started by hand after boot, then
it works about half the time.

so there's something time-critical in there...

Steps to Reproduce:
1. have a NFS dir in your fstab like:
someserver:/obj/cita-skel /cita               nfs ro,hard,intr 0 0

where /cita/ has 2 empty dirs in it: /cita/d/ and /cita/scratch/

3.
 - /etc/auto.master is
/cita/d         /var/adm/automount.map --ghost
/cita/scratch   /var/adm/auto.scratch

what's in the maps doesn't really matter, but anyway:

 - /var/adm/automount.map is:
www -hard,nosuid,intr,rw someserver:/obj/www
adm -hard,nosuid,intr,rw someserver:/obj/adm
...

 - /var/adm/auto.scratch is:
*       -hard,nosuid,intr,rw    &:/scratch

Actual results:

% ls -l /cita/
?---------  ? ?    ?       ?            ? /cita/d
?---------  ? ?    ?       ?            ? /cita/scratch

% /etc/init.d/autofs status
Configured Mount Points:
------------------------
/usr/sbin/automount --timeout=60 --ghost /cita/d file /var/adm/automount.map 
/usr/sbin/automount --timeout=60 /cita/scratch file /var/adm/auto.scratch 

Active Mount Points:
--------------------

Expected results:
% ls -l /cita/
drwxr-xr-x 77 root root    0 Oct 17 16:29 d/
drwxr-xr-x  2 root root    0 Oct 17 16:29 scratch/

% /etc/init.d/autofs status
Configured Mount Points:
------------------------
/usr/sbin/automount --timeout=60 --ghost /cita/d file /var/adm/automount.map 
/usr/sbin/automount --timeout=60 /cita/scratch file /var/adm/auto.scratch 

Active Mount Points:
--------------------
/usr/sbin/automount --timeout=60 --ghost /cita/d file /var/adm/automount.map
/usr/sbin/automount --timeout=60 /cita/scratch file /var/adm/auto.scratch

Additional info:

the mountpoints turning into ? ? dirs is odd. only umount'ing all the NFS dirs
on the machine and mounting them again wil fix it.

selinux is 'targetted', 'permissive' so shouldn't be affecting anything.

all the NFS related services are started before autofs, as they should be:
% ls -1 /etc/rc5.d/S*
...
/etc/rc5.d/S13portmap@
/etc/rc5.d/S14nfslock@
...
/etc/rc5.d/S18rpcidmapd@
/etc/rc5.d/S19rpcgssd@
...
/etc/rc5.d/S25netfs@
...
/etc/rc5.d/S28autofs@

Comment 1 Jeff Moyer 2006-10-17 21:26:21 UTC
Can you provide debug logs for the automounter please?  You can find
instructions on how to generate them at the following URL:
  http://people.redhat.com/jmoyer/

Thanks.

Comment 2 Robin Humble 2006-10-17 22:22:49 UTC
Oct 17 17:38:00 lemming automount[2125]: starting automounter version 4.1.4-29,
path = /cita/d, maptype = file, mapname = /var/adm/automount.map
Oct 17 17:38:00 lemming automount[2156]: starting automounter version 4.1.4-29,
path = /cita/scratch, maptype = file, mapname = /var/adm/auto.scratch
Oct 17 17:38:00 lemming kernel: audit(1161121080.222:8): avc:  denied  { read }
for  pid=2125 comm="automount" name="automount.map" dev=sda1 ino=12777385
scontext=system_u:system_r:automount_t:s0 tcontext=system_u:object_r:var_t:s0
tclass=file
Oct 17 17:38:00 lemming automount[2125]: failed to mount autofs path /cita/d
Oct 17 17:38:00 lemming automount[2125]: /cita/d: mount failed!
Oct 17 17:38:00 lemming automount[2156]: failed to mount autofs path /cita/scratch
Oct 17 17:38:00 lemming automount[2156]: /cita/scratch: mount failed!

note that the selinux warning appears with all other kernels as well, but it's
only 2.6.18-1.2200.fc5 that doesn't work. and as selinux is 'permissive' then
nothing should be going wrong.

and afterwards there is again:
% ls -l /cita/
?---------  ? ?    ?       ?            ? /cita/d
?---------  ? ?    ?       ?            ? /cita/scratch

/etc/init.d/autofs status is as per "Actual results" above.
nothing is shown by: ps auxwww | grep auto

so still looks like an interaction between NFS, FS-Cache and autofs to me... ??

to complete your standard autofs report:

% cat /etc/nsswitch.conf
passwd:     files
shadow:     files
group:      files
hosts:  dns files
services:   files
networks:   files
protocols:  files
rpc:        files
ethers:     files
netmasks:   files     
bootparams: files
netgroup:
publickey:
automount:  files
aliases:    files

 cat /etc/sysconfig/autofs
# Define custom options in /etc/sysconfig/autofs
# Use LOCALOPTIONS for defining variables, e.g. OSREL
# Use DAEMONOPTIONS to define the unmount timeout
# Define UNDERSCORETODOT as 1 to convert 
#     auto_home to auto.home and auto_mnt to auto.mnt
# Mount options, e.g. rsize=8192, should go in auto.master or
#     the auto_* map entry for a specific mount point
#
LOCALOPTIONS=""
DAEMONOPTIONS="--timeout=60 --debug"

#  UNDERSCORETODOT changes auto_home to auto.home and auto_mnt to auto.mnt
UNDERSCORETODOT=1
DISABLE_DIRECT=1

# Using a good number of maps can cause autofs to take
# some time to exit. If you get init script stop fails
# but find that a little while latter it's gone increase
# this value.
DAEMON_EXIT_WAIT=10

# LDAPAUTOMASTER contains command line arguments for the
# /usr/lib/autofs/autofs-ldap-auto-master program
# Run the program with --help to see available options
LDAPAUTOMASTER=""


please let me know if you'd like any more info...

Comment 3 Ian Kent 2006-10-18 01:19:46 UTC
Sorry to drag you into this David but it looks like we
didn't capture all the cases.

Can you help with this one please?

Ian


Comment 4 Ian Kent 2006-10-18 02:22:42 UTC
(In reply to comment #3)
> Sorry to drag you into this David but it looks like we
> didn't capture all the cases.
> 
> Can you help with this one please?

Davej,

I think your missing a patch.
dhowells, can you post the result of the discussion on
this please. There's nothing in nfs_lookup in 1.2200 that
looks like what was discussed on this.

Ian


Comment 5 Dave Jones 2006-10-18 06:26:50 UTC
Ian, bounce me the patch (or attach it to this bug) and I'll merge it for the
next update.  (Which at the rate that various low-hanging fruit is falling,
won't be too far away).


Comment 6 David Howells 2006-10-18 11:43:39 UTC
Created attachment 138774 [details]
Handle incorrectly null dentries created by autofs when permission was denied

This is fixed upstream by this change:

http://www.kernel.org/git/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=fd6840714d9cf6e93f1d42b904860a94df316b85

Comment 7 David Howells 2006-10-18 11:46:10 UTC
*** Bug 211207 has been marked as a duplicate of this bug. ***

Comment 8 Ian Kent 2006-10-23 05:25:03 UTC
(In reply to comment #6)
> Created an attachment (id=138774) [edit]
> Handle incorrectly null dentries created by autofs when permission was denied
> 
> This is fixed upstream by this change:
> 
>
http://www.kernel.org/git/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=fd6840714d9cf6e93f1d42b904860a94df316b85

Have you had a chance to check if this resolves your problem?

If it does but you still get a permission denied when autofs calls
mkdir_path you probably need the patch below.

Ian




Comment 9 Ian Kent 2006-10-23 05:31:23 UTC
Created attachment 139097 [details]
autofs-4.1.4-29 patch to deal with changed semantics of mkdir

Please try this patch if you find you're getting
permission denied errors following the above kernel
patch resolving the apparent directory corruption.

Comment 10 Robin Humble 2006-10-23 19:07:32 UTC
oh, sorry - are you talking to me? I thought you were chatting amongst
yourselves... umm, yeah - 
http://www.kernel.org/git/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=fd6840714d9cf6e93f1d42b904860a94df316b85

fixes the problem for me with the 2200 fc5 kernel. thanks!

I didn't try the autofs patch. let me know if you want me to.

cheers,
robin

Comment 11 Ian Kent 2006-10-24 01:32:57 UTC
(In reply to comment #10)
> oh, sorry - are you talking to me? I thought you were chatting amongst
> yourselves... umm, yeah - 

Hehe .. yep.

>
http://www.kernel.org/git/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=fd6840714d9cf6e93f1d42b904860a94df316b85
> 
> fixes the problem for me with the 2200 fc5 kernel. thanks!

Great.

> 
> I didn't try the autofs patch. let me know if you want me to.

That would be good if you have time.
The reason for this patch is that along with the apparent
directory corruption autofs can fail a mount request because
it gets a permission error when previously it got a directory
exists error.

Ian


Comment 12 Robin Humble 2006-10-24 22:22:50 UTC
sorry, I'm moving countries and won't have easy access to those machines any
more,  so I can't easily test your autofs patch. is autofs even still getting a
permission error after the kernel fix?

Comment 13 Ian Kent 2006-10-25 08:47:31 UTC
(In reply to comment #12)
> sorry, I'm moving countries and won't have easy access to those machines any
> more,  so I can't easily test your autofs patch. is autofs even still getting a
> permission error after the kernel fix?

No probelm.
I think it shoud be OK.

Ian


Comment 14 Dave Jones 2006-11-12 05:52:15 UTC
should be fixed in 2.6.18-1.2239.fc5 now in updates.