Bug 244779

Summary: autofs stopped finding LDAP master map
Product: [Fedora] Fedora Reporter: Jason Tibbitts <j>
Component: autofsAssignee: Ian Kent <ikent>
Status: CLOSED CURRENTRELEASE QA Contact: Brock Organ <borgan>
Severity: low Docs Contact:
Priority: low    
Version: rawhideCC: ikent, jmoyer
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: autofs-5.0.2-2 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2007-06-20 04:11:14 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
contents of /var/log/debug after restarting autofs
none
Patch to initialize local var in parse_server_string none

Description Jason Tibbitts 2007-06-19 05:25:39 UTC
Today I updated my i386 rawhide machine and autofs neglected to find my LDAP
master map.  I believe it was working under 1:5.0.1-14 but has broken in
1:5.0.1-16.  (I've been running the same LDAP server and autofs client
configuration for years now with no issues.).

Unfortunately I haven't much idea how to go about debugging this properly; I
have determined that if /etc/auto.master exists, autofs doesn't contact my ldap
servers at all.  If I remove it, it does contact the server but doesn't find
anything.  I haven't figured out how to do more verbose debugging on the ldap
server end at this point.  nsswitch.conf has "automount: files ldap".  I've set
LOGGING="debug" in /etc/sysconfig/autofs but it doesn't really log much and
nothing relating to LDAP lookups at all.

I see that 5.0.2-1 is in CVS; I'll build it and see if there's any change in
behavior.

Comment 1 Jason Tibbitts 2007-06-19 05:33:20 UTC
It's a pain to rebuild things with no home directory, but I managed to get it
done and unfortunately there's no change in behavior that I've been able to see.

Comment 2 Ian Kent 2007-06-19 07:39:42 UTC
(In reply to comment #1)
> It's a pain to rebuild things with no home directory, but I managed to get it
> done and unfortunately there's no change in behavior that I've been able to see.

This would have to be the addition of the ldaps support.
Can you give me an example of your LDAP maps please.

Ian

Comment 3 Ian Kent 2007-06-19 07:43:22 UTC
(In reply to comment #0)
> server end at this point.  nsswitch.conf has "automount: files ldap".  I've set
> LOGGING="debug" in /etc/sysconfig/autofs but it doesn't really log much and
> nothing relating to LDAP lookups at all.

You need to increase the log level in syslog.conf to capture
the logging. See http://people.redhat.com/jmoyer for an
explanation of this. The debug log would be very useful
to me.

Ian


Comment 4 Ian Kent 2007-06-19 11:29:02 UTC
(In reply to comment #0)
> Today I updated my i386 rawhide machine and autofs neglected to find my LDAP
> master map.  I believe it was working under 1:5.0.1-14 but has broken in
> 1:5.0.1-16.  (I've been running the same LDAP server and autofs client
> configuration for years now with no issues.).

I've re-tested all the valid syntaxes that I can think of and
I can't see any problem.

You need to give me more information.

I'm a little concerned that you say you've been using the same
configuration for years as the configuration for autofs version
5 is quite different to version 4.

In fact, if you have the line "automount: files nis" in
/etc/nsswicth.conf, the file /etc/auto.master exists and
you don't have the line (assuming the name of the LDAP
master map is auto.master):
+auto.master
in /etc/auto.master (also assuming the default master map
name in the autofs configuration) then autofs v5 won't try
to contact your LDAP server.

Ian

Comment 5 Ian Kent 2007-06-19 12:00:30 UTC
(In reply to comment #4)
> In fact, if you have the line "automount: files nis" in

That should read "automount: files ldap"

Ian


Comment 6 Jason Tibbitts 2007-06-19 15:06:32 UTC
Sorry, I know that's a terrible bug report.  Let's see:

nsswitch.conf has "automount: files ldap".

Here's "ldapsearch nismapname=auto.master":

# auto.master, math.uh.edu
dn: nisMapName=auto.master,dc=math,dc=uh,dc=edu
objectClass: top
objectClass: nisMap
nisMapName: auto.master

# /home, auto.master, math.uh.edu
dn: cn=/home,nisMapName=auto.master,dc=math,dc=uh,dc=edu
objectClass: nisObject
cn: /home
nisMapName: auto.master
nisMapEntry: ldap:nisMapName=auto.home,dc=math,dc=uh,dc=edu

# /nas, auto.master, math.uh.edu
dn: cn=/nas,nisMapName=auto.master,dc=math,dc=uh,dc=edu
objectClass: nisObject
cn: /nas
nisMapName: auto.master
nisMapEntry: ldap:nisMapName=auto.nas,dc=math,dc=uh,dc=edu

Here's a sample entry from the auto.home map:

# tibbs, auto.home, math.uh.edu
dn: cn=tibbs,nisMapName=auto.home,dc=math,dc=uh,dc=edu
objectClass: nisObject
cn: tibbs
nisMapEntry: epithumia:/export/h-tibbs/tibbs
nisMapName: auto.home

Generally I leave the default auto.master file unchanged in new installations
and things "just work"; so I'm using the default auto.master file here, which
does have "+auto.master" at the end.

I have tweaked syslog.conf as described on your web page and restarted the
server; here are a few entries which seem relevant:

Jun 19 09:58:33 temp2 automount[2777]: get_query_dn: lookup(ldap): query dn
nisMapName=auto.master,dc=math,dc=uh,dc=edu
Jun 19 09:58:33 temp2 automount[2777]: unbind_ldap_connection: use_tls: 1
...
Jun 19 09:58:33 temp2 automount[2777]: lookup_read_master: lookup(ldap):
searching for "(objectclass=nisObject)" under
"nisMapName=auto.master,dc=math,dc=uh,dc=edu"
Jun 19 09:58:33 temp2 automount[2777]: lookup_read_master: lookup(ldap):
examining entries
Jun 19 09:58:33 temp2 automount[2777]: master_do_mount: mounting /home
Jun 19 09:58:33 temp2 automount[2777]: lookup_nss_read_map: reading map ldap
ldap:nisMapName=auto.home,dc=math,dc=uh,dc=edu
Jun 19 09:58:33 temp2 automount[2777]: parse_server_string: lookup(ldap):
Attempting to parse LDAP information from string
"ldap:nisMapName=auto.home,dc=math,dc=uh,dc=edu".

Which looks to me like it was able to talk to the server and find at least one
map from it.  I'll attach the full log.

Comment 7 Jason Tibbitts 2007-06-19 15:07:22 UTC
Created attachment 157376 [details]
contents of /var/log/debug after restarting autofs

Comment 8 Ian Kent 2007-06-19 15:46:41 UTC
(In reply to comment #6)
> Sorry, I know that's a terrible bug report.  Let's see:
> 
> nsswitch.conf has "automount: files ldap".
> 
> Here's "ldapsearch nismapname=auto.master":
> 
> # auto.master, math.uh.edu
> dn: nisMapName=auto.master,dc=math,dc=uh,dc=edu
> objectClass: top
> objectClass: nisMap
> nisMapName: auto.master
> 
> # /home, auto.master, math.uh.edu
> dn: cn=/home,nisMapName=auto.master,dc=math,dc=uh,dc=edu
> objectClass: nisObject
> cn: /home
> nisMapName: auto.master
> nisMapEntry: ldap:nisMapName=auto.home,dc=math,dc=uh,dc=edu

OK, just to make sure I tested this old syntax and it looks
like I haven't broken it (yet).

> 
> # /nas, auto.master, math.uh.edu
> dn: cn=/nas,nisMapName=auto.master,dc=math,dc=uh,dc=edu
> objectClass: nisObject
> cn: /nas
> nisMapName: auto.master
> nisMapEntry: ldap:nisMapName=auto.nas,dc=math,dc=uh,dc=edu
> 
> Here's a sample entry from the auto.home map:
> 
> # tibbs, auto.home, math.uh.edu
> dn: cn=tibbs,nisMapName=auto.home,dc=math,dc=uh,dc=edu
> objectClass: nisObject
> cn: tibbs
> nisMapEntry: epithumia:/export/h-tibbs/tibbs
> nisMapName: auto.home

Yep, this looks fine to me.

> 
> Generally I leave the default auto.master file unchanged in new installations
> and things "just work"; so I'm using the default auto.master file here, which
> does have "+auto.master" at the end.

Yep, and that should work fine and locate the ldap master map
as it should.

Ian


Comment 9 Ian Kent 2007-06-19 15:48:05 UTC
(In reply to comment #7)
> Created an attachment (id=157376) [edit]
> contents of /var/log/debug after restarting autofs
> 

This looks like it's not a full log?
Did autofs crash, do you have a core?

Ian



Comment 10 Jason Tibbitts 2007-06-19 16:28:46 UTC
That's all I had; I don't see a core file dropped anywhere on the entire machine
and there was no indication of a crash.  I did another restart and the log ends
exactly the same way (with the parse_server_string call).  I attempted to strace
"/sbin/service autofs restart" but it hangs.  (Ctrl-c doesn't do anything, but
kill -9 from another terminal works.)  The last lines in the trace are:

3180  send(3, "<31>Jun 19 11:25:18 automount[31"..., 170, MSG_NOSIGNAL) = 170
3180  open("/dev/tty", O_RDWR|O_NOCTTY|O_NONBLOCK) = -1 ENXIO (No such device or
address)
3180  writev(2, [{"*** buffer overflow detected ***"..., 34}, {"automount", 9},
{" terminated\n", 12}], 3) = 55
3180  futex(0x8e4a14, FUTEX_WAKE, 2147483647) = 0
3180  futex(0xdeaae8, FUTEX_WAKE, 2147483647) = 0
3180  write(2, "======= Backtrace: =========\n", 29) = 29
3180  writev(2, [{"/lib/libc.so.6", 14}, {"(", 1}, {"__chk_fail", 10}, {"+0x",
3}, {"41", 2}, {")", 1}, {"[0x", 3}, {"876ce1", 6}

However, I can't find any notice of that in the logs anywhere.

Comment 11 Ian Kent 2007-06-19 17:14:20 UTC
(In reply to comment #10)
> That's all I had; I don't see a core file dropped anywhere on the entire machine
> and there was no indication of a crash.  I did another restart and the log 

Is there a core file in the root directory?

Ian

Comment 12 Jeff Moyer 2007-06-19 17:20:15 UTC
(In reply to comment #11)
> (In reply to comment #10)
> > That's all I had; I don't see a core file dropped anywhere on the entire machine
> > and there was no indication of a crash.  I did another restart and the log 
> 
> Is there a core file in the root directory?

Don't you need to do a ulimit -c unlimited to get that?  It would be just as
easy to ask if the automount daemon is still running, right?



Comment 13 Ian Kent 2007-06-19 17:30:40 UTC
(In reply to comment #12)
> (In reply to comment #11)
> > (In reply to comment #10)
> > > That's all I had; I don't see a core file dropped anywhere on the entire
machine
> > > and there was no indication of a crash.  I did another restart and the log 
> > 
> > Is there a core file in the root directory?
> 
> Don't you need to do a ulimit -c unlimited to get that?  It would be just as
> easy to ask if the automount daemon is still running, right?

Nop and yes.
The daemon is still setup to dump a core if it crashes.

Ian



Comment 14 Ian Kent 2007-06-19 17:32:20 UTC
Looks like I'll have to try to duplicate this on an
actual devel install. I'll set that up tomorrow.

Ian


Comment 15 Jason Tibbitts 2007-06-19 17:59:25 UTC
Doh, core files have the pid appended and my find call was deficient.

Yes, there are a boatload of core files in /.  I pulled in a pile of debuginfo
packages and got the following backtrace:

Core was generated by `automount'.
Program terminated with signal 6, Aborted.
#0  0x008aa402 in __kernel_vsyscall ()
(gdb) bt
#0  0x008aa402 in __kernel_vsyscall ()
#1  0x00138fa0 in *__GI_raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#2  0x0013a8b1 in *__GI_abort () at abort.c:88
#3  0x0016febb in __libc_message (do_abort=2, fmt=0x2370c0 "*** buffer overflow
detected ***: %s terminated\n")
    at ../sysdeps/unix/sysv/linux/libc_fatal.c:170
#4  0x001f4ce1 in *__GI___chk_fail () at chk_fail.c:31
#5  0x001f41a7 in __strcat_chk (dest=0xb7d45f7f "ldap:*#\001\200", src=0x27b569
"//", destlen=4294967295) at strcat_chk.c:51
#6  0x0026ab65 in lookup_init () from /usr/lib/autofs/lookup_ldap.so
#7  0x80012366 in open_lookup () from /usr/sbin/automount
#8  0x80012e2f in ?? () from /usr/sbin/automount
#9  0x8001357b in lookup_nss_read_map () from /usr/sbin/automount
#10 0x8000ac00 in mount_autofs_indirect () from /usr/sbin/automount
#11 0x800087bb in handle_mounts () from /usr/sbin/automount
#12 0x004fb2fb in start_thread (arg=0xb7d46b90) at pthread_create.c:296
#13 0x001e093e in clone () from /lib/libc.so.6

Does that help at all?

Comment 16 Ian Kent 2007-06-19 18:01:53 UTC
(In reply to comment #15)
> Doh, core files have the pid appended and my find call was deficient.
> 
> Yes, there are a boatload of core files in /.  I pulled in a pile of debuginfo
> packages and got the following backtrace:
> 
> Core was generated by `automount'.
> Program terminated with signal 6, Aborted.
> #0  0x008aa402 in __kernel_vsyscall ()
> (gdb) bt
> #0  0x008aa402 in __kernel_vsyscall ()
> #1  0x00138fa0 in *__GI_raise (sig=6) at
../nptl/sysdeps/unix/sysv/linux/raise.c:64
> #2  0x0013a8b1 in *__GI_abort () at abort.c:88
> #3  0x0016febb in __libc_message (do_abort=2, fmt=0x2370c0 "*** buffer overflow
> detected ***: %s terminated\n")
>     at ../sysdeps/unix/sysv/linux/libc_fatal.c:170
> #4  0x001f4ce1 in *__GI___chk_fail () at chk_fail.c:31
> #5  0x001f41a7 in __strcat_chk (dest=0xb7d45f7f "ldap:*#\001\200", src=0x27b569
> "//", destlen=4294967295) at strcat_chk.c:51
> #6  0x0026ab65 in lookup_init () from /usr/lib/autofs/lookup_ldap.so
> #7  0x80012366 in open_lookup () from /usr/sbin/automount
> #8  0x80012e2f in ?? () from /usr/sbin/automount
> #9  0x8001357b in lookup_nss_read_map () from /usr/sbin/automount
> #10 0x8000ac00 in mount_autofs_indirect () from /usr/sbin/automount
> #11 0x800087bb in handle_mounts () from /usr/sbin/automount
> #12 0x004fb2fb in start_thread (arg=0xb7d46b90) at pthread_create.c:296
> #13 0x001e093e in clone () from /lib/libc.so.6
> 
> Does that help at all?

Yep, that's what I need to point me in the right direction.
Thanks.



Comment 17 Ian Kent 2007-06-19 18:15:37 UTC
Created attachment 157406 [details]
Patch to initialize local var in parse_server_string

From the traceback it looks like I've assumed this
stack var is initialized to 0 which I really shouldn't
rely on. If you could try this patch and see if it
resolves the issue it would be great.

Comment 18 Jason Tibbitts 2007-06-19 18:55:52 UTC
I rebuilt the package with that patch applied and it seems to have solved the
problem for me.

Comment 19 Jason Tibbitts 2007-06-19 18:58:26 UTC
BTW, if the map syntax I'm using is old, which syntax is preferred these days? 
What I'm using seems to be the default one according to /etc/sysconfig/autofs.

Comment 20 Ian Kent 2007-06-20 03:19:30 UTC
(In reply to comment #19)
> BTW, if the map syntax I'm using is old, which syntax is preferred these days? 
> What I'm using seems to be the default one according to /etc/sysconfig/autofs.

Actually I was mistaken.
Your using the current format, ldap:[//servername/]dn.
It was leaving out the server name that made me think
it was the old format.

Also, with version 5, you can just use the map name alone
and rely on nsswitch to find your maps. But what you have
works fine so it's not worth changing.

Ian


Comment 21 Jason Tibbitts 2007-06-20 03:22:36 UTC
Cool, thanks for your help and the quick fix.

Comment 22 Ian Kent 2007-06-20 03:50:55 UTC
(In reply to comment #21)
> Cool, thanks for your help and the quick fix.

My pleasure.
I'll commit the fix after I check for any other
stupidity like that.

Ian