SSSD is looking for a plugin interface for autofs that would allow us to provide autofs with the data it needs. Once such an interface is available, the SSSD team would handle writing the plugin and necessary SSSD back-ends to cache and provide the automount data.

I'm optimistically setting devel_ack for RHEL-6.3 since we have
some discussion starting to occur on the implementation.

*** Bug 751710 has been marked as a duplicate of this bug. ***

So far we haven't made progress with this request.

There are a couple reasons for this. First both the sssd people
and myself have been occupied with other priority work. But the
other difficulty is that both of us seem to be trying to start
at a development point farther along than we have sufficient
information for. This has caused a couple of ill-defined and
unanswered questions to be left hanging.

The situation is that sssd are asking "what do you need and we
will provide it" and I'm asking "what can do provide, I'll use
that". But neither party can realistically answer this or work
out what needs to be done.

I think the only way to make progress is to take a couple of steps
back and have a look at what the situation is now and possibly
consider some (autofs) history.

Let me start by asking, how does sssd integrate into existing
systems now (from the technical view that is, aka. which libraries
what implementation, etc.)?

Currently, SSSD provides two primary interfaces to the system through NSS and PAM.

With NSS, we provide a very small utility library (nss_sss.so.2) which functions as little more than a proxy to forward name-service requests to the SSSD's NSS responder (sssd_nss). The SSSD then either responds to nss_sss.so.2 with a result from its internal LDB cache[1] or else notifies the SSSD Data Provider backend (sssd_be) to query its data source (such as LDAP) for information with which to populate the cache. Once this completes (either successfully or reporting network error), we return results from the (possibly now updated) cache.

Our interface internally with regards to LDAP lookups is highly generic. We are able to retrieve any data from LDAP that would be necessary for answering LDAP requests, and we can store them in the cache in whatever format makes the most sense (probably this will amount to roughly the same as in LDAP).

We don't currently have a public interface that automount can talk to directly, because automount does not have a defined interface in libc that I am aware of (and if it does, it's not implemented in glibc). So there are a few choices here.

1) We can define a standard libc interface for providing automount map data and lobby for its inclusion in glibc. (This will probably take years)
2) We can create a new plugin interface for automount, abstracting out the internal LDAP driver into a plugin. The SSSD team can implement its own plugin written to this interface, using an architecture similar to what is done for NSS as described above, creating an Automount Responder to communicate with the plugin (which would essentially be a proxy)
3) We can define a new standard wire protocol for passing automount data and allow automount to specify a local socket file to which it should connect. SSSD's Automount Responder would then create this socket at startup time and automount could contact it directly.
4) SSSD could provide an LDAP interface that automount's existing LDAP driver could talk to directly. (This will not happen. It's been suggested and rejected upstream due to ridiculous over-complexity several times now. I'm including it now just to identify that we will NOT support it.)


[1] LDB is a schema-less, LDAP-inspired embedded database.

(In reply to comment #13)
> 3) We can define a new standard wire protocol for passing automount data and
> allow automount to specify a local socket file to which it should connect.
> SSSD's Automount Responder would then create this socket at startup time and
> automount could contact it directly.

I believe this would mean creating a new autofs lookup module (/usr/lib64/autofs/lookup_sss.so) that would talk to SSSD. So 

Also, I believe we considered using the executable map at one point. Not that it would be the cleanest solution, but it should be noted for completeness.

(In reply to comment #14)
> (In reply to comment #13)
> > 3) We can define a new standard wire protocol for passing automount data and
> > allow automount to specify a local socket file to which it should connect.
> > SSSD's Automount Responder would then create this socket at startup time and
> > automount could contact it directly.
> 
> I believe this would mean creating a new autofs lookup module
> (/usr/lib64/autofs/lookup_sss.so) that would talk to SSSD. So 

Sorry, bad edit. I meant to say "So it seems that this approach wouldn't require changes to the core autofs".

ad 1)
As far as I know, there used to be an interface in glibc for autofs in the
past. But is is certainly not a runner now so let's forget it.

In future we also hope sssd would provide data for automounter for all backends
(NIS/FILES, LDAP, everything). We centrainly do not want to limit this
functionality to ldap so the interface should be generic.

ad 2) this would probably mean defining something like lookup_sss.so in autofs

Maybe in the long term 3) would be better however. Why? Because other
processes/daemons might need to talk to sssd, too (imagine thunderbird email
client requesting user's email address or some other info, not stored in the
passwd database). Glibc is simply hopelessly outdated for this - sssd might fill the
gap by offering a new, richer interface. But if we go that way, we should
consider some other (higher) IPC systems like D-BUS....?

(In reply to comment #16)
> Maybe in the long term 3) would be better however. Why? Because other
> processes/daemons might need to talk to sssd, too (imagine thunderbird email
> client requesting user's email address or some other info, not stored in the
> passwd database). Glibc is simply hopelessly outdated for this - sssd might
> fill the
> gap by offering a new, richer interface. But if we go that way, we should
> consider some other (higher) IPC systems like D-BUS....?

This is already in the works for SSSD 2.0, but it's a much more complicated discussion. Suffice to say, for now let's stick with solving this one problem on its own.

Agreed.

Would it be sufficient to take the lookup_nis.so autofs lookup library interface and make it the same for the forthcoming lookup_sss.so?

This would require sssd to have bit of inteligence as it would have to transform standard tree-like ldap maps into flat NIS maps but it would be general enough, abstracting the ldap layer altogether.

How does this sound?

(In reply to comment #18)
> Agreed.
> 
> Would it be sufficient to take the lookup_nis.so autofs lookup library
> interface and make it the same for the forthcoming lookup_sss.so?
> 
> This would require sssd to have bit of inteligence as it would have to
> transform standard tree-like ldap maps into flat NIS maps but it would be
> general enough, abstracting the ldap layer altogether.
> 
> How does this sound?

Whatever approach we take is going to require SSSD to massage the data into a format that's easily digestible by automount. However we decide to store the data internally, we'll be post-processing it before returning it to automount. (This will be done in part because we plan to eventually support other backends than LDAP for providing this data).

(In reply to comment #19)
> (In reply to comment #18)
> > Agreed.
> > 
> > Would it be sufficient to take the lookup_nis.so autofs lookup library
> > interface and make it the same for the forthcoming lookup_sss.so?
> > 
> > This would require sssd to have bit of inteligence as it would have to
> > transform standard tree-like ldap maps into flat NIS maps but it would be
> > general enough, abstracting the ldap layer altogether.
> > 
> > How does this sound?
> 
> Whatever approach we take is going to require SSSD to massage the data into a
> format that's easily digestible by automount. However we decide to store the
> data internally, we'll be post-processing it before returning it to automount.
> (This will be done in part because we plan to eventually support other backends
> than LDAP for providing this data).

Sorry I haven't got any replies together yet but rest assured I have
been taking notice, giving it thought and poking around looking for
old code that should provide a basis for discussion.

In fact it's the old code from nss_ldap-255 that I originally worked
on to port for use with autofs version 5 and eventually abandoned due
to the amount of work it would have taken and problem of getting a
largish amount of code into glibc.

While that code is out of date (and was even out of date when I
first looked at it) the internal autofs interfaces haven't actually
changed that much so the principles of the approach still apply. I'll
post that with some discussion about what it is and where the two
source files fit in so there is a picture what is happening.

The sssd guys are likely familiar with nss and I can talk about the
autofs parts so we should end up with a reasonably good understanding
of one way to deal with this. Hopefully that will turn on some lights
as to where to go with this or perhaps enlighten me as to what needs
to be done differently for sssd.

Ian

Let us get to the goal of all this. People do not want to configure LDAP connections multiple times. SSSD provides the right level of security to access central LDAP store. It is also starting to become a preferred single place to configure all LDAP connections coming from the node to the central server. So the goal is to have a simple configuration in autofs like "use_sssd=true" to get all autofs information from SSSD if SSSD is present and configured. Would be even nice if this is default and it would figure out automatically if SSSD is configured and would fallback to standard means if it does not. This way out of the box the admin would have to configure just SSSD and if the central server provides automount info it would become available to autofs. This is the use case we are shooting for.
Autofs can do its own caching so the SSSD caching is not the primary reason to use SSSD. It is simplified single point configuration that people are looking for. I have seen it several times mentioned on the list.

(In reply to comment #21)
> Let us get to the goal of all this. People do not want to configure LDAP
> connections multiple times. SSSD provides the right level of security to access
> central LDAP store. It is also starting to become a preferred single place to
> configure all LDAP connections coming from the node to the central server. So
> the goal is to have a simple configuration in autofs like "use_sssd=true" to
> get all autofs information from SSSD if SSSD is present and configured. Would
> be even nice if this is default and it would figure out automatically if SSSD
> is configured and would fallback to standard means if it does not. This way out
> of the box the admin would have to configure just SSSD and if the central
> server provides automount info it would become available to autofs. This is the
> use case we are shooting for.
> Autofs can do its own caching so the SSSD caching is not the primary reason to
> use SSSD. It is simplified single point configuration that people are looking
> for. I have seen it several times mentioned on the list.

I think that the single-point-of-configuration is a clear win, but I don't know that it's the only one. As I mentioned above, the possibility exists for also having additional backends for SSSD to abstract the source of the automount map data. There's no reason that NIS or DNS or some other as-yet unconsidered mechanism would be out of the question with such an abstraction layer.

I think that's what Ian mentioned (Ian to confirm):
A part of the nss_ldap-255 update was glibc interface update including support for automount maps - and that's what we are going to use for sssd. Do not be confused by the "ldap" in its name. It is generic enough for our usage.

Also no "use_sssd=true" is needed as automouter is already parsing /etc/nsswitch.conf so I imagine that
automount:     files sss

Would do that job once we have the lookup_sss.so library for autofs. But I'll stop babbling around for now - you are the developers so you know the best :-)

It's worth knowing more about the autofs internal interfaces and to
have a look at example code that demonstrates how this could be done.

There are several internal interfaces within autofs implemented as
shared libraries, one is the lookup module.

A lookup module is implemented for each information source and they
each have a fixed interface. Upon loading, automount will get the
library entry points via dlopen() which are:

int lookup_init(const char *mapfmt,
                int argc, const char *const *argv,
                void **context);

int lookup_read_master(struct master *master,
                       time_t age, void *context);

int lookup_read_map(struct autofs_point *ap,
                    time_t age, void *context);

int lookup_mount(struct autofs_point *ap,
                 const char *name, int name_len,
                 void *context);

int lookup_done(void *context);

lookup_init() and lookup_done() are called when the module is
first used and when the module is no longer needed. Note that
doesn't mean an open and close per lookup at all, the scope is
somewhat more broad.

lookup_read_master() is called at program start to read the
master map.

lookup_read_map() reads the entire map.

lookup_mount() looks up an automount map key.

The problem with these is that internal autofs structures are
passed to them so they can do their thing, such as check/update/add
the entry to the internal map entry cache etc. and most likely
shouldn't be exposed to external callers. That's probably not a
big problem anyway because the lookup module is probably the right
place for autofs to handle special casing for incoming data and
probably should be developed as part of autofs.

The lookup modules are named <autofs library dir>/lookup_<source>.so
where <source> is the source name from the "automount:" line of
/etc/nssswitch.conf. So, to use sss as a source amounts to having
"automount: files sss" (to allow for local client overrides) or
just "automount: sss".

In nss_ldap-255 there is an example of an autofs interface with a
lookup module and an iterator. I think this is an example of the
model we should use. I'll post the source files but note they are
based on version 4 of autofs so the library interface is different
but the approach should be ok as long as it fits in with what's
doable in sssd.

Created attachment 532963 [details]
autofs lookup module from nss_ldap-255

Created attachment 532964 [details]
nss_ldap automount interface header file

Created attachment 532965 [details]
nss_ldap automount interface c source

Ok so in short we need to:
1. rename lookup_nssldap.c to lookup_sss.c & update for autofs 5 interface (Ian's responsibility probably)
2. modify libnss_sss.so library to implement all the getautomntent/setautomntent/getautomntbyname functions...(sssd team)

Nice.

I don't think we need to REPLACE the lookuo_nssldap.c. I think we can coexist.

From the looks of the interface, it's very nameservice-like, so I don't foresee any tremendous issues with implementing a very similar interface for SSSD.

In fact, it will probably make things a bit easier on my end, since I should be able to use my existing NSS socket protocol to talk to SSSD.

I'm not sure whether we'd want to modify libnss_sss.so.2 for this or just create a plugin library just for this purpose. Although, I suppose it could be possible to use libnss_sss.so.2 and just symlink it into the autofs plugin directory. I'll investigate that.

In either case, they'll share most of their code (which is just a forwarder to the SSSD where the processing happens).

(In reply to comment #29)
> I don't think we need to REPLACE the lookuo_nssldap.c. I think we can coexist.
Sure it will co-exist. Just a note that recent autofs has lookup_ldap and NOT lookup_nssldap. Ian provided this for us just as a nice starting point.

> I'm not sure whether we'd want to modify libnss_sss.so.2 for this or just
> create a plugin library just for this purpose. Although, I suppose it could be
> possible to use libnss_sss.so.2 and just symlink it into the autofs plugin
> directory. I'll investigate that.
No, it is not possible as autofs has a different interface. That's why we need lookup_sss.so in the autofs area in order to talk to libnss_sss.so.2 (or whichever different interface) - remember that interface autofs expect in its plugin directory is different to GLIBC NSS interface.
Ok theoretically it would be possible if you implemented the autofs native interface directly in the NSS module - but that is too ugly.
> 
> In either case, they'll share most of their code (which is just a forwarder to
> the SSSD where the processing happens).

We probably also need to disable automounter's internal caching/negative caching as double caching would probably lead to confusion/problems with debugging.

(In reply to comment #30)
> (In reply to comment #29)
> > I don't think we need to REPLACE the lookuo_nssldap.c. I think we can coexist.
> Sure it will co-exist. Just a note that recent autofs has lookup_ldap and NOT
> lookup_nssldap. Ian provided this for us just as a nice starting point.
> 
> > I'm not sure whether we'd want to modify libnss_sss.so.2 for this or just
> > create a plugin library just for this purpose. Although, I suppose it could be
> > possible to use libnss_sss.so.2 and just symlink it into the autofs plugin
> > directory. I'll investigate that.
> No, it is not possible as autofs has a different interface. That's why we need
> lookup_sss.so in the autofs area in order to talk to libnss_sss.so.2 (or
> whichever different interface) - remember that interface autofs expect in its
> plugin directory is different to GLIBC NSS interface.
> Ok theoretically it would be possible if you implemented the autofs native
> interface directly in the NSS module - but that is too ugly.


Well, I haven't seen the interface in the current model, but the files Ian supplied in this ticked show an interface made to look almost exactly like name-service would. So I was assuming that it wouldn't be TOO ugly.

> > 
> > In either case, they'll share most of their code (which is just a forwarder to
> > the SSSD where the processing happens).
> 
> We probably also need to disable automounter's internal caching/negative
> caching as double caching would probably lead to confusion/problems with
> debugging.

Yeah, if possible I'd like for the _init() function to be extended so that the plugin can return a boolean stating whether it expects to handle caching itself or it should be done internally by autofs.

 
> 
> Well, I haven't seen the interface in the current model, but the files Ian
> supplied in this ticked show an interface made to look almost exactly like
> name-service would. So I was assuming that it wouldn't be TOO ugly.
> 
Check Ian's comment #24 - this is the _real_ autofs interface it expects in its plugin area in /usr/lib/autofs/.....

(In reply to comment #29)
> I don't think we need to REPLACE the lookuo_nssldap.c. I think we can coexist.

There isn't any lookup_nssldap.c. This is an example that was
never actually used.

> 
> From the looks of the interface, it's very nameservice-like, so I don't foresee
> any tremendous issues with implementing a very similar interface for SSSD.

That's not surprising since autofs needs <name, value> pairs with
some special cases that need to be handled. We'll need to work
that out but I expect that most of that would be handled in the
lookup_sss.so that I will write.

> 
> In fact, it will probably make things a bit easier on my end, since I should be
> able to use my existing NSS socket protocol to talk to SSSD.
> 
> I'm not sure whether we'd want to modify libnss_sss.so.2 for this or just
> create a plugin library just for this purpose. Although, I suppose it could be
> possible to use libnss_sss.so.2 and just symlink it into the autofs plugin
> directory. I'll investigate that.

The sssd bits don't need to be in the autofs library directory. All
that's needed is some way for autofs configure to obtain the path to
the library and then decide on a name and the function entry points
we will use.

> 
> In either case, they'll share most of their code (which is just a forwarder to
> the SSSD where the processing happens).

I thought that be the case which just means most of the work will
be done in lookup_sss.so in autofs, which is fine. One difficulty
might be return codes but we'll need to work that out as we go.

Ian

(In reply to comment #33)
> The sssd bits don't need to be in the autofs library directory. All
> that's needed is some way for autofs configure to obtain the path to
> the library and then decide on a name and the function entry points
> we will use.
> 

In general, I think it's better for autofs to standardize on a location and we'll rely on SSSD's configure rules to put it in the correct place.

> > 
> > In either case, they'll share most of their code (which is just a forwarder to
> > the SSSD where the processing happens).
> 
> I thought that be the case which just means most of the work will
> be done in lookup_sss.so in autofs, which is fine. One difficulty
> might be return codes but we'll need to work that out as we go.

I'm not sure I understand this part. Maybe I'm making wrong assumptions.

My assumption is that we will discuss and define a set of entry points that autofs will want to talk to. Then, I'll implement those in a lookup_sss.so, built from the SSSD sources that I will install into the appropriate place. Then you need only add configuration options (probably just reading 'sss' from the automount line in nsswitch.conf) to tell autofs to use the library we provided.

Behind the scenes, all of the lookup logic should be inside the SSSD. Do I have that right?

(In reply to comment #34)
> > > 
> > > In either case, they'll share most of their code (which is just a forwarder to
> > > the SSSD where the processing happens).
> > 
> > I thought that be the case which just means most of the work will
> > be done in lookup_sss.so in autofs, which is fine. One difficulty
> > might be return codes but we'll need to work that out as we go.
> 
> I'm not sure I understand this part. Maybe I'm making wrong assumptions.
> 
> My assumption is that we will discuss and define a set of entry points that
> autofs will want to talk to. Then, I'll implement those in a lookup_sss.so,
> built from the SSSD sources that I will install into the appropriate place.
> Then you need only add configuration options (probably just reading 'sss' from
> the automount line in nsswitch.conf) to tell autofs to use the library we
> provided.

Not quite.

I'm recommending that lookup_sss.so be the place (as is the case
with all other autofs lookup modules) where autofs requests the
<key, value> pairs and does the autofs specific stuff like adding
to the internal map entry store, handle special cases etc.

The structure of the example above was essentially that where
the autofs lookup module would call an iterator to walk through
the <key, value> pairs or lookup a key by name. If that
implementation happens to be the slightly modified NSS Responder
then that would be great.

This approach allows autofs to handle the more difficult special
cases. For example, the Sun mixed case hack, where there can be
multiple instances of the key attribute in an LDAP entry with a
couple of special characters to indicate upper case when a case
insensitive schema is being used.

> 
> Behind the scenes, all of the lookup logic should be inside the SSSD. Do I have
> that right?

Yes, the actual LDAP retrieval needs to be done in sssd, for
sure. But I think we need this division of function to make
the code that is supported by each of us code that we are
familiar with that relates to the product within which it
is located.

Ian

So let me see if I have a clear picture of this now. We would produce a shared library that offers the following functions:

NSS_STATUS _nss_ldap_am_context_alloc(ldap_automount_context_t **pContext);
void _nss_ldap_am_context_free(ldap_automount_context_t **pContext);
NSS_STATUS _nss_ldap_am_context_init(const char *mapname, ldap_automount_context_t **pContext);

NSS_STATUS _nss_ldap_setautomntent(const char *mapname, void **context);
NSS_STATUS _nss_ldap_getautomntent(void *context, const char **key, const char **value,
				   char *buffer, size_t buflen, int *errnop);
NSS_STATUS _nss_ldap_endautomntent(void **context);
NSS_STATUS _nss_ldap_getautomntbyname_r(void *private, const char *key,
					const char **canon_key, const char **value,
					char *buffer, size_t buflen, int *errnop);



We would add this to our libnss_sss.so.2 library, so that it would forward the requests to the SSSD process to do the lookups and caching.

In turn, you (Ian) would create the lookup_sss.so module that would dlopen() or link against our libnss_sss.so.2 module. In lookup_sss.so, you would implement:

int lookup_init(const char *mapfmt,
                int argc, const char *const *argv,
                void **context);

int lookup_read_master(struct master *master,
                       time_t age, void *context);

int lookup_read_map(struct autofs_point *ap,
                    time_t age, void *context);

int lookup_mount(struct autofs_point *ap,
                 const char *name, int name_len,
                 void *context);

int lookup_done(void *context);


This would internally call the libnss_sss.so routines as needed. Do I have the separation correct?

(In reply to comment #36)
> So let me see if I have a clear picture of this now. We would produce a shared
> library that offers the following functions:
> 
> NSS_STATUS _nss_ldap_am_context_alloc(ldap_automount_context_t **pContext);
> void _nss_ldap_am_context_free(ldap_automount_context_t **pContext);
> NSS_STATUS _nss_ldap_am_context_init(const char *mapname,
> ldap_automount_context_t **pContext);
> 
> NSS_STATUS _nss_ldap_setautomntent(const char *mapname, void **context);
> NSS_STATUS _nss_ldap_getautomntent(void *context, const char **key, const char
> **value,
>        char *buffer, size_t buflen, int *errnop);
> NSS_STATUS _nss_ldap_endautomntent(void **context);
> NSS_STATUS _nss_ldap_getautomntbyname_r(void *private, const char *key,
>      const char **canon_key, const char **value,
>      char *buffer, size_t buflen, int *errnop);

Yes, but now would be the time if we want to change anything,
like function names or parameters passed, add or remove functions.

> 
> We would add this to our libnss_sss.so.2 library, so that it would forward the
> requests to the SSSD process to do the lookups and caching.

Yes, of course it is up to you how best to do that. If any
existing functions functions can be used rather than adding
functions that may be better for you. Although having separate
functions might be a good idea in case I need to ask for
specific return codes for certain cases, so we don't interfere
with existing users.

> 
> In turn, you (Ian) would create the lookup_sss.so module that would dlopen() or
> link against our libnss_sss.so.2 module. In lookup_sss.so, you would implement:
> 
> int lookup_init(const char *mapfmt,
>                 int argc, const char *const *argv,
>                 void **context);
> 
> int lookup_read_master(struct master *master,
>                        time_t age, void *context);
> 
> int lookup_read_map(struct autofs_point *ap,
>                     time_t age, void *context);
> 
> int lookup_mount(struct autofs_point *ap,
>                  const char *name, int name_len,
>                  void *context);
> 
> int lookup_done(void *context);

Yes, this is the internal autofs interface that I need to
implement to add a map source.

> 
> This would internally call the libnss_sss.so routines as needed. Do I have the
> separation correct?

Exactly right.

Remember that we are able to change any part of this in any way
that makes things easier or better for us, except for the above
autofs internal entry points that are required to add a new map
source to autofs. But since implementing that module is up to me
that shouldn't impose any restrictions on what the sssd code does.

Ideally I would be able to add something to configure to
discover the presence of sssd and the link library, etc.


Ian

(In reply to comment #37)
> 
> Remember that we are able to change any part of this in any way
> that makes things easier or better for us, except for the above
> autofs internal entry points that are required to add a new map
> source to autofs. But since implementing that module is up to me
> that shouldn't impose any restrictions on what the sssd code does.
> 

While were at it lets not forget that this all needs to
be thread safe and be able to handle multiple concurrent
thread accesses.

One thing that seems to be overlooked in shared library
implementations is that allocating thread specific data
objects within the library data segment is not safe when
a library in indirectly linked to an application via
another library.

Ian

(In reply to comment #36)
> So let me see if I have a clear picture of this now. We would produce a shared
> library that offers the following functions:
> 
> NSS_STATUS _nss_ldap_am_context_alloc(ldap_automount_context_t **pContext);
> void _nss_ldap_am_context_free(ldap_automount_context_t **pContext);
> NSS_STATUS _nss_ldap_am_context_init(const char *mapname,
> ldap_automount_context_t **pContext);
> 
> NSS_STATUS _nss_ldap_setautomntent(const char *mapname, void **context);
> NSS_STATUS _nss_ldap_getautomntent(void *context, const char **key, const char
> **value,
>        char *buffer, size_t buflen, int *errnop);
> NSS_STATUS _nss_ldap_endautomntent(void **context);
> NSS_STATUS _nss_ldap_getautomntbyname_r(void *private, const char *key,
>      const char **canon_key, const char **value,
>      char *buffer, size_t buflen, int *errnop);
> 
> 
> 

I don't think we need to be bound to this interface although we'll probably stay close.

The setautomtent/getautomtent/endautomtent iterator interface seems sane to me.

There probably should also be an init and shutdown function -- currently context_init and context_free are called from setautomntent and endautomntent respectively, I think having them separate and treating setautomtent as "iterator create and rewind" and endautomtend as "iterator destroy" is cleaner.

The prefix should obviously be _nss_sss_* to avoid name-clash with what lookup_nssldap.c uses.

Adding the interface to libnss_sss.so means polluting the library a little with functions that are not strictly name-service-switch related, but would allow us to reuse a fair amount of code and talk to the NSS responder socket easily, so ack on that. Do we need to stick to the enum nss_status return codes, though? Using errno would allow us to be a little more verbose on error reporting.

One thing that is still not very clear to me is how autofs does caching, both positive and negative. Is that something that user configures or just internal autofs mechanism?

With a little more detail..

1) SSSD would define a private data structure in libnss_sss:

struct automtent {
    const char *mapname;
    size_t cursor;
    /* Other data TBD as needed */
};

2) SSSD would define public interface in libnss_sss:

I decided to ditch the _nss prefix  - these are not name service switch calls and the name should not be confusing. Also, if, even though it is unlikely, the glibc folks added an autofs interface, the names might clash. The return value is errno_t instead of NSS_STATUS for pretty much the same reason.

a) iterator start that would allocate the private struct automtent and pass it out as context
    errno_t _sss_setautomntent(const char *mapname, void **context);
b) iterator end that would free the private struct automtent
    erno_t _sss_endautomntent(void **context);
c) function that returns the next (key,value) pair given a context
    errno_t _sss_getautomntent_r(void *context, const char **key, const char **value);

    In comparison with the original _nss_ldap_getautomntbyname_r it is missing the buffer and buflen parameters. I think we can just malloc the values instead of passing around a buffer.

    The functions can either guarantee that the values are allocated using malloc() or provide corresponding free() functions.

    Return values:
        0 - sucess, the key, value pointers point to allocated data
        ENOENT - no more (key,value) tuples in the map, pointers set to NULL.
        ..and other error codes as applicable (ENOEMEM when allocation fails etc.)

d) function that looks up data for a given key
    NSS_STATUS _sss_getautomntbyname_r(void *context, const char *key, const char **value, int *errnop);

    Same comment about the buffer as in _sss_getautomntent_r

    Return values:
        0 - sucess, the key, value pointers point to allocated data
        ENOENT - no more values, value pointer is NULL
        ..and other error codes as applicable

    I believe that in general, there can be duplicate keys in a map, right? At least in autofs5 user can define multiple direct maps. This may not be the case with the LDAP schema because it uses the key as RDN, but our interface should be generic.

3) Internally, SSSD would handle the requests from the autofs client similar to how it handles enumeration - a periodic task that dumps all data into the cache. The format of the cache is TBD but I assume we just copy the LDAP data pretty much verbatim.

4) Autofs would use the above interface in pretty much the same way lookup_nssldap.c uses the _nss_ldap_* interface.

5) Caching - still not clear. I would prefer that autofs would just relay all cache handling to sssd so there is a single point of configuration and get data from sssd with every request. Would that work with the internal autofs cache API?

(In reply to comment #40)
> With a little more detail..
> 
> 1) SSSD would define a private data structure in libnss_sss:
> 
> struct automtent {
>     const char *mapname;
>     size_t cursor;
>     /* Other data TBD as needed */
> };
> 
> 2) SSSD would define public interface in libnss_sss:
> 
> I decided to ditch the _nss prefix  - these are not name service switch calls
> and the name should not be confusing. Also, if, even though it is unlikely, the
> glibc folks added an autofs interface, the names might clash. The return value
> is errno_t instead of NSS_STATUS for pretty much the same reason.
> 
> a) iterator start that would allocate the private struct automtent and pass it
> out as context
>     errno_t _sss_setautomntent(const char *mapname, void **context);
> b) iterator end that would free the private struct automtent
>     erno_t _sss_endautomntent(void **context);
> c) function that returns the next (key,value) pair given a context
>     errno_t _sss_getautomntent_r(void *context, const char **key, const char
> **value);
> 

This all sounds fine to me.
Putting the (opaque) context arg first seems to be common
in many APIs although the internal autofs tend to put it
last. I'm not fused either way.

>     In comparison with the original _nss_ldap_getautomntbyname_r it is missing
> the buffer and buflen parameters. I think we can just malloc the values instead
> of passing around a buffer.
> 
>     The functions can either guarantee that the values are allocated using
> malloc() or provide corresponding free() functions.

Sorry, I'm not clear on this.
A pointer to a pointer usually indicates to me that the
called function allocates the space and the calling
function is then responsible for freeing it. How would
you like to handle it?

> 
>     Return values:
>         0 - sucess, the key, value pointers point to allocated data
>         ENOENT - no more (key,value) tuples in the map, pointers set to NULL.
>         ..and other error codes as applicable (ENOEMEM when allocation fails
> etc.)
> 
> d) function that looks up data for a given key
>     NSS_STATUS _sss_getautomntbyname_r(void *context, const char *key, const
> char **value, int *errnop);

NSS_STATUS seems out of place compared to the previous
functions, why not errno_t?

> 
>     Same comment about the buffer as in _sss_getautomntent_r
> 
>     Return values:
>         0 - sucess, the key, value pointers point to allocated data
>         ENOENT - no more values, value pointer is NULL
>         ..and other error codes as applicable
> 
>     I believe that in general, there can be duplicate keys in a map, right? At
> least in autofs5 user can define multiple direct maps. This may not be the case
> with the LDAP schema because it uses the key as RDN, but our interface should
> be generic.

Yes, direct map master map entries will all have the pseudo
key "/-" so there can be more than on entry with the same key.
All the schema that autofs can use seem to handle this ok.

There is also the Sun mixed case hack I have had to support
which will have multiple (but distinct) values in the key
attribute. I thought it best for the autofs module to deal
with working out what to search for in that case but haven't
got down to specifics yet.

> 
> 3) Internally, SSSD would handle the requests from the autofs client similar to
> how it handles enumeration - a periodic task that dumps all data into the
> cache. The format of the cache is TBD but I assume we just copy the LDAP data
> pretty much verbatim.

You could do that if you wish I guess but for clients with large maps
that will be a fair network overhead. I think we may need to talk
further about that but what you suggest would be a start.

> 
> 4) Autofs would use the above interface in pretty much the same way
> lookup_nssldap.c uses the _nss_ldap_* interface.
> 
> 5) Caching - still not clear. I would prefer that autofs would just relay all
> cache handling to sssd so there is a single point of configuration and get data
> from sssd with every request. Would that work with the internal autofs cache
> API?

It's best not to think of autofs as "caching" map data.

Due to complaints about map entries not being sufficiently
always up to date I ended up having to query the server for
every lookup and update the internal map data, at least that
is what's done for indirect maps.

Direct maps are read in their entirety at startup and upon
a HUP signal, since all the map entries are needed for
startup. I should actually be checking the "value" for the
"key" for every lookup for these too, in case it has changed,
but I currently don't.

For indirect mounts the map data held within autofs is
heavily used to detect if a map may have changed and if
so a readmap task gets scheduled which may not do anything
other than clean stale map data entries if the mount does
not use the "browse" option. If it does use the browse
option the entire map must be read (same at startup for
this case) because the mount point directories must be
created.

Bottom line is that autofs stores map data rather than
caches it and just happens to behave a bit like a cache.
The use of the map data is widespread within autofs so
I'll need to continue to store it for internal use.

OTOH, since I should be consulting the source on every
lookup for both map types that should leave sssd to take
care of caching as it sees fit.

One last thought, autofs can use any one of three different
schema and, by using the configuration, virtually any schema
can actually be used. But how to do that isn't documented
and is not likely used. autofs will dynamically work out
what schema is being used if a specific schema is not
specified in the configuration.

Ian


Ian

(In reply to comment #41)
> This all sounds fine to me.
> Putting the (opaque) context arg first seems to be common
> in many APIs although the internal autofs tend to put it
> last. I'm not fused either way.
> 

OK, I don't mind either. The sole consumer should be autofs so let's do it the autofs way and put the context as the last parameter.

> >     In comparison with the original _nss_ldap_getautomntbyname_r it is missing
> > the buffer and buflen parameters. I think we can just malloc the values instead
> > of passing around a buffer.
> > 
> >     The functions can either guarantee that the values are allocated using
> > malloc() or provide corresponding free() functions.
> 
> Sorry, I'm not clear on this.
> A pointer to a pointer usually indicates to me that the
> called function allocates the space and the calling
> function is then responsible for freeing it. How would
> you like to handle it?
> 

Sorry, I think my statement was ambiguous. I was proposing that the flow is something like:

_sss_setautomntent("mymap", &context);
while (_sss_getautomntent_r(context, &key, &value) == 0) {
  process(key, value);
  free(key);
  free(value);
}
_sss_endautomntent(context);

Would that work?

> > 
> >     Return values:
> >         0 - sucess, the key, value pointers point to allocated data
> >         ENOENT - no more (key,value) tuples in the map, pointers set to NULL.
> >         ..and other error codes as applicable (ENOEMEM when allocation fails
> > etc.)
> > 
> > d) function that looks up data for a given key
> >     NSS_STATUS _sss_getautomntbyname_r(void *context, const char *key, const
> > char **value, int *errnop);
> 
> NSS_STATUS seems out of place compared to the previous
> functions, why not errno_t?
> 

I just forgot to change the declaration when I copy-pasted the original functions. Here's what I had in mind:
errno_t _sss_getautomntbyname_r(void *context, const char *key, const **value);

> > 
> >     Same comment about the buffer as in _sss_getautomntent_r
> > 
> >     Return values:
> >         0 - sucess, the key, value pointers point to allocated data
> >         ENOENT - no more values, value pointer is NULL
> >         ..and other error codes as applicable
> > 
> >     I believe that in general, there can be duplicate keys in a map, right? At
> > least in autofs5 user can define multiple direct maps. This may not be the case
> > with the LDAP schema because it uses the key as RDN, but our interface should
> > be generic.
> 
> Yes, direct map master map entries will all have the pseudo
> key "/-" so there can be more than on entry with the same key.
> All the schema that autofs can use seem to handle this ok.
> 

OK, the reason I was asking was that if the key was unique, the API could have been simplified to:

errno_t _sss_getautomntbyname_r(const char *map, const char *key, const char **value);

But if there can be more values for a single key, then we need to iterate over them as well.

> There is also the Sun mixed case hack I have had to support
> which will have multiple (but distinct) values in the key
> attribute. I thought it best for the autofs module to deal
> with working out what to search for in that case but haven't
> got down to specifics yet.
> 
> > 
> > 3) Internally, SSSD would handle the requests from the autofs client similar to
> > how it handles enumeration - a periodic task that dumps all data into the
> > cache. The format of the cache is TBD but I assume we just copy the LDAP data
> > pretty much verbatim.
> 
> You could do that if you wish I guess but for clients with large maps
> that will be a fair network overhead. I think we may need to talk
> further about that but what you suggest would be a start.
> 

Right, there's certainly room for improvement, but I think that whatever we do on SSSD side shouldn't have any effect on the API we expose towards the autofs module.

While at the subject, how big are the large maps people are using? Are we talking a thousand entries, ten thousand..?

> > 
> > 4) Autofs would use the above interface in pretty much the same way
> > lookup_nssldap.c uses the _nss_ldap_* interface.
> > 
> > 5) Caching - still not clear. I would prefer that autofs would just relay all
> > cache handling to sssd so there is a single point of configuration and get data
> > from sssd with every request. Would that work with the internal autofs cache
> > API?
> 
> It's best not to think of autofs as "caching" map data.
> 

<snip>

Thank you for the detailed explanation. I guess I was confused by the function names when I saw cache_add/clean/etc in the lookup module.

> One last thought, autofs can use any one of three different
> schema and, by using the configuration, virtually any schema
> can actually be used. But how to do that isn't documented
> and is not likely used. autofs will dynamically work out
> what schema is being used if a specific schema is not
> specified in the configuration.
> 

I believe we are talking about what is documented in /etc/sysconfig/autofs? Which of the three is the default?

SSSD supports a mechanism to override attribute names (called a "map" in SSSD terminology), so users who are using esoteric schemas should be safe provided that they configure SSSD correctly. 

SSSD is not able to figure out the schema by itself at the moment. I think we should just pick a sensible default and document how the attributes can be modified.

(In reply to comment #42)
> (In reply to comment #41)
> > This all sounds fine to me.
> > Putting the (opaque) context arg first seems to be common
> > in many APIs although the internal autofs tend to put it
> > last. I'm not fused either way.
> > 
> 
> OK, I don't mind either. The sole consumer should be autofs so let's do it the
> autofs way and put the context as the last parameter.

Sound good.

> 
> > >     In comparison with the original _nss_ldap_getautomntbyname_r it is missing
> > > the buffer and buflen parameters. I think we can just malloc the values instead
> > > of passing around a buffer.
> > > 
> > >     The functions can either guarantee that the values are allocated using
> > > malloc() or provide corresponding free() functions.
> > 
> > Sorry, I'm not clear on this.
> > A pointer to a pointer usually indicates to me that the
> > called function allocates the space and the calling
> > function is then responsible for freeing it. How would
> > you like to handle it?
> > 
> 
> Sorry, I think my statement was ambiguous. I was proposing that the flow is
> something like:
> 
> _sss_setautomntent("mymap", &context);
> while (_sss_getautomntent_r(context, &key, &value) == 0) {
>   process(key, value);
>   free(key);
>   free(value);
> }
> _sss_endautomntent(context);
> 
> Would that work?

Sure. So the key and value are malloc()ed by _sss_getautomntent_r()
and the caller is responsible for freeing them, that's fine.

> 
> > > 
> > >     Return values:
> > >         0 - sucess, the key, value pointers point to allocated data
> > >         ENOENT - no more (key,value) tuples in the map, pointers set to NULL.
> > >         ..and other error codes as applicable (ENOEMEM when allocation fails
> > > etc.)
> > > 
> > > d) function that looks up data for a given key
> > >     NSS_STATUS _sss_getautomntbyname_r(void *context, const char *key, const
> > > char **value, int *errnop);
> > 
> > NSS_STATUS seems out of place compared to the previous
> > functions, why not errno_t?
> > 
> 
> I just forgot to change the declaration when I copy-pasted the original
> functions. Here's what I had in mind:
> errno_t _sss_getautomntbyname_r(void *context, const char *key, const **value);

Yep, thought that might be the case.

> 
> > > 
> > >     Same comment about the buffer as in _sss_getautomntent_r
> > > 
> > >     Return values:
> > >         0 - sucess, the key, value pointers point to allocated data
> > >         ENOENT - no more values, value pointer is NULL
> > >         ..and other error codes as applicable
> > > 
> > >     I believe that in general, there can be duplicate keys in a map, right? At
> > > least in autofs5 user can define multiple direct maps. This may not be the case
> > > with the LDAP schema because it uses the key as RDN, but our interface should
> > > be generic.
> > 
> > Yes, direct map master map entries will all have the pseudo
> > key "/-" so there can be more than on entry with the same key.
> > All the schema that autofs can use seem to handle this ok.
> > 
> 
> OK, the reason I was asking was that if the key was unique, the API could have
> been simplified to:
> 
> errno_t _sss_getautomntbyname_r(const char *map, const char *key, const char
> **value);
> 
> But if there can be more values for a single key, then we need to iterate over
> them as well.

Yep, I was thinking the, for the case where the key changes
but the value doesn't, the value would be resent along with
the changed key and where the value changes but the key doesn't
the key would be resent along with the changed value.

> 
> > There is also the Sun mixed case hack I have had to support
> > which will have multiple (but distinct) values in the key
> > attribute. I thought it best for the autofs module to deal
> > with working out what to search for in that case but haven't
> > got down to specifics yet.
> > 
> > > 
> > > 3) Internally, SSSD would handle the requests from the autofs client similar to
> > > how it handles enumeration - a periodic task that dumps all data into the
> > > cache. The format of the cache is TBD but I assume we just copy the LDAP data
> > > pretty much verbatim.
> > 
> > You could do that if you wish I guess but for clients with large maps
> > that will be a fair network overhead. I think we may need to talk
> > further about that but what you suggest would be a start.
> > 
> 
> Right, there's certainly room for improvement, but I think that whatever we do
> on SSSD side shouldn't have any effect on the API we expose towards the autofs
> module.

Sure.

> 
> While at the subject, how big are the large maps people are using? Are we
> talking a thousand entries, ten thousand..?

Yeah, that's difficult question to tie down so the answer has
to be vague.

How large a map has to be to be considered "large" is the first
question. Lets say more than around a thousand entries. Probably
the most common cases of large maps would be under ten thousand
but we know there are maps a large as fifty thousand entries which
is rare.

> 
> > > 
> > > 4) Autofs would use the above interface in pretty much the same way
> > > lookup_nssldap.c uses the _nss_ldap_* interface.
> > > 
> > > 5) Caching - still not clear. I would prefer that autofs would just relay all
> > > cache handling to sssd so there is a single point of configuration and get data
> > > from sssd with every request. Would that work with the internal autofs cache
> > > API?
> > 
> > It's best not to think of autofs as "caching" map data.
> > 
> 
> <snip>
> 
> Thank you for the detailed explanation. I guess I was confused by the function
> names when I saw cache_add/clean/etc in the lookup module.
> 
> > One last thought, autofs can use any one of three different
> > schema and, by using the configuration, virtually any schema
> > can actually be used. But how to do that isn't documented
> > and is not likely used. autofs will dynamically work out
> > what schema is being used if a specific schema is not
> > specified in the configuration.
> > 
> 
> I believe we are talking about what is documented in /etc/sysconfig/autofs?
> Which of the three is the default?

There isn't a default.
autofs calculates a query dn by trying each of the commonly
used schema and uses that schema for subsequent queries.

If I had to say what we should use as a default I'd say the
rfc2307-bis would be the one to use. There is also some issue
with the OIDs used by one of the schema, I can't remember which
now but the rfc2307-bis is used by other implementations so it
probably isn't that one.

> 
> SSSD supports a mechanism to override attribute names (called a "map" in SSSD
> terminology), so users who are using esoteric schemas should be safe provided
> that they configure SSSD correctly. 

Yep.

> 
> SSSD is not able to figure out the schema by itself at the moment. I think we
> should just pick a sensible default and document how the attributes can be
> modified.

No problem, as above.

Something that occurred to me just now while looking at the
autofs LDAP code is that as I work through this I'm going to
notice features that are present that I haven't mentioned and
special cases I have forgotten about.

The one I realized will be a challenge this morning is the
autofs feature whereby DNS SRV records are queried to obtain
a list of URIs to be used for map queries. Of course I have
plenty of example code but I'm not sure how we should handle
this. In short, if the LDAP URI is ldap[s]:///, ask DNS for
a list of servers. The [] meaning optional ssl.

I presume that sssd is paged LDAP query aware?

Ian

(In reply to comment #43)
> > > One last thought, autofs can use any one of three different
> > > schema and, by using the configuration, virtually any schema
> > > can actually be used. But how to do that isn't documented
> > > and is not likely used. autofs will dynamically work out
> > > what schema is being used if a specific schema is not
> > > specified in the configuration.
> > > 
> > 
> > I believe we are talking about what is documented in /etc/sysconfig/autofs?
> > Which of the three is the default?
> 
> There isn't a default.
> autofs calculates a query dn by trying each of the commonly
> used schema and uses that schema for subsequent queries.
> 
> If I had to say what we should use as a default I'd say the
> rfc2307-bis would be the one to use. There is also some issue
> with the OIDs used by one of the schema, I can't remember which
> now but the rfc2307-bis is used by other implementations so it
> probably isn't that one.
> 

I agree, the rfc2307bis schema seems to be prominenetly mentioned in Red Hat documentation as well.

It seems to me that in the autofs case, the schema differ in attributes and objectclass naming (and perhaps how the DNs are constructed on the server, which doesn't seem to be important for the autofs client). I haven't found any semantical differences between the schemas as there are between RFC2307 and RFC2307bis schemas for users. Is that correct?

> > 
> > SSSD supports a mechanism to override attribute names (called a "map" in SSSD
> > terminology), so users who are using esoteric schemas should be safe provided
> > that they configure SSSD correctly. 
> 
> Yep.
> 
> > 
> > SSSD is not able to figure out the schema by itself at the moment. I think we
> > should just pick a sensible default and document how the attributes can be
> > modified.
> 
> No problem, as above.
> 
> Something that occurred to me just now while looking at the
> autofs LDAP code is that as I work through this I'm going to
> notice features that are present that I haven't mentioned and
> special cases I have forgotten about.
> 
> The one I realized will be a challenge this morning is the
> autofs feature whereby DNS SRV records are queried to obtain
> a list of URIs to be used for map queries. Of course I have
> plenty of example code but I'm not sure how we should handle
> this. In short, if the LDAP URI is ldap[s]:///, ask DNS for
> a list of servers. The [] meaning optional ssl.
> 

SSSD is aware of SRV records. The configuration is slightly different - if the  ldap_uri option is either completely missing or contains a special "__srv__" value, then SSSD queries DNS for the list of servers.

I don't remember how ldaps is selected - perhaps based on the port number that is returned, I'm not sure from the top of my head. TLS support must be turned on explicitly for non-authentication lookups in the config file.

> I presume that sssd is paged LDAP query aware?
> 

It is since upstream 1.6. The paging support has also been backported to RHEL6.2.

> Ian

Thank you for discussing the design with me into such details. It is very helpful.

From auto.master man page:
" The format of a master map entry is:

       mount-point [map-type[,format]:]map [options]
...
       map-type
              Type of map used for this mount point.  The following are  valid
              map types:

              file,program, yp, hesiod
..."
Are we aiming to continue supporting all this? Personally, I can not imagine an LDAP master map pointing to the NIS based indirect maps....

(In reply to comment #45)
> Are we aiming to continue supporting all this? Personally, I can not imagine an
> LDAP master map pointing to the NIS based indirect maps....

No, SSSD will not support such "mixed" domains.

(In reply to comment #45)
> From auto.master man page:
> " The format of a master map entry is:
> 
>        mount-point [map-type[,format]:]map [options]
> ...
>        map-type
>               Type of map used for this mount point.  The following are  valid
>               map types:
> 
>               file,program, yp, hesiod
                                          and sss
> ..."
> Are we aiming to continue supporting all this? Personally, I can not imagine an
> LDAP master map pointing to the NIS based indirect maps....

autofs must keep this, partly for backward compatibility and
partly because specifying the type (or source) is the only
way to tell autofs to use a format (parser) other than the
sun parser, for example hesoid.

sss doesn't have any need to interpret the value part of the
key,value pair, which would be:

key: mount-point
value: [map-type[,format]:]map [options]

> 
> autofs must keep this, partly for backward compatibility and
> partly because specifying the type (or source) is the only
> way to tell autofs to use a format (parser) other than the
> sun parser, for example hesoid.
> 
> sss doesn't have any need to interpret the value part of the
> key,value pair, which would be:
> 
> key: mount-point
> value: [map-type[,format]:]map [options]

Ok I do not know if I understand.
Keys in auto.master point to indirect maps. That means that if sssd is not supposed to parse values in the auto.master, it will expect the full ldap DN of the indirect map as an argument in the interface functions like getautomntbyname, mentioned above right?

And if that is true then we are locking our doors in terms of sssd providing an interface to different backends (like NIS).

Sorry for messing around again, but this interests me.

(In reply to comment #48)
> > 
> > autofs must keep this, partly for backward compatibility and
> > partly because specifying the type (or source) is the only
> > way to tell autofs to use a format (parser) other than the
> > sun parser, for example hesoid.
> > 
> > sss doesn't have any need to interpret the value part of the
> > key,value pair, which would be:
> > 
> > key: mount-point
> > value: [map-type[,format]:]map [options]
> 
> Ok I do not know if I understand.
> Keys in auto.master point to indirect maps. That means that if sssd is not
> supposed to parse values in the auto.master, it will expect the full ldap DN of
> the indirect map as an argument in the interface functions like
> getautomntbyname, mentioned above right?
> 

No, SSSD will still expect a plain string in the functions such as getautomt and will limit the search based on this string and the objectclass of the entry it looks for. 

A search for mountpoint "foo" in map "auto.home" might use filter "(&(automountKey=foo)(objectclass=automount))" and be based at "automountMapName=auto.home,$DC" for example.

SSSD will return the map value verbatim to autofs which will handle all the semantic details.

> A search for mountpoint "foo" in map "auto.home" might use filter
> "(&(automountKey=foo)(objectclass=automount))" and be based at
> "automountMapName=auto.home,$DC" for example.
The thing is, that you can not expect to have auto.home in "automountMapName=auto.home,$DC". The location (DN) of the auto.home map is stored as a value of /home key in auto.master.
For example in auto.master there could be a single key called /home
with automountInformation attribute="automount=/home,OU=autofs_for_site1,DC=example,DC=com" so you have to parse auto.parse in order to stay consistent with what automount does in its current lookup_ldap.so module.
Hence my question earlier....

(In reply to comment #50)
> > A search for mountpoint "foo" in map "auto.home" might use filter
> > "(&(automountKey=foo)(objectclass=automount))" and be based at
> > "automountMapName=auto.home,$DC" for example.
> The thing is, that you can not expect to have auto.home in
> "automountMapName=auto.home,$DC". The location (DN) of the auto.home map is
> stored as a value of /home key in auto.master.
> For example in auto.master there could be a single key called /home
> with automountInformation
> attribute="automount=/home,OU=autofs_for_site1,DC=example,DC=com" so you have
> to parse auto.parse in order to stay consistent with what automount does in its
> current lookup_ldap.so module.
> Hence my question earlier....

That's a keen observation.

We are talking about the "map" component only of the entry as
given in comment 45 and that is btw a master map entry. Map
entries are themselves quite similar.

Maybe you've forgotten that there is a context passed in the
calls. That context needs to store such things as the base DN
and should probably be opaque to autofs or at least something
it leaves alone.

The base DN would need to be calculated when initializing the
interface and remembered in the context. Then the map name
can be a trivial map name or a full DN. If it is a full DN
that would override the stored base DN in the context.

Later when map keys are requested or a map is iterated the map
name passed could also be either a trivial name or a full DN,
again a full DN would override the base DN stored in the
context.

I don't believe it is sensible to allow partial DNs other
than a map name alone.

In case what I'm saying isn't quite clear, let me add that
when I say base DN above I actually mean the search base DN
used along with the schema bits to form the search filter,
as the schema is already known. And, from autofs point of
view, it is only valid to use a single schema to store map
information, once chosen (perhaps we require it to be
configured, not calculated as autofs does) everything must
be stored using the same schema.

Not sure if that answers the question you raised and it also
begs the question of how to handle multiple search base DNs?

Ian

hi Ian,
Could you give me some test strategy about the feature? 
And do you know any document about how to configure sssd+autofs in RHEL? thanks.

(In reply to comment #52)
> hi Ian,
> Could you give me some test strategy about the feature? 

We can simply run our existing tests, there's plenty of LDAP
usage in them.

> And do you know any document about how to configure sssd+autofs in RHEL?

It hasn't been done yet so there is no documentation but,
from an autofs POV adding sss as a map source to nsswitch.conf
should be all that is needed.

We'll need to consult the person doing the sssd bits when it's
done for that part and work out how we can change the existing
tests to allow us to specify sss as the source and setup sssd
then run the tests. Probably not quite as simple as that but
it's a start.

(In reply to comment #53)
> (In reply to comment #52)
> > hi Ian,
> > Could you give me some test strategy about the feature? 
> 
> We can simply run our existing tests, there's plenty of LDAP
> usage in them.
> 
> > And do you know any document about how to configure sssd+autofs in RHEL?
> 
> It hasn't been done yet so there is no documentation but,
> from an autofs POV adding sss as a map source to nsswitch.conf
> should be all that is needed.
> 
> We'll need to consult the person doing the sssd bits when it's
> done for that part and work out how we can change the existing
> tests to allow us to specify sss as the source and setup sssd
> then run the tests. Probably not quite as simple as that but
> it's a start.

Thank you, qa_ack+ now.

Created attachment 562152 [details]
Patch - add sss lookup module

Created attachment 562153 [details]
Patch - teach automount about sss source

There is still a little more to do wrt special case handling
but the previous two patches add the core functionality to
use sss as a map source, provided the sss library is detected
during the build configure step.

Created attachment 562156 [details]
Patch - add sss lookup module

Created attachment 562158 [details]
Patch - teach automount about sss source

Verified that the sssd autofs library (from the current RHEL
sssd update) was seen during the build of autofs revision 44
on all architectures. This is required for sss support to be
built into autofs.

Created attachment 575059 [details]
Patch - fix sss wildcard match

Created attachment 575244 [details]
Patch - fix dlopen() error handling in sss module

Created attachment 575294 [details]
Patch - fix configure string length tests

(In reply to comment #69)
> Verified that the sssd autofs library (from the current RHEL
> sssd update) was seen during the build of autofs revision 44
> on all architectures. This is required for sss support to be
> built into autofs.

Now that the minor error in configure have been fixed and the
requirement of sssd >= 1.8.0-5 added I see that that the build
environment does not have sssd at 1.8.0-5 or above even though
it is included in the 6.3 compose, *sigh*.

(In reply to comment #76)
> Now that the minor error in configure have been fixed and the
> requirement of sssd >= 1.8.0-5 added I see that that the build
> environment does not have sssd at 1.8.0-5 or above even though
> it is included in the 6.3 compose, *sigh*.

Yes, you need to send an email to rel-eng and ask them to include the current version of SSSD in the buildroot. By default, the buildroot only contains the most recently SHIPPED versions of packages, not the in-development ones.

I'm having a problem with the autofs <-> sssd interface.

I keep getting an EINVAL return from _sss_getautomntbyname_r().
I can't see anything wrong with what I'm doing and I've been
looking at it for a while. It did work at one point, of course,
since I was able to successfully test it.

I get the same behaviour on RHEL and Fedora 16.
I'm using the current revision on RHEL, sssd-1.8.0-22.el6 I think
and on Fedora I have sssd-1.8.1-7.fc16.

I'm able to read the master map and setup the automounts but I
can't lookup a key.

Here is a log fragment from /var/log/sssd/sssd_autofs.log where
I have waited a little while and re-tried the call:

(Thu Apr 12 18:18:24 2012) [sssd[autofs]]
  [sss_autofs_cmd_setautomntent_done] (0x2000): setautomntent done
(Thu Apr 12 18:18:24 2012) [sssd[autofs]]
  [sss_autofs_cmd_setautomntent_done] (0x0400): setautomntent found data
(Thu Apr 12 18:18:24 2012) [sssd[autofs]]
  [sss_dp_req_destructor] (0x0400): Deleting request:
      0xb7778bb0:0:auto.indirect@ipaldap]
(Thu Apr 12 18:18:24 2012) [sssd[autofs]]
  [sss_autofs_cmd_getautomntbyname] (0x2000): sss_autofs_cmd_getautomntbyname
(Thu Apr 12 18:18:24 2012) [sssd[autofs]]
  [client_recv] (0x0200): Client disconnected!
(Thu Apr 12 18:18:24 2012) [sssd[autofs]]
  [get_client_cred] (0x4000): Client creds: euid[0] egid[0] pid[26333].
(Thu Apr 12 18:18:24 2012) [sssd[autofs]]
  [accept_fd_handler] (0x0100): Client connected!
(Thu Apr 12 18:18:24 2012) [sssd[autofs]]
  [sss_cmd_get_version] (0x0200): Received client version [1].
(Thu Apr 12 18:18:24 2012) [sssd[autofs]]
  [sss_cmd_get_version] (0x0200): Offered version [1].
(Thu Apr 12 18:18:24 2012) [sssd[autofs]]
  [sss_autofs_cmd_getautomntbyname] (0x2000): sss_autofs_cmd_getautomntbyname
(Thu Apr 12 18:18:24 2012) [sssd[autofs]]
  [client_recv] (0x0200): Client disconnected!
(Thu Apr 12 18:18:24 2012) [sssd[autofs]]
  [get_client_cred] (0x4000): Client creds: euid[0] egid[0] pid[26333].
(Thu Apr 12 18:18:24 2012) [sssd[autofs]]
  [accept_fd_handler] (0x0100): Client connected!
(Thu Apr 12 18:18:24 2012) [sssd[autofs]]
  [sss_cmd_get_version] (0x0200): Received client version [1].
(Thu Apr 12 18:18:24 2012) [sssd[autofs]]
  [sss_cmd_get_version] (0x0200): Offered version [1].
(Thu Apr 12 18:18:24 2012) [sssd[autofs]]
  [sss_autofs_cmd_endautomntent] (0x0400): endautomntent called
(Thu Apr 12 18:18:25 2012) [sssd[autofs]]
  [client_recv] (0x0200): Client disconnected!

Any ideas?

I've fixed one getbyname bug upstream (0a78bac242d0172a858d6b65402276aa734a4671), that fix is not in any of the released packages. I'll build a scratch RPM with that patch, so you can test.

(In reply to comment #79)
> I've fixed one getbyname bug upstream
> (0a78bac242d0172a858d6b65402276aa734a4671), that fix is not in any of the
> released packages. I'll build a scratch RPM with that patch, so you can test.

I noticed a couple of other things while I was looking around.

The length validation at the top of the getbyname function
checks the mapname twice, I suspect the second should be
the passed in key.

The other observation is the allowed length of a key.

While it's true that autofs itself only allows NAME_MAX for
key length and we haven't had reports of a problem with it,
it's probably too short. Indirect maps have keys that are
essentially a directory name but direct maps have keys that
are full paths so the maximum key length should probably be
PATH_MAX or 1024, whichever is greater.

(In reply to comment #80)
> (In reply to comment #79)
> > I've fixed one getbyname bug upstream
> > (0a78bac242d0172a858d6b65402276aa734a4671), that fix is not in any of the
> > released packages. I'll build a scratch RPM with that patch, so you can test.
> 
> I noticed a couple of other things while I was looking around.
> 
> The length validation at the top of the getbyname function
> checks the mapname twice, I suspect the second should be
> the passed in key.
> 

Yeah, that's the fix I was mentioning :-) 

> The other observation is the allowed length of a key.
> 
> While it's true that autofs itself only allows NAME_MAX for
> key length and we haven't had reports of a problem with it,
> it's probably too short. Indirect maps have keys that are
> essentially a directory name but direct maps have keys that
> are full paths so the maximum key length should probably be
> PATH_MAX or 1024, whichever is greater.

Thank you, I'll fix this.

Created attachment 577220 [details]
Patch to check calling endmntent then setmntent and performing a ny name query

Adding this patch to sssd and performing an rpmbuild -bc sssd.spec
yields a test program that functions.

Created attachment 577221 [details]
Patch to check calling endmntent then setmntent and performing a ny name query using libsss_autofs as a shared library

Add file containing /usr/lib/sssd/modules to /etc/ld.so.conf.d
and call ldconfig.

Make a copy of the test program and sss_autofs_private.h then
apply this patch and compile with:

gcc -o autofs_test_client \
 autofs_test_client.c -lpopt \
 -L/usr/lib/sssd/modules -lsss_autofs

[raven@perseus sssd]$ ./autofs_test_client auto.indirect
setautomntent done for auto.indirect
key: bar		value: zeus.themaw.net:/export/bar
key: baz		value: zeus.themaw.net:/export/baz
key: foo		value: zeus.themaw.net:/export/foo
key: /		value: zeus.themaw.net:/export/&
getautomntbyname_r failed [22]: Invalid argument
endautomntent done for auto.indirect

whereas the patched test program built within sssd, apparently
linked directly with sss_autofs.c produces this:

[raven@perseus sssd-1.8.2]$ ./autofs_test_client auto.indirect
setautomntent done for auto.indirect
key: bar		value: zeus.themaw.net:/export/bar
key: baz		value: zeus.themaw.net:/export/baz
key: foo		value: zeus.themaw.net:/export/foo
key: /		value: zeus.themaw.net:/export/&
get key bar in map:		value: zeus.themaw.net:/export/bar
endautomntent done for auto.indirect

Interesting, I can't reproduce the problem with git HEAD of SSSD. Initially when I saw /usr/lib I was suspecting an architecture-dependent bug, but everything seems to be working fine on both i686 and x86_64.

What exact version of SSSD was this? Can you add "debug_level=10" into the [autofs] section of sssd.conf, restart sssd and check if sss_autofs_cmd_getautomntbyname appear in the sssd logs? In particular, autofs logs into /var/log/sssd/sssd_autofs.log _sss_getautomntbyname_r() can also return EINVAL if either key name of the context are NULL, can you add one more printf into the test program to see if that is the case?

Sorry, normally I would be able to do these all myself, but I was unable to reproduce the issue on my test machine.

(In reply to comment #87)
> 
> What exact version of SSSD was this? Can you add "debug_level=10" into the
> [autofs] section of sssd.conf, restart sssd and check if
> sss_autofs_cmd_getautomntbyname appear in the sssd logs? In particular, autofs
> logs into /var/log/sssd/sssd_autofs.log _sss_getautomntbyname_r() can also
> return EINVAL if either key name of the context are NULL, can you add one more
> printf into the test program to see if that is the case?

This particular example is sssd-1.8.2-10 compiled on F16, ie. against
libldb-1.1.0 instead of libldb-1.1.4.

When you say git HEAD what does that mean, Fedora, RHEL or upstream.
I will try this against Rawhide sssd HEAD and current RHEL versions
and see what happens.

(In reply to comment #87)
> Interesting, I can't reproduce the problem with git HEAD of SSSD. Initially
> when I saw /usr/lib I was suspecting an architecture-dependent bug, but
> everything seems to be working fine on both i686 and x86_64.

Now that I try again, after adding a couple of printfs, I see
I had a mistake in my patch against the test program. It seems
that against this version of sssd I'm actually seeing the second
problem I had not yet reported.

Created attachment 577604 [details]
Patch to check calling endmntent then setmntent and performing a ny name query using libsss_autofs as a shared library (rev 2)

Updated patch for independent autofs test program used against
sssd-1.8.2-10 on F16.

Now I get:

[raven@perseus sssd]$ ./autofs_test_client auto.indirect
setautomntent done for auto.indirect
key: bar		value: zeus.themaw.net:/export/bar
key: baz		value: zeus.themaw.net:/export/baz
key: foo		value: zeus.themaw.net:/export/foo
key: /		value: zeus.themaw.net:/export/&
get key "bar" in map:		value: zeus.themaw.net:/export/bar
getautomntbyname_r for "/" failed [22]: Invalid argument
endautomntent done for auto.indirect

So the lookup of key "bar" does work in this sssd revision but
the autofs wildcard lookup of key "/" doesn't.

Thoughts?

And I see the same thing on RHEL6.3-20120413.n.1:

[root@dell-pem710-02 sssd]# ./autofs_test_client auto.indirect
setautomntent done for auto.indirect
key: bar		value: dell-per610-02.lab.bos.redhat.com:/export/bar
key: baz		value: dell-per610-02.lab.bos.redhat.com:/export/baz
key: foo		value: dell-per610-02.lab.bos.redhat.com:/export/foo
key: /		value: dell-per610-02.lab.bos.redhat.com:/export/&
get key "bar" in map:		value: dell-per610-02.lab.bos.redhat.com:/export/bar
getautomntbyname_r for "/" failed [22]: Invalid argument
endautomntent done for auto.indirect
[root@dell-pem710-02 sssd]# 
[root@dell-pem710-02 sssd]# 
[root@dell-pem710-02 sssd]# rpm -q sssd
sssd-1.8.0-22.el6.i686
[root@dell-pem710-02 sssd]# uname -a
Linux dell-pem710-02.rhts.eng.bos.redhat.com 2.6.32-262.el6.i686 #1 SMP Sun Apr 8 18:37:45 EDT 2012 i686 i686 i386 GNU/Linux
[root@dell-pem710-02 sssd]# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 6.3 Beta (Santiago)

(In reply to comment #89)
> When you say git HEAD what does that mean, Fedora, RHEL or upstream.
> I will try this against Rawhide sssd HEAD and current RHEL versions
> and see what happens.

upstream git

Any chance this will get in Fedora 16 any time soon? I would like to give it a try - it is still a long time to wait for RHEL 6.3.

(In reply to comment #100)
> Any chance this will get in Fedora 16 any time soon? I would like to give it a
> try - it is still a long time to wait for RHEL 6.3.

It's available in F17 beta right now.

Test basic autofs functionality using sss as a source passed after update sssd
to sssd-1.8.0-23.el6 by manual run:
[root@sun-cp3270-01 sssd]# rpm -qa|grep sssd
sssd-debuginfo-1.8.0-22.el6.i686
sssd-client-1.8.0-23.el6.i686
sssd-1.8.0-23.el6.i686
[root@sun-cp3270-01 sssd]# rpm -qa|grep autofs
autofs-5.0.5-54.el6.i686
autofs-debuginfo-5.0.5-54.el6.i686
autofs-CoreOS-autofs-bugzillas-1.1-23.noarch
[root@sun-cp3270-01 sssd]# uname -a
Linux sun-cp3270-01.rhts.eng.bos.redhat.com 2.6.32-262.el6.i686 #1 SMP Sun Apr
8 18:37:45 EDT 2012 i686 i686 i386 GNU/Linux

[root@sun-cp3270-01 bugzillas]# service sssd start
[  OK  ] sssd: [  OK  ]
[root@sun-cp3270-01 bugzillas]# ps -ef|grep sssd
root     10014     1  0 02:30 ?        00:00:00 /usr/sbin/sssd -f -D
root     10016 10014  0 02:30 ?        00:00:00 /usr/libexec/sssd/sssd_be
--domain ipaldap --debug-to-files
root     10017 10014  0 02:30 ?        00:00:00 /usr/libexec/sssd/sssd_autofs
--debug-to-files
root     10019  9044  0 02:30 pts/0    00:00:00 grep sssd
[root@sun-cp3270-01 sssd]# ./autofs_test_client auto.indirect
setautomntent done for auto.indirect
key: bar  value: dell-pem710-02.rhts.eng.bos.redhat.com:/export/bar
key: baz  value: dell-pem710-02.rhts.eng.bos.redhat.com:/export/baz
key: foo  value: dell-pem710-02.rhts.eng.bos.redhat.com:/export/foo
key: /  value: dell-pem710-02.rhts.eng.bos.redhat.com:/export/&
get key "bar" in map:  value:
dell-pem710-02.rhts.eng.bos.redhat.com:/export/bar
get key "/" in map:  value: dell-pem710-02.rhts.eng.bos.redhat.com:/export/&
endautomntent done for auto.indirect

Client:
Running test for bz683523 using autofs-5.0.5-54.el6.i686
result_server not set, assuming developer mode.
Setting sun-cp3270-01.rhts.eng.bos.redhat.com to state bz683523-READY
result_server not set, assuming developer mode.
Enter STATE:STATE:etc. when the following machines
 ['dell-pem710-02.rhts.eng.bos.redhat.com']
are in one of these states: ['bz683523-READY']
[  OK  ] sssd: [  OK  ]
Stopping automount: [  OK  ]
Starting automount: [  OK  ]
result_server not set, assuming developer mode.
Setting sun-cp3270-01.rhts.eng.bos.redhat.com to state bz683523-DONE
Stopping automount: [  OK  ]
Stopping sssd: [  OK  ]
Test bz683523: SUCCESS
bugzillas/bz683523/client result: PASS

Server:
result_server not set, assuming developer mode.
Enter STATE:STATE:etc. when the following machines
 ['sun-cp3270-01.rhts.eng.bos.redhat.com']
are in one of these states: ['bz683523-READY']
Shutting down NFS mountd: [  OK  ]
Shutting down NFS daemon: [  OK  ]
Shutting down NFS quotas: [  OK  ]
Shutting down NFS services:  [  OK  ]
Starting NFS services:  exportfs: duplicated export entries:
exportfs:  *:/export/foo
exportfs:  *:/export/foo
exportfs: duplicated export entries:
exportfs:  *:/export/bar
exportfs:  *:/export/bar
exportfs: duplicated export entries:
exportfs:  *:/export/baz
exportfs:  *:/export/baz
exportfs: duplicated export entries:
exportfs:  *:/export/zoo
exportfs:  *:/export/zoo
[  OK  ]
Starting NFS quotas: [  OK  ]
Starting NFS daemon: [  OK  ]
Starting NFS mountd: [  OK  ]
Stopping slapd: [FAILED]
Setting up ldap client to point at dell-pem710-02.rhts.eng.bos.redhat.com
bdb_monitor_db_open: monitoring disabled; configure monitor database to enable
bdb_monitor_db_open: monitoring disabled; configure monitor database to enable
bdb_monitor_db_open: monitoring disabled; configure monitor database to enable
bdb_monitor_db_open: monitoring disabled; configure monitor database to enable
Starting slapd: [  OK  ]
result_server not set, assuming developer mode.
Setting dell-pem710-02.rhts.eng.bos.redhat.com to state bz683523-READY
result_server not set, assuming developer mode.
Enter STATE:STATE:etc. when the following machines
 ['sun-cp3270-01.rhts.eng.bos.redhat.com']
are in one of these states: ['bz683523-DONE']
Stopping slapd: [  OK  ]
Shutting down NFS mountd: [  OK  ]
Shutting down NFS daemon: [  OK  ]
Shutting down NFS quotas: [  OK  ]
Shutting down NFS services:  [  OK  ]
Starting NFS services:  [  OK  ]
Starting NFS quotas: [  OK  ]
Starting NFS daemon: [  OK  ]
Starting NFS mountd: [  OK  ]
Test bz683523: SUCCESS
bugzillas/bz683523/server result: PASS

Run rhts testcase also passed:
https://beaker.engineering.redhat.com/jobs/222982
https://beaker.engineering.redhat.com/jobs/222980
https://beaker.engineering.redhat.com/jobs/222978

Tried this on F17 (eventually, had to fight with bugs #811518 and #811375), followed manual on http://fedoraproject.org/wiki/Features/SSSDAutoFSSupport, and I am unable to make it working:
1. 'automount -m' does not show my maps
2. sssd_autofs.log files does not show even any attempt to search for auto.master map.

Am I doing anything wrong here?

(In reply to comment #103)
> Tried this on F17 (eventually, had to fight with bugs #811518 and #811375),
> followed manual on http://fedoraproject.org/wiki/Features/SSSDAutoFSSupport,
> and I am unable to make it working:
> 1. 'automount -m' does not show my maps
> 2. sssd_autofs.log files does not show even any attempt to search for
> auto.master map.
> 
> Am I doing anything wrong here?

Can you run automount -f -d and paste the output?

Also, you may need to make sure the SSSD starts *before* automounter of F-17. I need to check the systemd unit files whether they ensure the correct order automatically.

using kernel protocol version 5.02
lookup_nss_read_master: reading master sss auto.master
lookup_init:98: lookup(sss): dlopen: Success
lookup_nss_read_master: auto.master not found, replacing '.' with '_'
lookup_init:98: lookup(sss): dlopen: Success
no mounts in table

Everything looks fine on the automount side - it just seems to me that sssd never gets request for auto.master - or at least from the logs, I can not see anything.
My config:
[sssd]
services = nss, pam, autofs
config_file_version = 2
debug_level = 17

domains = default
[nss]

[pam]

[domain/example.com]
cache_credentials = True
ipa_domain = example.com
id_provider = ipa
auth_provider = ipa
access_provider = ipa
chpass_provider = ipa
ipa_server = _srv_, polaris.example.com

[domain/default]
debug_level = 17
ldap_id_use_start_tls = False
ldap_search_base = dc=dublin,dc=ad,dc=s3group,dc=com
id_provider = ldap
auth_provider = krb5
chpass_provider = krb5
cache_credentials = True
ldap_sasl_authid = DORADO3$@DUBLIN.AD.S3GROUP.COM
dns_discovery_domain = Prague._sites.dublin.ad.s3group.com
krb5_realm = DUBLIN.AD.S3GROUP.COM
# interval (in seconds) to renew Kerberos TGTs
krb5_renew_interval = 3600
# request renewable Kerberos tickets
krb5_renewable_lifetime = 30d
krb5_canonicalize = False
ldap_sasl_mech = GSSAPI
ldap_user_object_class = user
ldap_group_object_class = group
ldap_user_home_directory = unixHomeDirectory
ldap_tls_cacertdir = /etc/openldap/cacerts
ldap_schema = rfc2307bis
# ldap_account_expire_policy = ad
# ldap_uri = ldap://dcpra1

[autofs]
debug_level = 10
ldap_autofs_search_base = CN=prague,CN=NIS,DC=dublin,DC=ad,DC=s3group,DC=com

(In reply to comment #105)
> using kernel protocol version 5.02
> lookup_nss_read_master: reading master sss auto.master
> lookup_init:98: lookup(sss): dlopen: Success
> lookup_nss_read_master: auto.master not found, replacing '.' with '_'
> lookup_init:98: lookup(sss): dlopen: Success
> no mounts in table
> 
> Everything looks fine on the automount side - it just seems to me that sssd
> never gets request for auto.master - or at least from the logs, I can not see
> anything.

Is the sssd_autofs.log empty?

> ldap_autofs_search_base = CN=prague,CN=NIS,DC=dublin,DC=ad,DC=s3group,DC=com

This option belongs to the [domain] section, not autofs provider, because different domains might need different search bases.

Can you also paste a single map entry from the autofs tree? Feel free to sanitize it, I'm mostly interested in whether the schema is correct for your setup.

Agreed with Jakub that automounter does not seem to communicate with sssd in Fedora 17 beta

Before we get carried away let me update autofs with my latest
changes. I see the build today didn't detect sss autofs library
so, for a start, I need to fix that.

Can you please try autofs rev 16 located:
https://koji.fedoraproject.org/koji/buildinfo?buildID=316567

(In reply to comment #107)
> Agreed with Jakub that automounter does not seem to communicate with sssd in
> Fedora 17 beta

Ondrej, make sure the package libsss_autofs is installed:
# yum install libsss_autofs

Fine, I still think sssd should be shipped with libsss_autofs automatically - no options here.
Installing it did not help, anyway. What made things moving is the updated autofs package Ian provided in his last comment.

But it is still not working. Test scenario:

1.
auto.master:

/proj    sss:auto.proj

2.
$ cd /proj/tmp

3. cat /var/log/sssd/sssd_default.log

(Thu May  3 14:49:25 2012) [sssd[be[default]]] [sdap_get_automntmap_next_base] (0x0400): Searching for automount maps with base [CN=prague,CN=NIS,DC=dublin,DC=ad,DC=s3group,DC=com]
(Thu May  3 14:49:25 2012) [sssd[be[default]]] [sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [(&(nisMapName=auto.proj)(objectclass=nisMap))][CN=prague,CN=NIS,DC=dublin,DC=ad,DC=s3group,DC=com].
(Thu May  3 14:49:25 2012) [sssd[be[default]]] [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [objectClass]
(Thu May  3 14:49:25 2012) [sssd[be[default]]] [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [nisMapName]
(Thu May  3 14:49:25 2012) [sssd[be[default]]] [sdap_get_generic_ext_step] (0x2000): ldap_search_ext called, msgid = 15
(Thu May  3 14:49:25 2012) [sssd[be[default]]] [sdap_process_result] (0x2000): Trace: sh[0x7f161e5e4b20], connected[1], ops[0x7f161e5dd830], ldap[0x7f161e59e510]
(Thu May  3 14:49:25 2012) [sssd[be[default]]] [sdap_process_message] (0x4000): Message type: [LDAP_RES_SEARCH_ENTRY]
(Thu May  3 14:49:25 2012) [sssd[be[default]]] [sdap_parse_entry] (0x4000): OriginalDN: [CN=auto.proj,CN=prague,CN=NIS,DC=dublin,DC=ad,DC=s3group,DC=com].
(Thu May  3 14:49:25 2012) [sssd[be[default]]] [sdap_process_result] (0x2000): Trace: sh[0x7f161e5e4b20], connected[1], ops[0x7f161e5dd830], ldap[0x7f161e59e510]
(Thu May  3 14:49:25 2012) [sssd[be[default]]] [sdap_process_message] (0x4000): Message type: [LDAP_RES_SEARCH_RESULT]
(Thu May  3 14:49:25 2012) [sssd[be[default]]] [sdap_get_generic_ext_done] (0x0400): Search result: Success(0), no errmsg set
(Thu May  3 14:49:25 2012) [sssd[be[default]]] [sdap_get_automntmap_process] (0x0400): Search for autofs maps, returned 1 results.
(Thu May  3 14:49:25 2012) [sssd[be[default]]] [sdap_get_automntmap_process] (0x2000): Processing autofs maps
(Thu May  3 14:49:25 2012) [sssd[be[default]]] [automntmaps_process_members_send] (0x0400): Examining autofs map [CN=auto.proj,CN=prague,CN=NIS,DC=dublin,DC=ad,DC=s3group,DC=com]
(Thu May  3 14:49:25 2012) [sssd[be[default]]] [automntmaps_process_members_next_base] (0x0400): Searching for automount map entries with base [CN=prague,CN=NIS,DC=dublin,DC=ad,DC=s3group,DC=com]
(Thu May  3 14:49:25 2012) [sssd[be[default]]] [sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [(&(cn=*)(objectclass=nisObject))][CN=auto.proj,CN=prague,CN=NIS,DC=dublin,DC=ad,DC=s3group,DC=com].
(Thu May  3 14:49:25 2012) [sssd[be[default]]] [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [objectClass]
(Thu May  3 14:49:25 2012) [sssd[be[default]]] [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [cn]
(Thu May  3 14:49:25 2012) [sssd[be[default]]] [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [nisMapEntry]
(Thu May  3 14:49:25 2012) [sssd[be[default]]] [sdap_get_generic_ext_step] (0x2000): ldap_search_ext called, msgid = 16
(Thu May  3 14:49:25 2012) [sssd[be[default]]] [sdap_process_result] (0x2000): Trace: sh[0x7f161e5e4b20], connected[1], ops[0x7f161e5d2590], ldap[0x7f161e59e510]
(Thu May  3 14:49:25 2012) [sssd[be[default]]] [sdap_process_result] (0x2000): Trace: ldap_result found nothing!
(Thu May  3 14:49:25 2012) [sssd[be[default]]] [sdap_process_result] (0x2000): Trace: sh[0x7f161e5e4b20], connected[1], ops[0x7f161e5d2590], ldap[0x7f161e59e510]
(Thu May  3 14:49:25 2012) [sssd[be[default]]] [sdap_process_message] (0x4000): Message type: [LDAP_RES_SEARCH_ENTRY]
(Thu May  3 14:49:25 2012) [sssd[be[default]]] [sdap_parse_entry] (0x4000): OriginalDN: [CN=tmp,CN=auto.proj,CN=prague,CN=NIS,DC=dublin,DC=ad,DC=s3group,DC=com].
(Thu May  3 14:49:25 2012) [sssd[be[default]]] [sdap_process_result] (0x2000): Trace: sh[0x7f161e5e4b20], connected[1], ops[0x7f161e5d2590], ldap[0x7f161e59e510]
(Thu May  3 14:49:25 2012) [sssd[be[default]]] [sdap_process_message] (0x4000): Message type: [LDAP_RES_SEARCH_ENTRY]
...

as you can see, entry 'tmp' is there.

4. cat /var/log/sssd/sssd_autofs.log
(Thu May  3 14:52:37 2012) [sssd[autofs]] [sss_autofs_cmd_setautomntent] (0x2000): sss_autofs_cmd_setautomntent
(Thu May  3 14:52:37 2012) [sssd[autofs]] [sss_autofs_cmd_setautomntent] (0x0400): Got request for automount map named auto.proj
(Thu May  3 14:52:37 2012) [sssd[autofs]] [setautomntent_send] (0x0400): Requesting info for automount map [auto.proj] from [<ALL>]
(Thu May  3 14:52:37 2012) [sssd[autofs]] [setautomntent_send] (0x1000): Map auto.proj is ready to be processed
(Thu May  3 14:52:37 2012) [sssd[autofs]] [sss_autofs_cmd_setautomntent_done] (0x2000): setautomntent done
(Thu May  3 14:52:37 2012) [sssd[autofs]] [sss_autofs_cmd_setautomntent_done] (0x0400): setautomntent found data
(Thu May  3 14:52:37 2012) [sssd[autofs]] [sss_autofs_cmd_getautomntbyname] (0x2000): sss_autofs_cmd_getautomntbyname
(Thu May  3 14:52:37 2012) [sssd[autofs]] [sss_autofs_cmd_getautomntbyname] (0x0400): Requested data of map auto.proj key tmp
(Thu May  3 14:52:37 2012) [sssd[autofs]] [sss_autofs_cmd_getautomntbyname] (0x2000): Looking up value for [tmp] in [auto.proj]
(Thu May  3 14:52:37 2012) [sssd[autofs]] [getautomntbyname_process] (0x0080): No entries found
...
but for some reason, sssd is unable to see it/parse is correctly.


So I would guess, there is something wrong with sssd here - automounter works correctly.

Ondrej

also note that 'automount -m' does not show anything, too.
Ondrej

On an F17 install my simple test works OK.

[root@f17-i386 ~]# rpm -qa|egrep '(autofs|sss)'|grep -v docbook
libsss_autofs-1.8.2-10.fc17.i686
libsss_sudo-1.8.3-11.fc17.i686
sssd-client-1.8.3-11.fc17.i686
sssd-1.8.3-11.fc17.i686
autofs-5.0.6-16.fc17.i686

and trivial sssd configuration:

[root@f17-i386 ~]# cat /etc/sssd/sssd.conf 
[sssd]
config_file_version = 2
services = autofs
domains = ipaldap

[autofs]
debug_level = 10

[domain/ipaldap]
debug_level = 9
id_provider = ldap
autofs_provider = ldap
ldap_uri = ldap://zeus.themaw.net
timeout = 30000

entry_cache_timeout = 10

ldap_search_base = dc=autofs,dc=test
ldap_autofs_search_base = dc=autofs,dc=test
ldap_schema = rfc2307bis

and

[root@f17-i386 ~]# grep automount /etc/nsswitch.conf
#automount:  files nisplus
automount:  sss

LDAP maps used will be posted next.

Created attachment 582837 [details]
Output of "ldapsearch -x -h zeus -b dc=autofs,dc=test" on client

Not that I add the sss: to the indirect map to check if that
was causing a problem.

Anyone have any suggestions of changes I could make to test
specific things, like with the LDAP map for example.

We should probably open a separate F-17 bug at this point to avoid mixing up with the 6.3 one.

Ondrej, can you install the package ldb-tools and check what is in the sssd cache:

$ ldbsearch -H /var/lib/sss/db/cache_default.ldb '(|(objectclass=automountMap)(objectclass=automount))'

One thing I did notice when I preformed this test is that it
initially didn't work at all with selinux enforce after
installing the beta DVD, Performing a :yum update" resolved
that.

(In reply to comment #116)
> We should probably open a separate F-17 bug at this point to avoid mixing up
> with the 6.3 one.

Yes, that should be done, for sure.

> (In reply to comment #116)
> > We should probably open a separate F-17 bug at this point to avoid mixing up
> > with the 6.3 one.
> 
> Yes, that should be done, for sure.
Will do - but probably after sharp F17 is out. I just tried 'yum update' and the whole thing died completely again - so it is probably not very meaningful to open it right now....

So Ian, is there anything else need to do against this 6.3 bug? could I think the bug is done now?

Found one more thing to (possibly, not urgent - I am quite happy already) fix:
If we are letting automounter to read auto.master from sss resource, we also
need to handle entries in auto.master differently when they contain full ldap
path (which is also correct). Example case:
Entry in auto master:
CN=/proj,CN=auto.master,DC=example,DC=com
has a value:
CN=auto.proj,DC=example,DC=com
(i.e. not just simply auto.proj)

In this case sssd tells automounter that auto.master contains an indirect map
CN=auto.proj,DC=example,DC=com

...which looks like ldap path and indeed automounter tries on its own searching
for entries in this indirect map via its lookup_ldap module which will
consequently fail if do not have automounter configured properly.

Now, I believe this is a correctly defined indirect map (as both automounter 
and sssd understand it well), but together it makes a small problem here.

Question:
Are we willing to support this configuration?
If yes, how?

(In reply to comment #123)
> Found one more thing to (possibly, not urgent - I am quite happy already) fix:
> If we are letting automounter to read auto.master from sss resource, we also
> need to handle entries in auto.master differently when they contain full ldap
> path (which is also correct). Example case:
> Entry in auto master:
> CN=/proj,CN=auto.master,DC=example,DC=com
> has a value:
> CN=auto.proj,DC=example,DC=com
> (i.e. not just simply auto.proj)
> 
> In this case sssd tells automounter that auto.master contains an indirect map
> CN=auto.proj,DC=example,DC=com
> 
> ...which looks like ldap path and indeed automounter tries on its own searching
> for entries in this indirect map via its lookup_ldap module which will
> consequently fail if do not have automounter configured properly.

Right, so the master map parser knows it's an ldap map name.

> 
> Now, I believe this is a correctly defined indirect map (as both automounter 
> and sssd understand it well), but together it makes a small problem here.

I think so, either a lone map name or a full attribute name.
Partially completed names are not allowed.

> 
> Question:
> Are we willing to support this configuration?
> If yes, how?

That's a rather interesting question.

I think that would require changes in both autofs and sss.

In autofs giving a name like this essentially overrides
configuration settings such as search base etc.

I'm not sure that it would be possible to do this override
in sss.

I'm also not sure yet how it could be done in autofs but
it is just an override which probably should work for both
sources.

Aka,
sss:CN=auto.proj,DC=example,DC=com looks only at sss.
ldap:CN=auto.proj,DC=example,DC=com looks only at ldap.
CN=auto.proj,DC=example,DC=com looks at the source(s) defined
in nsswitch.conf in the order given.
If neither is given then ldap should be tried, for backward
compatibility only, since if no source is given I think it
should actually be a failure case.

But, like I said, I don't know how difficult this will be, or
if it is even doable, for either product.

Ian

Ian, could you create a new BZ to track this configuration? I don't think it's appropriate to keep it in this BZ any longer. It's clearly new functionality if we do it.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2012-0951.html