Bug 137220 - automount exits immediately on *any* error enumerating LDAP maps
automount exits immediately on *any* error enumerating LDAP maps
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 4
Classification: Red Hat
Component: autofs (Show other bugs)
4.0
All Linux
medium Severity medium
: ---
: ---
Assigned To: Chris Feist
Brock Organ
:
Depends On:
Blocks: 135876
  Show dependency treegraph
 
Reported: 2004-10-26 15:27 EDT by Nalin Dahyabhai
Modified: 2007-11-30 17:07 EST (History)
3 users (show)

See Also:
Fixed In Version: RHBA-2005-178
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2004-12-13 05:33:10 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Sample LDIF creating an auto.master map, and an auto.misc with 7 entries. (2.11 KB, text/plain)
2004-10-26 15:31 EDT, Nalin Dahyabhai
no flags Details
suggested patch which returns LKP_NOTSUP for certain LDAP errors, and always retries lookups with a specific query if a first lookup attempt fails (3.49 KB, patch)
2004-10-26 15:33 EDT, Nalin Dahyabhai
no flags Details | Diff

  None (edit)
Description Nalin Dahyabhai 2004-10-26 15:27:58 EDT
Description of problem:

When automount is started with an LDAP map, it attempts cache the
contents of the map by querying the server for all matching objects. 
If there is any error, it exits immediately.

This happens even if the error is LDAP_SIZELIMIT_EXCEEDED, which does
not indicate that a query for a single object would also fail (in
fact, it would succeed).

Version-Release number of selected component (if applicable):
4.1.3-28

How reproducible:
Always

Steps to Reproduce:
1. Import a map with 8 entries into your directory, along with an
auto.master map which refers to it.  I'll attach LDIF.
2. Set the sizelimit to 5 (for example, in /etc/openldap/slapd.conf if
you're using OpenLDAP).
3. Start autofs so that it pulls auto.master from LDAP.
  
Actual results:
Syslog will receive "failed to load map, exiting", and automount will
exit.

Expected results:
automount should fall back to mimicking the autofs3 "indirect" behavior.
Comment 1 Nalin Dahyabhai 2004-10-26 15:31:08 EDT
Created attachment 105809 [details]
Sample LDIF creating an auto.master map, and an auto.misc with 7 entries.
Comment 2 Nalin Dahyabhai 2004-10-26 15:33:11 EDT
Created attachment 105810 [details]
suggested patch which returns LKP_NOTSUP for certain LDAP errors, and always retries lookups with a specific query if a first lookup attempt fails
Comment 3 Nalin Dahyabhai 2004-10-26 15:35:17 EDT
This may be related to bug 133063 or bug 128103, depending on the size
of the maps which are being used by the reporters of those bugs.
Comment 5 Jack Neely 2004-10-27 09:36:17 EDT
Tested patch at NCSU.  Our LDAP server has over 5,000 entries that are
automounted which doesn't include the 60,000 some odd user accounts
that will have home directories automounted as well.  Needless to say
I was hitting the problem where openldap would return
LDAP_SIZELIMIT_EXCEEDED and blow up.
                                                                     
          
This patch is working great.  I plan on using this feature in RHEL 4.
Comment 6 Jeffrey Moyer 2004-10-27 15:38:57 EDT
nalin, shouldn't we attempt to read the entire map in lookup_ghost? 
Isn't there a way to iterate over the map in smaller chunks?  Is there
some reason this is not advisable?  The specific chunk I'm talking
about is here:

@@ -289,13 +307,19 @@
 {
 	struct lookup_context *ctxt = (struct lookup_context *) context;
 	struct mapent_cache *me;
-	int status = 1;
+	int status = 1, rv = LDAP_SUCCESS;
 	char *mapname;
 
 	chdir("/");
 
-	if (!read_map(root, ctxt))
-		return LKP_FAIL;
+	if (!read_map(root, ctxt, NULL, 0, &rv))
+		switch (rv) {
+		case LDAP_SIZELIMIT_EXCEEDED:
+		case LDAP_UNWILLING_TO_PERFORM:
+			return LKP_NOTSUP;
+		default:
+			return LKP_FAIL;
+		}

I think the change I describe is necessary for ghosting to work properly.

Comments?
Comment 7 Nalin Dahyabhai 2004-10-27 16:51:15 EDT
The lookup_ghost function is trying to read the entire map (via
read_map), but it's failing because the server is configured to cut
short the number of answers which it will send to a client for any
single search request.  When automount gets this error (along with a
partial set of results), it currently exits.

Caching just the results which it gets back would allow the
automounter to ghost some random subset of the map, but because it
currently expects the entire map to be in the cache, it would never
mount anything which wasn't in the subset of results which the server
returned.

Breaking up the search, by attempting to walk the tree under the map's
base DN using multiple searches with scope="one", might work in some
cases, but a sufficiently-large single map would still trigger a
size-limit-exceeded error, and it would be very time-consuming.

I don't know of any way to circumvent a server-imposed size limit.
Comment 8 Jeffrey Moyer 2004-10-28 11:48:24 EDT
See RFC 2696 - LDAP Control Extension for Simple Paged Results Manipulation.  
Surely we can do an ldap_get_option for the LDAP_OPT_SIZELIMIT and base our page
size on that.  This only really makes sense if we can determine how many results
there are for a query.  If we can get both of these values, then we can make an
informed decision on whether to parse the entire map, or produce a partial
cache.  (i.e. if the result set size was large and the page size was low, we
would not want to parse the full map.)

According to RFC 2696:

   Each time the server returns a set of results to the client when
   processing a search request containing the pagedResultsControl, the
   server includes the pagedResultsControl control in the
   searchResultDone message. In the control returned to the client, the
   size MAY be set to the server's estimate of the total number of
   entries in the entire result set. Servers that cannot provide such an
   estimate MAY set this size to zero (0).  The cookie MUST be set to an
   empty value if there are no more entries to return (i.e., the page of
   search results returned was the last), or, if there are more entries
   to return, to an octet string of the server's choosing,used to resume
   the search.

So, we may or may not get the estimate depending on the implementation.

In the case where we don't parse the entire map, should we ghost any of the
entries?  This could result in user confusion, to be sure.  Whatever method we
choose, this will require corresponding documentation.
Comment 9 Chris Feist 2004-10-28 11:56:35 EDT
I have tried out the patch and it works as advertised.  If the the
search limit error is recieved the only difference seems to be that
the map is not cached.

If you attempt to cd into an autofs directory using ldap (and the
results are greater than the search limit), the automounter will just
query the ldap server for that specific directory.

It may be a good idea to document this, as well as print an error/info
message notifying the user that the map will not be cached.

The fix is in autofs-4.1.3-33 & autofs-4.1.3-35.
Comment 10 Jack Neely 2004-10-28 13:02:21 EDT
Jeffrey,

Would the page size option only be used when ghosting is turned on, or
when the client /needs/ to have the entire cache?  

With LDAP maps for some 60,000 accounts I really don't want my linux
clients to attempt to get more than they need from the LDAP servers.
Comment 11 Jeffrey Moyer 2004-10-28 14:24:52 EDT
I just spoke with Nalin, and he cleared up a misconception I had about rfc2696.
 It is not a way to iterate through an entire map.  Rather, it is a way for the
client to rate limit the results it receives from the server (not to exceed the
defined maximum result set size imposed by the server).

As such, I think nalin's patch is okay as-is.  Thanks, Nalin!
Comment 12 Jay Turner 2004-12-04 07:47:04 EST
Can someone comfirm this is fixed with the latest automount packages available?
 Thanks.
Comment 13 Chris Feist 2004-12-07 14:36:32 EST
The patch as currenlty applied causes a problem if the user is trying
to use ghost and the LDAP_SIZELIMIT_EXCEEDED error is recieved.
Comment 14 Nalin Dahyabhai 2004-12-07 14:38:03 EST
(In reply to comment #13)
> The patch as currently applied causes a problem if the user is trying
> to use ghost and the LDAP_SIZELIMIT_EXCEEDED error is received.

FWIW, this is wrt 4.1.3-44.
Comment 15 Nalin Dahyabhai 2004-12-08 14:05:18 EST
Confirmed fixed in 4.1.3-67.
Comment 17 Jay Turner 2004-12-09 03:12:25 EST
Throwing in ON_QA since we're just waiting on the package to land in a tree.
Comment 19 Jay Turner 2004-12-13 05:33:10 EST
autofs-4.1.3-67 included in re1212.0 tree.  Closing out.
Comment 20 Dennis Gregorovic 2005-05-19 18:05:42 EDT
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2005-177.html

Note You need to log in before you can comment on or make changes to this bug.