Bugzilla will be upgraded to version 5.0 on a still to be determined date in the near future. The original upgrade date has been delayed.
Bug 615454 - sssd backend gets "UNAVAIL error" message when configured for proxy identity and auth.
sssd backend gets "UNAVAIL error" message when configured for proxy identity ...
Status: CLOSED WONTFIX
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: nss-pam-ldapd (Show other bugs)
6.1
All Linux
medium Severity medium
: rc
: ---
Assigned To: Nalin Dahyabhai
BaseOS QE Security Team
:
Depends On:
Blocks: 579778
  Show dependency treegraph
 
Reported: 2010-07-16 14:54 EDT by Gowrishankar Rajaiyan
Modified: 2011-01-28 10:34 EST (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2011-01-28 10:34:16 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
sample test (1.83 KB, text/plain)
2010-07-16 17:18 EDT, Nalin Dahyabhai
no flags Details

  None (edit)
Description Gowrishankar Rajaiyan 2010-07-16 14:54:04 EDT
Description of problem:


Version-Release number of selected component (if applicable):
nss-pam-ldapd-0.7.5-3.el6.x86_64
sssd-1.2.1-20.el6.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Configured nslcd.conf as:

cat /etc/nslcd.conf  | grep -v "#"
uid nslcd
gid ldap
uri ldaps://sssdldap.idm.lab.bos.redhat.com:636
base dc=example,dc=com
tls_cacertdir /etc/openldap/cacerts
ssl yes

2. service nslcd restart
3. getent -s ldap passwd
puser1:*:1001:1001:p user1:/home/puser1:/bin/bash
puser2:*:1002:1002:p user2:/home/puser2:/bin/bash

4. Configured sssd.conf for proxy auth.
# cat /etc/sssd/sssd.conf
[sssd]
config_file_version = 2
reconnection_retries = 3
sbus_timeout = 30
services = nss, pam
 
domains = PROXY
 
[nss]
filter_groups = root
filter_users = root
reconnection_retries = 3
 
[pam]
reconnection_retries = 3
 
[domain/PROXY]
auth_provider = proxy
id_provider = proxy
cache_credentials = true
enumerate = TRUE
debug_level = 9
proxy_lib_name = ldap
proxy_pam_target = sssdproxyldap
ldap_tls_cacert = /etc/openldap/cacerts/cacert-sssdldap.asc

5. create # cat /etc/pam.d/sssdproxyldap 
auth            required      pam_ldap.so 
account         required      pam_ldap.so 
password        required      pam_ldap.so 
session         required      pam_ldap.so 

6. service sssd restart
7. getent -s sss passwd
  
Actual results:
Users are not enumerated.

snippet of /var/log/sss/sssd_PROXY.log:
(Sat Jul 17 00:03:20 2010) [sssd[be[PROXY]]] [sbus_message_handler] (9): Received SBUS method [getAccountInfo]
(Sat Jul 17 00:03:20 2010) [sssd[be[PROXY]]] [be_get_account_info] (4): Got request for [4097][1][name=*]
(Sat Jul 17 00:03:20 2010) [sssd[be[PROXY]]] [be_get_account_info] (4): Request processed. Returned 1,11,Fast reply - offline
(Sat Jul 17 00:03:20 2010) [sssd[be[PROXY]]] [enum_users_send] (7): Enumerating users
(Sat Jul 17 00:03:20 2010) [sssd[be[PROXY]]] [ldb] (9): start ldb transaction (nesting: 0)
(Sat Jul 17 00:03:22 2010) [sssd[be[PROXY]]] [enum_users_process] (7): User found (puser1, 1001, 1001)
(Sat Jul 17 00:03:22 2010) [sssd[be[PROXY]]] [sysdb_search_entry_done] (6): Error: Entry not Found!
(Sat Jul 17 00:03:22 2010) [sssd[be[PROXY]]] [sysdb_search_entry_done] (6): Error: Entry not Found!
(Sat Jul 17 00:03:23 2010) [sssd[be[PROXY]]] [enum_users_process] (7): User found (puser2, 1002, 1002)
(Sat Jul 17 00:03:23 2010) [sssd[be[PROXY]]] [sysdb_search_entry_done] (6): Error: Entry not Found!
(Sat Jul 17 00:03:23 2010) [sssd[be[PROXY]]] [sysdb_search_entry_done] (6): Error: Entry not Found!
(Sat Jul 17 00:03:23 2010) [sssd[be[PROXY]]] [ldb] (9): cancel ldb transaction (nesting: 0)
(Sat Jul 17 00:03:23 2010) [sssd[be[PROXY]]] [proxy_get_account_info_done] (2): proxy returned UNAVAIL error, going offline!
(Sat Jul 17 00:03:23 2010) [sssd[be[PROXY]]] [be_mark_offline] (8): Going offline!
(Sat Jul 17 00:03:23 2010) [sssd[be[PROXY]]] [be_run_offline_cb] (9): Offline call back list is empty, nothing to do.
(Sat Jul 17 00:03:29 2010) [sssd[be[PROXY]]] [sbus_dispatch] (9): dbus conn: 12444D0
(Sat Jul 17 00:03:29 2010) [sssd[be[PROXY]]] [sbus_dispatch] (9): Dispatching.


Expected results:
Users should be enumerated successfully.

Additional info:
Comment 2 RHEL Product and Program Management 2010-07-16 15:17:43 EDT
This issue has been proposed when we are only considering blocker
issues in the current Red Hat Enterprise Linux release. It has
been denied for the current Red Hat Enterprise Linux release.

** If you would still like this issue considered for the current
release, ask your support representative to file as a blocker on
your behalf. Otherwise ask that it be considered for the next
Red Hat Enterprise Linux release. **
Comment 3 Nalin Dahyabhai 2010-07-16 17:18:05 EDT
Created attachment 432506 [details]
sample test
Comment 4 Nalin Dahyabhai 2010-07-16 17:46:27 EDT
On my test box, pointing at the same server, I can reproduce this without involving sssd.  The error logged by nslcd is "ldap_result() failed: Referral", and can be logged when ldap_parse_result() returns LDAP_REFERRAL as a result.

Sure enough, when I run a mostly-equivalent ldapsearch from the command-line, the server is returning LDAP_REFERRAL as the result of the search operation:

  [root@dhcp-100-2-228 ~]# ldapsearch -x -H
  ldap://sssdldap.idm.lab.bos.redhat.com -b dc=example,dc=com uid=puser1
  # extended LDIF
  #
  # LDAPv3
  # base <dc=example,dc=com> with scope subtree
  # filter: uid=puser1
  # requesting: ALL
  #

  # puser1, People, example.com
  dn: uid=puser1,ou=People, dc=example,dc=com
  givenName: p
  sn: user1
  loginShell: /bin/bash
  uidNumber: 1001
  gidNumber: 1001
  objectClass: top
  objectClass: person
  objectClass: organizationalPerson
  objectClass: inetorgperson
  objectClass: posixAccount
  uid: puser1
  cn: p user1
  homeDirectory: /home/puser1

  # search reference
  ref: ldap://sssdldap.idm.lab.bos.redhat.com:2389/ou=vtv,ou=People,dc=example,d
   c=com

  # search result
  search: 2
  result: 10 Referral
  ref: ldap://sssdldap.idm.lab.bos.redhat.com:2389/ou=mtv,dc=example,dc=com

  # numResponses: 3
  # numEntries: 1
  # numReferences: 1

In its default configuration nslcd enables LDAP_OPT_REFERRALS, and I'm not sure if libldap is supposed to be chasing this one and masking the error, too.  CCing the openldap package maintainer.

Jan, is libldap supposed to be chasing the referral that's returned in the result message when LDAP_OPT_REFERRALS is enabled?  If not, then I guess nslcd will either have to learn to chase the referral itself or suppress the error and hope for the best.
Comment 5 Nalin Dahyabhai 2010-07-16 18:44:50 EDT
(In reply to comment #4)
> Jan, is libldap supposed to be chasing the referral that's returned in the
> result message when LDAP_OPT_REFERRALS is enabled?  If not, then I guess nslcd
> will either have to learn to chase the referral itself or suppress the error
> and hope for the best.    

Answering myself here, it looks like yes, libldap tries to handle that.  Tracing through the library turns up this:

Unable to chase referral "ldap://sssdldap.idm.lab.bos.redhat.com:2389/ou=vtv,ou=People,dc=example,dc=com" (-1: Can't contact LDAP server)

It looks like libldap is silently treating the ldap:// referral that it gets over the ldaps:// connection as if it were an ldaps:// URL, and the directory server isn't having that on this port.

Switching the client configuration to not use SSL, the nslcd starts logging
  nslcd: [8b4567] ldap_result() failed: No such object

This appears to be caused by the second referral:
  ldap://sssdldap.idm.lab.bos.redhat.com:2389/ou=mtv,dc=example,dc=com

A manual search of that location returns a "no such object" error.
Comment 6 Jan Zeleny 2010-07-19 03:47:56 EDT
(In reply to comment #5)
> Answering myself here, it looks like yes, libldap tries to handle that. 
> Tracing through the library turns up this:

You are right, openldap client library should handle referral chasing.
Comment 7 Nalin Dahyabhai 2010-07-19 13:37:18 EDT
(In reply to comment #6)
> (In reply to comment #5)
> > Answering myself here, it looks like yes, libldap tries to handle that. 
> > Tracing through the library turns up this:
> 
> You are right, openldap client library should handle referral chasing.    

Are there any circumstances in which ldap_parse_result() should return LDAP_REFERRAL when LDAP_OPT_REFERRALS is enabled?
Comment 8 Jan Zeleny 2010-07-20 02:53:16 EDT
I'm not aware of any. But to be honest, I'm not that familiar with libldap code, so it is possible that there is a hidden case somewhere in it. I'll try to look at it later today.
Comment 9 Nalin Dahyabhai 2010-07-20 17:38:16 EDT
So far, attempting to reproduce this with a server I can control, I'm seeing the LDAP_REFERRAL result code only when I configure a default referral (using the nsslapd-referral setting for 389) and the client's search base doesn't fall within a naming context that the server knows about.  Either that, or when following a referral.  I'm still not sure of how to trigger the error without what I'd consider to be a server-side misconfiguration, though.
Comment 10 Nalin Dahyabhai 2010-09-17 15:15:29 EDT
Jan, did you find anything more?
Comment 11 Jan Zeleny 2010-09-20 03:22:47 EDT
Sorry, since I no longer maintain openldap I didn't have a chance to look at this. I'm adding Jan Vcelak, current maintainer of openldap, to CC. I'll also brief him about this issue in person.
Comment 12 Jan Vcelak 2010-09-21 07:48:22 EDT
Hi Nalin,

libldap should follow the referrals only if you set LDAP_OPT_REFERRALS: ldap_set_option(ld, LDAP_OPT_REFERRALS, LDAP_OTP_ON)

And if the referral can't be chased, ldap_parse_result should give you LDAP_REFERAL through *errcodep parameter, return value is undefined.

I don't manage to reproduce this behavior with openldap-clients tools. I set up two LDAP servers (F14, Rawhide), configured referral and tried ldapsearching from RHEL-6. It works well. But: 'ldapsearch' doesn't follow referrals by default, you have to add -C option.

openldap-clients-2.4.19-15.el6.x86_64
openldap-2.4.19-15.el6.x86_64

Is this helpful?

Jan
Comment 13 Nalin Dahyabhai 2010-09-21 14:21:18 EDT
(In reply to comment #12)
> Hi Nalin,
> 
> libldap should follow the referrals only if you set LDAP_OPT_REFERRALS:
> ldap_set_option(ld, LDAP_OPT_REFERRALS, LDAP_OTP_ON)
> 
> And if the referral can't be chased, ldap_parse_result should give you
> LDAP_REFERAL through *errcodep parameter, return value is undefined.

Is this behavior correct, though?

> I don't manage to reproduce this behavior with openldap-clients tools. I set up
> two LDAP servers (F14, Rawhide), configured referral and tried ldapsearching
> from RHEL-6. It works well. But: 'ldapsearch' doesn't follow referrals by
> default, you have to add -C option.

As you've noticed, it's very dependent on the server configuration.
Comment 14 Jan Vcelak 2010-09-27 07:08:14 EDT
(In reply to comment #13)
> Is this behavior correct, though?

I think so. It's in documentation [1], and it's definitely in the code [2] (ldap_chase_v3referrals, line 1036; see around line 1234).

[1] http://www.openldap.org/software/man.cgi?query=ldap_parse_result&sektion=3&apropos=0&manpath=OpenLDAP+2.4-Release

[2] http://www.openldap.org/devel/cvsweb.cgi/~checkout~/libraries/libldap/request.c?rev=1.168&hideattic=1&sortbydate=0
Comment 16 Nalin Dahyabhai 2010-10-13 14:02:57 EDT
The upshot here is that when nslcd encounters an LDAP_REFERRAL error it returns NSS_UNAVAIL.  If sssd is acting as a caching proxy, it will not consult nslcd for anything again until after a timeout has expired, even if nslcd would have successfully answered sssd's queries.  This can cause some problems.

It looks like we need nslcd either swallow the error and try to muddle along with what it's managed to find, or fail the whole query with a different result code.
Comment 20 Nalin Dahyabhai 2011-01-27 11:00:43 EST
On further reflection I'm more inclined to just call this a server-side configuration problem -- if the server's handing out referrals to invalid locations (the no-such-object case), papering over it in nslcd isn't going to fix anything for any other client of the directory.

The ldap:// referral being quietly rewritten as an ldaps:// referral that doesn't work isn't specific to nslcd, either, as that's a security measure taken by the LDAP client library .

Either way, assuming that we've gathered enough data and just suppressing the error (which, other than the current behavior of returning a failure code, is our only option) is more likely to generate inconsistent results, which I really dislike.  So I'm now leaning toward marking this won't-fix.
Comment 21 Ondrej Moriš 2011-01-27 18:15:30 EST
To sum it up: 

(a) nslcd enables LDAP_OPT_REFERRALS and hence openldap client library is supposed to chase (possible) referrals

(b) if the referral can't be chased (e.g. referral to an invalid location), then ldap_parse_result gives an LDAP_REFERAL error

(c) if nslcd get an LDAP_REFERRAL error, it returns NSS_UNAVAIL

(d) NSS_UNAVAIL is not an appropriate return code for this situation (?)

(BTW: I totally agree that (b) represents a server-side configuration issue.)

As you have pointed out in Comment #16, there are two possible solution in nslcd: (1) to suppress the error and (2) to return a more appropriate return code.

The first solution may clearly cause serious problems and, from QA point of view, I strongly do not recommend it. But what about the second solution? Could it be somehow helpful (e.g. for sssd)? If not then there is probably nothing to fix.
Comment 22 Nalin Dahyabhai 2011-01-27 18:28:30 EST
(In reply to comment #21)

> The first solution may clearly cause serious problems and, from QA point of
> view, I strongly do not recommend it. But what about the second solution? Could
> it be somehow helpful (e.g. for sssd)? If not then there is probably nothing to
> fix.

The other values we can return instead of NSS_STATUS_UNAVAIL are NSS_STATUS_NOTFOUND and NSS_STATUS_TRYAGAIN (assuming that NSS_STATUS_SUCCESS is not an option).  Neither of those values is going to provide more information to the calling application (and NSS_STATUS_NOTFOUND could be used for negative caching).  There's more on this in the libc info pages, but that's the short version.
Comment 23 Ondrej Moriš 2011-01-28 04:30:29 EST
Well, it really seems that this problem cannot be reliably resolved in nss-pam-ldapd and, moreover, it can be "bypassed" in server-side (by not using unavailable referrals). Hence it might be better to close this bug as WONTFIX.
Comment 24 Gowrishankar Rajaiyan 2011-01-28 06:58:35 EST
I agree on the fact that suppressing the error message would more likely generate inconsistent results for a security measure at the server configuration end. Since there is nothing much that can be done here we may close this bug as WONTFIX.
Comment 25 Nalin Dahyabhai 2011-01-28 10:34:16 EST
Closing.

Note You need to log in before you can comment on or make changes to this bug.