Bug 826275 - sssd uses cacert from ldap.conf instead of sssd.conf
sssd uses cacert from ldap.conf instead of sssd.conf
Status: CLOSED WONTFIX
Product: Fedora
Classification: Fedora
Component: openldap (Show other bugs)
17
x86_64 Linux
unspecified Severity medium
: ---
: ---
Assigned To: Jan Synacek
Fedora Extras Quality Assurance
: Regression
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-05-29 17:56 EDT by Andrew McNabb
Modified: 2013-07-31 22:02 EDT (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-07-31 22:02:02 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)
sssd.conf (2.17 KB, text/plain)
2012-06-05 14:22 EDT, Andrew McNabb
no flags Details

  None (edit)
Description Andrew McNabb 2012-05-29 17:56:30 EDT
The sssd.conf file includes the line:

ldap_tls_cacert = /admin/etc/openldap/cacerts/cacert.pem

SSSD gives the following error:

May 29 15:39:59 testvm sssd[be[LDAP]]: Could not start TLS encryption. TLS error -8
172:Peer's certificate issuer has been marked as not trusted by the user.

But if I add the following line to /etc/openldap/ldap.conf, the problem goes away:

TLS_CACERT	/admin/etc/openldap/cacerts/cacert.pem

I am happy to work around the problem by using the entry in /etc/openldap/ldap.conf, but I figured I should report the problem in the meantime.


Version-Release number of selected component (if applicable):

sssd-1.8.3-11.fc17.x86_64


How reproducible:

Every time. I have commented and uncommented the line in ldap.conf several times, and this toggles whether password authentication works in sssd.
Comment 1 Jakub Hrozek 2012-05-31 06:07:54 EDT
That's strange, our usage of ldap_tls_cacert is nothing more than:
ldap_set_option(NULL, LDAP_OPT_X_TLS_CACERTFILE, value_of_the_option);

libldap reads ldap.conf, too, so it seems that for some reason the option is not set from sssd.conf

Can you raise the debug_level in the [domain] section of sssd.conf, restart SSSD and check debug messages from [dp_get_options] ? On my machine, the debug message looks like:
(Thu May 31 12:02:07 2012) [sssd[be[mydomain]]] [dp_get_options] (0x0400): Option ldap_tls_cacert has value /etc/openldap/cacerts/path-to-cert.

Can you also check that there is no error message from "setup_tls_config"? That's where the SSSD really sets the option, so if the setter would fail, we would see the failure here.

Thank you!
Comment 2 Jan Vcelak 2012-05-31 06:49:52 EDT
Andrew, please, also provide which version of openldap are you using.

I enabled TLS for LDAP provider in my sssd.conf and it works just fine with ldap_tls_cacert without adjusting ldap.conf.

openldap-2.4.31-2.fc17.x86_64
sssd-1.8.3-11.fc17.x86_64

Isn't it possible that you have a typo in your sssd.conf?
Comment 3 Andrew McNabb 2012-05-31 11:50:09 EDT
(In reply to comment #1)
> 
> Can you raise the debug_level in the [domain] section of sssd.conf, restart
> SSSD and check debug messages from [dp_get_options] ? On my machine, the
> debug message looks like:
> (Thu May 31 12:02:07 2012) [sssd[be[mydomain]]] [dp_get_options] (0x0400):
> Option ldap_tls_cacert has value /etc/openldap/cacerts/path-to-cert.
> 
> Can you also check that there is no error message from "setup_tls_config"?
> That's where the SSSD really sets the option, so if the setter would fail,
> we would see the failure here.

I set the following in sssd.conf:

debug_level = 0xFFFF

When I restarted sssd, the /var/log/sssd/sssd.log file got much larger, but grepping for "dp_get_options" or "ldap_tls_cacert" or "setup_tls_config" isn't showing anything. Am I setting the debug_level wrong?

(In reply to comment #2)
> Andrew, please, also provide which version of openldap are you using.

openldap-2.4.30-2.fc17.x86_64

> Isn't it possible that you have a typo in your sssd.conf?

The weird thing is that this is the exact same config file I used in Fedora 16. Furthermore, I didn't run into this problem with a Fedora 17 prerelease that I installed. So I agree that something weird is going on here, but I can't put a finger on it. Thanks.
Comment 4 Jakub Hrozek 2012-05-31 13:09:54 EDT
(In reply to comment #3)
> (In reply to comment #1)
> > 
> > Can you raise the debug_level in the [domain] section of sssd.conf, restart
> > SSSD and check debug messages from [dp_get_options] ? On my machine, the
> > debug message looks like:
> > (Thu May 31 12:02:07 2012) [sssd[be[mydomain]]] [dp_get_options] (0x0400):
> > Option ldap_tls_cacert has value /etc/openldap/cacerts/path-to-cert.
> > 
> > Can you also check that there is no error message from "setup_tls_config"?
> > That's where the SSSD really sets the option, so if the setter would fail,
> > we would see the failure here.
> 
> I set the following in sssd.conf:
> 
> debug_level = 0xFFFF
> 
> When I restarted sssd, the /var/log/sssd/sssd.log file got much larger, but
> grepping for "dp_get_options" or "ldap_tls_cacert" or "setup_tls_config"
> isn't showing anything. Am I setting the debug_level wrong?
> 

The debug_level should be set in the [domain/DOMNAME] section of sssd.conf. Then the debug messages we're looking for would appear in /var/log/sssd/sssd_$DOMNAME.log
Comment 5 Andrew McNabb 2012-05-31 13:13:39 EDT
Hmm. The domain is called domain_LDAP, and the sssd_LDAP.log file only contains a bunch of copies of the following line:

(Thu May 31 09:44:09 2012) [sssd[be[LDAP]]] [load_backend_module] (0x0010): Error (2) in module (ldap) initialization (sssm_ldap_autofs_init)!
Comment 6 Jakub Hrozek 2012-05-31 13:32:32 EDT
(In reply to comment #5)
> Hmm. The domain is called domain_LDAP, and the sssd_LDAP.log file only
> contains a bunch of copies of the following line:
> 
> (Thu May 31 09:44:09 2012) [sssd[be[LDAP]]] [load_backend_module] (0x0010):
> Error (2) in module (ldap) initialization (sssm_ldap_autofs_init)!

Are you sure the debug_level parameter is in the [domain/LDAP] section and not just in the [sssd] section? The latter used to work in older releases but does not anymore -- it mostly worked just by accident.
Comment 7 Andrew McNabb 2012-05-31 14:00:08 EDT
(In reply to comment #6)
> 
> Are you sure the debug_level parameter is in the [domain/LDAP] section and
> not just in the [sssd] section? The latter used to work in older releases
> but does not anymore -- it mostly worked just by accident.

Oops. I've now added debug_level options to all of the sections. By the way, is there a reason that the default level is fatal failures rather than serious failures or at least critical failures? At the default level, even the TLS error doesn't appear in sssd_LDAP.log.

Anyway, now that I have working log messages, I can readdress the following requested information:

(In reply to comment #1)
> 
> Can you raise the debug_level in the [domain] section of sssd.conf, restart
> SSSD and check debug messages from [dp_get_options] ? On my machine, the
> debug message looks like:
> (Thu May 31 12:02:07 2012) [sssd[be[mydomain]]] [dp_get_options] (0x0400):
> Option ldap_tls_cacert has value /etc/openldap/cacerts/path-to-cert.

(Thu May 31 11:56:09 2012) [sssd[be[LDAP]]] [dp_get_options] (0x0400): Option ldap_tls_cacert has value /admin/etc/openldap/cacerts/cacert.pem
(Thu May 31 11:56:09 2012) [sssd[be[LDAP]]] [dp_get_options] (0x0400): Option ldap_tls_cacertdir has value (null)

> Can you also check that there is no error message from "setup_tls_config"?
> That's where the SSSD really sets the option, so if the setter would fail,
> we would see the failure here.

I grepped for setup_tls_config, but no lines matched.
Comment 8 Jakub Hrozek 2012-06-05 07:34:50 EDT
(In reply to comment #7)
> (In reply to comment #6)
> > 
> > Are you sure the debug_level parameter is in the [domain/LDAP] section and
> > not just in the [sssd] section? The latter used to work in older releases
> > but does not anymore -- it mostly worked just by accident.
> 
> Oops. I've now added debug_level options to all of the sections. By the way,
> is there a reason that the default level is fatal failures rather than
> serious failures or at least critical failures? At the default level, even
> the TLS error doesn't appear in sssd_LDAP.log.
> 

By default, we only print debug messages that would have prevented the SSSD from starting. Anything else must be explicitly turned on. Some messages (including some TLS error messages) are also sent to syslog.

> Anyway, now that I have working log messages, I can readdress the following
> requested information:
> 
> (In reply to comment #1)
> > 
> > Can you raise the debug_level in the [domain] section of sssd.conf, restart
> > SSSD and check debug messages from [dp_get_options] ? On my machine, the
> > debug message looks like:
> > (Thu May 31 12:02:07 2012) [sssd[be[mydomain]]] [dp_get_options] (0x0400):
> > Option ldap_tls_cacert has value /etc/openldap/cacerts/path-to-cert.
> 
> (Thu May 31 11:56:09 2012) [sssd[be[LDAP]]] [dp_get_options] (0x0400):
> Option ldap_tls_cacert has value /admin/etc/openldap/cacerts/cacert.pem
> (Thu May 31 11:56:09 2012) [sssd[be[LDAP]]] [dp_get_options] (0x0400):
> Option ldap_tls_cacertdir has value (null)
> 
> > Can you also check that there is no error message from "setup_tls_config"?
> > That's where the SSSD really sets the option, so if the setter would fail,
> > we would see the failure here.
> 
> I grepped for setup_tls_config, but no lines matched.

OK, that means no error occurred in setup_tls_config.

Can you attach your config file and and also the debug log files (sanitized, if needed)?

Can you also check there are no AVC denials in the audit logs?
Comment 9 Andrew McNabb 2012-06-05 14:21:50 EDT
(In reply to comment #8)
> 
> By default, we only print debug messages that would have prevented the SSSD
> from starting. Anything else must be explicitly turned on. Some messages
> (including some TLS error messages) are also sent to syslog.

Yeah, my personal opinion is that this default makes it a lot harder to track down problems, but now that I've set the debug_level option in a few places, I've worked around the problem.


> Can you attach your config file and and also the debug log files (sanitized,
> if needed)?

Yes, I'll post these momentarily.

> Can you also check there are no AVC denials in the audit logs?

I just confirmed that SELinux is disabled.
Comment 10 Andrew McNabb 2012-06-05 14:22:47 EDT
Created attachment 589605 [details]
sssd.conf
Comment 11 Andrew McNabb 2012-06-05 14:28:09 EDT
The sssd_LDAP file is too big to post as an attachment, but it's available here:

http://aml.cs.byu.edu/~amcnabb/sssd_LDAP.log

This contains runs both with and without /etc/openldap/ldap.conf pointing to the cacert file. The TLS error appears when the cacert is in sssd.conf but not in ldap.conf.
Comment 12 Stephen Gallagher 2012-06-06 07:18:41 EDT
(In reply to comment #11)
> The sssd_LDAP file is too big to post as an attachment, but it's available
> here:
> 
> http://aml.cs.byu.edu/~amcnabb/sssd_LDAP.log
> 
> This contains runs both with and without /etc/openldap/ldap.conf pointing to
> the cacert file. The TLS error appears when the cacert is in sssd.conf but
> not in ldap.conf.

I'm getting a 403 on that link.
Comment 13 Andrew McNabb 2012-06-06 09:44:34 EDT
(In reply to comment #12)
> 
> I'm getting a 403 on that link.

Fixed. Sorry about that.
Comment 14 Stephen Gallagher 2012-06-06 10:12:27 EDT
(Tue Jun  5 12:19:27 2012) [sssd[be[LDAP]]] [sdap_connect_done] (0x0080): ldap_install_tls failed: [Connect error] [TLS error -8172:Peer's certificate issuer has been marked as not trusted by the user.]


This message tells me that it's trying to connect and it's being told that the certificate is untrusted. This usually means that the certificate file is incorrectly specified or stored.

Can you tell me what specific version of the 'openldap' package you have?

Also, could you try setting the ldap_tls_cacertdir option instead of ldap_tls_cacert? If this works, openldap has regressed an old bug they had where they wouldn't honor the tls_cacert option.

Could you attach ldap.conf as well, please (without the TLS_CACERT line)? I'd like to see if there's anything else in there that could be causing the poor interaction.

Finally, can you confirm that you have run cacertdir_rehash in the location holding the certificate? If the hash hasn't been created, that may cause an issue.
Comment 15 Andrew McNabb 2012-06-06 12:28:13 EDT
(In reply to comment #14)
> 
> This message tells me that it's trying to connect and it's being told that
> the certificate is untrusted. This usually means that the certificate file
> is incorrectly specified or stored.
> 
> Can you tell me what specific version of the 'openldap' package you have?

openldap-2.4.30-2.fc17.x86_64

> Also, could you try setting the ldap_tls_cacertdir option instead of
> ldap_tls_cacert? If this works, openldap has regressed an old bug they had
> where they wouldn't honor the tls_cacert option.

If I set "ldap_tls_cacertdir = /admin/etc/openldap/cacerts" in sssd.conf, I no longer see the "Could not start TLS encryption" error, as long as I run "cacertdir_rehash /admin/etc/openldap/cacerts".

> Could you attach ldap.conf as well, please (without the TLS_CACERT line)?
> I'd like to see if there's anything else in there that could be causing the
> poor interaction.

Other than the TLS_CACERT line, /etc/openldap/ldap.conf is empty.

> Finally, can you confirm that you have run cacertdir_rehash in the location
> holding the certificate? If the hash hasn't been created, that may cause an
> issue.

I did not run cacertdir_rehash, but the documentation implies that this is necessary for the ldap_tls_cacertdir option but not for the ldap_tls_cacert option. In any case, I just tried the "ldap_tls_cacert" option again after doing "cacertdir_rehash" on the directory, and I'm still seeing the "Could not start TLS encryption" error.
Comment 16 Stephen Gallagher 2012-06-06 12:50:17 EDT
Reassigning this to openldap. It appears that openldap is not honoring:

ldap_set_option(NULL, LDAP_OPT_X_TLS_CACERTFILE, true);
Comment 17 Jan Vcelak 2012-06-07 04:21:27 EDT
> ldap_set_option(NULL, LDAP_OPT_X_TLS_CACERTFILE, true);

This make no sense. LDAP_OPT_X_TLS_CACERTFILE sets the full-path of the CA certificate file, invalue must be const char *.
Comment 18 Stephen Gallagher 2012-06-07 07:24:22 EDT
Sorry, that was me not paying attention when I was translating the variable we had in our sources into this BZ. It would more appropriately be

ldap_set_option(NULL, LDAP_OPT_X_TLS_CACERTFILE, "/path/to/CAcert");

So we're doing this correctly, it's just not having any effect.
Comment 19 Jan Vcelak 2012-06-07 10:37:05 EDT
Stephen, can you create a minimal reproducer? It works for me.

Are you sure, that you initialize the LDAP handle (ldap_initialize) after setting the CA certificate file? Like this:

ldap_set_option(NULL, LDAP_OPT_X_TLS_CACERTFILE, &pemfile);
ldap_initialize(&handle, uri);

Because following code will not work:

ldap_initialize(&handle, uri);
ldap_set_option(NULL, LDAP_OPT_X_TLS_CACERTFILE, &pemfile);

If you need to create the handle before setting the option, you have to use a different approach with non-global TLS context:

int one = 1;
ldap_initialize(&handle, uri);
ldap_set_option(handle, LDAP_OPT_X_TLS_NEWCTX, &one);
ldap_set_option(handle, LDAP_OPT_X_TLS_CACERTFILE, &pemfile);

I have no other idea, why it is not working...
Comment 20 Stephen Gallagher 2012-06-07 10:52:12 EDT
We don't use ldap_initialize(), we actually use ldap_init_fd(). I'm not sure whether that would make a difference. Do you suspect it could be relevant?

Also, for the record we set the option during our initialization routines when starting up SSSD, well before calling ldap_init_fd() (This only happens the first time we have a cache-miss).

I don't know that I can provide you with a trivial reproducer. (I haven't reproduced this myself with SSSD yet).
Comment 21 Jan Vcelak 2012-06-08 04:56:17 EDT
ldap_init_fd() should be fine as well. I cannot reproduce it with my sssd.conf either.

Andrew, please, can you check that your installation is sane?
# rpm -qaV openldap\* sssd\* libsss\* libipa\*
Comment 22 Andrew McNabb 2012-06-08 11:34:12 EDT
amcnabb@testvm:~ :) rpm -qaV openldap\* sssd\* libsss\* libipa\*
S.5......  c /etc/openldap/ldap.conf
S.5......  c /etc/openldap/ldap.conf
.M.......  n /etc/openldap/slapd.d
Comment 23 Fedora Admin XMLRPC Client 2013-01-30 06:10:48 EST
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 24 Fedora End Of Life 2013-07-03 19:23:04 EDT
This message is a reminder that Fedora 17 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 17. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '17'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 17's end of life.

Bug Reporter:  Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 17 is end of life. If you 
would still like  to see this bug fixed and are able to reproduce it 
against a later version  of Fedora, you are encouraged  change the 
'version' to a later Fedora version prior to Fedora 17's end of life.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.
Comment 25 Fedora End Of Life 2013-07-31 22:02:10 EDT
Fedora 17 changed to end-of-life (EOL) status on 2013-07-30. Fedora 17 is 
no longer maintained, which means that it will not receive any further 
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.

Note You need to log in before you can comment on or make changes to this bug.