Bug 712491 - Admin server NSS errors when full SSL access (http+ldap+console) required
Summary: Admin server NSS errors when full SSL access (http+ldap+console) required
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: 389
Classification: Retired
Component: Admin
Version: 1.2.8
Hardware: x86_64
OS: Linux
high
high
Target Milestone: ---
Assignee: Rich Megginson
QA Contact: Viktor Ashirov
URL:
Whiteboard:
Depends On: 717730
Blocks: 434915
TreeView+ depends on / blocked
 
Reported: 2011-06-10 18:59 UTC by Rich Megginson
Modified: 2015-12-07 16:40 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-12-07 16:40:40 UTC
Embargoed:


Attachments (Terms of Use)
No SSL displayed for connection (15.63 KB, image/png)
2011-07-05 16:15 UTC, Anthony Messina
no flags Details
After enabling SSL on Admin Serv, I am no longer able to manage certs on the directory server via the console (36.80 KB, image/png)
2011-07-05 16:16 UTC, Anthony Messina
no flags Details
Modified User Directory settings for SSL (16.50 KB, image/png)
2011-07-05 16:46 UTC, Anthony Messina
no flags Details
Manually changed console settings to display secure connection (9.56 KB, image/png)
2011-07-05 16:46 UTC, Anthony Messina
no flags Details

Description Rich Megginson 2011-06-10 18:59:02 UTC
+++ This bug was initially created as a clone of Bug #664671 +++
[Thu Jun 09 20:13:18 2011] [notice] caught SIGTERM, shutting down
[Thu Jun 09 20:13:30 2011] [notice] SELinux policy enabled; httpd running as context unconfined_u:system_r:httpd_t:s0
[Thu Jun 09 20:13:31 2011] [notice] Access Host filter is: *.elburn.messinet.com
[Thu Jun 09 20:13:31 2011] [notice] Access Address filter is: *
[Thu Jun 09 20:13:32 2011] [notice] Apache/2.2.17 (Unix) mod_nss/2.2.17 NSS/3.12.9.0 configured -- resuming normal operations
[Thu Jun 09 20:13:32 2011] [error] NSS_Initialize failed. Certificate database: /etc/dirsrv/admin-serv.
[Thu Jun 09 20:13:32 2011] [error] SSL Library Error: -8038 SEC_ERROR_NOT_INITIALIZED
[Thu Jun 09 20:13:32 2011] [error] Could not bind as []: ldap error -1: Can't contact LDAP server
[Thu Jun 09 20:13:32 2011] [error] Could not bind as []: ldap error -1: Can't contact LDAP server
[Thu Jun 09 20:13:32 2011] [warn] Unable to bind as LocalAdmin to populate LocalAdmin tasks into cache.
[Thu Jun 09 20:13:32 2011] [crit] sslinit: NSS is required to use LDAPS, but security initialization failed [-8128:security library: no security module can perform the requested operation.].  Cannot start server

--- Additional comment from rmeggins on 2011-06-10 13:22:28 EDT ---

(In reply to comment #7)
> I continue to see this problem with: 389-admin-1.1.16-2.fc15.i686
> 
> [Thu Jun 09 20:13:18 2011] [notice] caught SIGTERM, shutting down
> [Thu Jun 09 20:13:30 2011] [notice] SELinux policy enabled; httpd running as
> context unconfined_u:system_r:httpd_t:s0
> [Thu Jun 09 20:13:31 2011] [notice] Access Host filter is:
> *.elburn.messinet.com
> [Thu Jun 09 20:13:31 2011] [notice] Access Address filter is: *
> [Thu Jun 09 20:13:32 2011] [notice] Apache/2.2.17 (Unix) mod_nss/2.2.17
> NSS/3.12.9.0 configured -- resuming normal operations
> [Thu Jun 09 20:13:32 2011] [error] NSS_Initialize failed. Certificate database:
> /etc/dirsrv/admin-serv.
> [Thu Jun 09 20:13:32 2011] [error] SSL Library Error: -8038
> SEC_ERROR_NOT_INITIALIZED
> [Thu Jun 09 20:13:32 2011] [error] Could not bind as []: ldap error -1: Can't
> contact LDAP server
> [Thu Jun 09 20:13:32 2011] [error] Could not bind as []: ldap error -1: Can't
> contact LDAP server
> [Thu Jun 09 20:13:32 2011] [warn] Unable to bind as LocalAdmin to populate
> LocalAdmin tasks into cache.
> [Thu Jun 09 20:13:32 2011] [crit] sslinit: NSS is required to use LDAPS, but
> security initialization failed [-8128:security library: no security module can
> perform the requested operation.].  Cannot start server

This doesn't look like a crash/segfault.  Can you attach /etc/dirsrv/admin-serv/adm.conf /etc/dirsrv/admin-serv/console.conf /etc/dirsrv/admin-serv/nss.conf

ls -al /etc/dirsrv/admin-serv

certutil -d /etc/dirsrv/admin-serv -L
modutil -dbdir /etc/dirsrv/admin-serv -list

--- Additional comment from amessina on 2011-06-10 14:42:05 EDT ---

Created attachment 504184 [details]
adm.conf

--- Additional comment from amessina on 2011-06-10 14:42:33 EDT ---

Created attachment 504185 [details]
console.conf

--- Additional comment from amessina on 2011-06-10 14:42:55 EDT ---

Created attachment 504186 [details]
nss.conf

--- Additional comment from amessina on 2011-06-10 14:43:20 EDT ---

Created attachment 504187 [details]
admin-serv directory listing

--- Additional comment from amessina on 2011-06-10 14:43:42 EDT ---

Created attachment 504188 [details]
admin-serv certutil listing

--- Additional comment from amessina on 2011-06-10 14:44:04 EDT ---

Created attachment 504189 [details]
admin-serv modutil output

--- Additional comment from amessina on 2011-06-10 14:44:24 EDT ---

(In reply to comment #8)
> 
> This doesn't look like a crash/segfault.  Can you attach
> /etc/dirsrv/admin-serv/adm.conf /etc/dirsrv/admin-serv/console.conf
> /etc/dirsrv/admin-serv/nss.conf
> 
> ls -al /etc/dirsrv/admin-serv
> 
> certutil -d /etc/dirsrv/admin-serv -L
> modutil -dbdir /etc/dirsrv/admin-serv -list

Of course, you are right Rich.  There is no segfault, but the admin-serv is not will not connect to the dirsrv when SSL is enabled.

I have since reverted the problem using ldapmodify:

dn: cn=slapd-elburn,cn=389 Directory Server,cn=Server Group,cn=elburn.messinet
 .com,ou=elburn.messinet.com,o=NetscapeRoot
changetype: modify
replace: nsServerSecurity
nsServerSecurity: off

and by reverting adm.conf from:
ldapurl: ldaps://elburn.messinet.com:636/o=NetscapeRoot

to
ldapurl: ldap://elburn.messinet.com:389/o=NetscapeRoot


In this way, I am able to connect to this remote admin-serv via https on the console, but keep the link between admin-serv and dirsrv without SSL (they are on the same machine).

Comment 1 Rich Megginson 2011-06-29 20:19:01 UTC
This is caused by the memleak in https://bugzilla.redhat.com/show_bug.cgi?id=717730

Because of the way Apache works, loading and unloading modules, and forking workers, we call NSS_Shutdown and NSS_Initialize several times.  If mod_admserv makes a TLS/SSL call to the directory server between one of the inits and shutdowns, the leak will cause shutdown to fail, which will then cause init to fail.

In the meantime, you can use the NSS_STRICT_NOFORK=DISABLED hack to make this work.  Just add something like this to /etc/sysconfig/dirsrv-admin:

export NSS_STRICT_NOFORK=DISABLED

Please let me know if the workaround works for you.

Comment 2 Anthony Messina 2011-07-05 16:15:15 UTC
Thanks Rich.  I can confirm that adding "export NSS_STRICT_NOFORK=DISABLED" to /etc/sysconfig/dirsrv-admin does work.

I also followed instructions at: http://docs.redhat.com/docs/en-US/Red_Hat_Directory_Server/8.2/html/Administration_Guide/Managing_SSL.html, especially section 14.2.3.2. Enabling TLS/SSL in the Directory Server, Admin Server, and Console.

After configuration, things seem to be working properly, but I notice two strange things:

1) When connecting with the console, in the windows, it doesn't display that I am using a secure connection (screenshot attached).

2) Also through the console, I am no longer able to "Manage Certificates" for the directory server, although it does work for the admin-serv (screenshot attached).

Comment 3 Anthony Messina 2011-07-05 16:15:41 UTC
Created attachment 511345 [details]
No SSL displayed for connection

Comment 4 Anthony Messina 2011-07-05 16:16:22 UTC
Created attachment 511346 [details]
After enabling SSL on Admin Serv, I am no longer able to manage certs on the directory server via the console

Comment 5 Rich Megginson 2011-07-05 16:25:23 UTC
(In reply to comment #3)
> Created attachment 511345 [details]
> No SSL displayed for connection

You have to go into the Admin Server configuration for the User Directory and configure it to use SSL, in order for the console to display that it is using TLS/SSL for the User Directory.

Comment 6 Rich Megginson 2011-07-05 16:26:58 UTC
(In reply to comment #4)
> Created attachment 511346 [details]
> After enabling SSL on Admin Serv, I am no longer able to manage certs on the
> directory server via the console

This is https://bugzilla.redhat.com/show_bug.cgi?id=710372

You might be able to work around this by adding a SetEnv directive to console.conf:

SetEnv NSS_STRICT_NOFORK DISABLED

not sure if this will work for Manage Certificates

Comment 7 Anthony Messina 2011-07-05 16:42:08 UTC
(In reply to comment #6)
> (In reply to comment #4)
> > Created attachment 511346 [details]
> > After enabling SSL on Admin Serv, I am no longer able to manage certs on the
> > directory server via the console
> 
> This is https://bugzilla.redhat.com/show_bug.cgi?id=710372
> 
> You might be able to work around this by adding a SetEnv directive to
> console.conf:
> 
> SetEnv NSS_STRICT_NOFORK DISABLED
> 
> not sure if this will work for Manage Certificates

I tried this, unfortunately without success.  I am still unable to manage dirsrv certificates from the console.

Comment 8 Anthony Messina 2011-07-05 16:45:31 UTC
(In reply to comment #5)
> (In reply to comment #3)
> > Created attachment 511345 [details]
> > No SSL displayed for connection
> 
> You have to go into the Admin Server configuration for the User Directory and
> configure it to use SSL, in order for the console to display that it is using
> TLS/SSL for the User Directory.

I believe I did do this according to the instructions (see screenshot), however, I found that I needed to "edit" the settings in the console to get it to stick (see screenshot).

Comment 9 Anthony Messina 2011-07-05 16:46:04 UTC
Created attachment 511347 [details]
Modified User Directory settings for SSL

Comment 10 Anthony Messina 2011-07-05 16:46:35 UTC
Created attachment 511348 [details]
Manually changed console settings to display secure connection

Comment 11 Rich Megginson 2011-08-12 16:45:49 UTC
I believe this is related to https://bugzilla.redhat.com/show_bug.cgi?id=717730

Please confirm what version of openldap you have, update to the latest 389 packages in updates-testing, and try to reproduce the problem.

Comment 12 Anthony Messina 2011-08-14 18:26:16 UTC
On the servers:
openldap-2.4.24-3.fc15
openldap-clients-2.4.24-3.fc15

I can confirm that the updates from testing resolve this issue and I was able to remove SetEnv NSS_STRICT_NOFORK DISABLED.

I am now using SSL in dirsrv, admin-serv, and console.  Thank you.

Comment 13 Rich Megginson 2011-08-31 18:16:13 UTC
Should be able to verify this now that there is a build of openldap containing the fix for this issue.

Comment 14 Amita Sharma 2011-09-12 10:15:37 UTC
I have used SSL in dirsrv, admin-serv, and console with build 

[root@snmaptest /]# rpm -qa | grep 389
389-ds-console-1.2.6-1.el6.noarch
389-admin-console-doc-1.1.8-1.el6.noarch
389-ds-1.2.1-2.el6.noarch
389-ds-base-devel-1.2.8.2-1.el6_1.12.x86_64
389-admin-1.1.23-1.el6.x86_64
389-admin-console-1.1.8-1.el6.noarch
389-ds-console-doc-1.2.6-1.el6.noarch
389-adminutil-1.1.14-1.el6.x86_64
389-console-1.1.7-1.el6.noarch
389-adminutil-devel-1.1.14-1.el6.x86_64
389-ds-base-debuginfo-1.2.8.2-1.el6_1.3.x86_64
389-ds-base-1.2.8.2-1.el6_1.12.x86_64
389-ds-base-libs-1.2.8.2-1.el6_1.12.x86_64

Hence VERIFIED.

Comment 15 Victor Hugo dos Santos 2013-04-19 15:16:47 UTC
This bug is currently active at Centos and EPEL 6.

my version is:
389-console-1.1.7-1.el6.noarch
389-admin-1.1.29-1.el6.x86_64
389-admin-console-1.1.8-1.el6.noarch
389-ds-console-doc-1.2.6-1.el6.noarch
389-ds-1.2.2-1.el6.noarch
389-adminutil-1.1.15-1.el6.x86_64
389-ds-base-1.2.11.15-14.el6_4.x86_64
389-ds-console-1.2.6-1.el6.noarch
389-admin-console-doc-1.1.8-1.el6.noarch
389-dsgw-1.1.10-1.el6.x86_64
389-ds-base-libs-1.2.11.15-14.el6_4.x86_64
openldap-2.4.23-32.el6_4.x86_64
openldap-clients-2.4.23-32.el6_4.x86_64

and for us, is imposible to change to testing version.. because is a production server !!! 

any idea ???

thanks

Comment 16 Rich Megginson 2013-04-19 15:27:50 UTC
(In reply to comment #15)
> This bug is currently active at Centos and EPEL 6.
> 
> my version is:
> 389-console-1.1.7-1.el6.noarch
> 389-admin-1.1.29-1.el6.x86_64
> 389-admin-console-1.1.8-1.el6.noarch
> 389-ds-console-doc-1.2.6-1.el6.noarch
> 389-ds-1.2.2-1.el6.noarch
> 389-adminutil-1.1.15-1.el6.x86_64
> 389-ds-base-1.2.11.15-14.el6_4.x86_64
> 389-ds-console-1.2.6-1.el6.noarch
> 389-admin-console-doc-1.1.8-1.el6.noarch
> 389-dsgw-1.1.10-1.el6.x86_64
> 389-ds-base-libs-1.2.11.15-14.el6_4.x86_64
> openldap-2.4.23-32.el6_4.x86_64
> openldap-clients-2.4.23-32.el6_4.x86_64
> 
> and for us, is imposible to change to testing version.. because is a
> production server !!! 
> 
> any idea ???
> 
> thanks

What version of CentOS?  6.3?  6.4?

Is it possible you are not running into 712491 but instead are running into https://bugzilla.redhat.com/show_bug.cgi?id=923122?

Have you tried the workaround in https://bugzilla.redhat.com/show_bug.cgi?id=712491#c6 ?

Comment 17 Victor Hugo dos Santos 2013-04-19 15:37:13 UTC
(In reply to comment #16)
> (In reply to comment #15)
> > This bug is currently active at Centos and EPEL 6.

[...]

> > any idea ???
> > 
> > thanks
> 
> What version of CentOS?  6.3?  6.4?

6.4 full update.


> Is it possible you are not running into 712491 but instead are running into
> https://bugzilla.redhat.com/show_bug.cgi?id=923122?

I dont have access to it :-(
 "You are not authorized to access bug #923122."


> Have you tried the workaround in
> https://bugzilla.redhat.com/show_bug.cgi?id=712491#c6 ?

yes.. I tried it.. but the resulted is the same.
and my problem is not in the console.. is in dirsrv-admin
so, I tried to put "export NSS_STRICT_NOFORK=DISABLED" in "/etc/sysconfig/dirsrv-admin" too, but the resulted is the same.

thanks and antetive.

Comment 18 Rich Megginson 2013-04-19 15:45:11 UTC
(In reply to comment #17)
> (In reply to comment #16)
> > (In reply to comment #15)
> > > This bug is currently active at Centos and EPEL 6.
> 
> [...]
> 
> > > any idea ???
> > > 
> > > thanks
> > 
> > What version of CentOS?  6.3?  6.4?
> 
> 6.4 full update.
> 
> 
> > Is it possible you are not running into 712491 but instead are running into
> > https://bugzilla.redhat.com/show_bug.cgi?id=923122?
> 
> I dont have access to it :-(
>  "You are not authorized to access bug #923122."

Ok.  Are you seeing error messages like this?
[Mon Mar 18 21:06:04 2013] [error] NSS_Initialize failed. Certificate database: /etc/dirsrv/admin-serv.
[Mon Mar 18 21:06:04 2013] [error] SSL Library Error: -8038 SEC_ERROR_NOT_INITIALIZED
[Mon Mar 18 21:06:05 2013] [notice] child pid 5725 exit signal Segmentation fault (11)

That is, do you see a Segmentation fault?

> 
> 
> > Have you tried the workaround in
> > https://bugzilla.redhat.com/show_bug.cgi?id=712491#c6 ?
> 
> yes.. I tried it.. but the resulted is the same.
> and my problem is not in the console.. is in dirsrv-admin
> so, I tried to put "export NSS_STRICT_NOFORK=DISABLED" in
> "/etc/sysconfig/dirsrv-admin" too, but the resulted is the same.
> 
> thanks and antetive.

Can you provide the exact error messages you are seeing in your logs?

Comment 19 Victor Hugo dos Santos 2013-04-19 15:58:50 UTC
(In reply to comment #18)

[...]

> > > Is it possible you are not running into 712491 but instead are running into
> > > https://bugzilla.redhat.com/show_bug.cgi?id=923122?
> > 
> > I dont have access to it :-(
> >  "You are not authorized to access bug #923122."
> 
> Ok.  Are you seeing error messages like this?
> [Mon Mar 18 21:06:04 2013] [error] NSS_Initialize failed. Certificate
> database: /etc/dirsrv/admin-serv.
> [Mon Mar 18 21:06:04 2013] [error] SSL Library Error: -8038
> SEC_ERROR_NOT_INITIALIZED
> [Mon Mar 18 21:06:05 2013] [notice] child pid 5725 exit signal Segmentation
> fault (11)
> 
> That is, do you see a Segmentation fault?


No.. I dont have this errors !!!


> > > Have you tried the workaround in
> > > https://bugzilla.redhat.com/show_bug.cgi?id=712491#c6 ?
> > 
> > yes.. I tried it.. but the resulted is the same.
> > and my problem is not in the console.. is in dirsrv-admin
> > so, I tried to put "export NSS_STRICT_NOFORK=DISABLED" in
> > "/etc/sysconfig/dirsrv-admin" too, but the resulted is the same.
> > 
> > thanks and antetive.
> 
> Can you provide the exact error messages you are seeing in your logs?

of course:

================
[Fri Apr 19 12:35:35 2013] [notice] SELinux policy enabled; httpd running as context unconfined_u:system_r:httpd_t:s0
[Fri Apr 19 12:35:36 2013] [crit] sslinit: NSS is required to use LDAPS, but security initialization failed [-8018:(null)].  Cannot start server
================

that is all the errors in /var/log/dirsrv/admin-serv/error when I restarted the service dirsrv-admin.

thanks

Comment 20 Rich Megginson 2013-04-19 16:16:44 UTC
(In reply to comment #19)
> (In reply to comment #18)
> 
> [...]
> 
> > > > Is it possible you are not running into 712491 but instead are running into
> > > > https://bugzilla.redhat.com/show_bug.cgi?id=923122?
> > > 
> > > I dont have access to it :-(
> > >  "You are not authorized to access bug #923122."
> > 
> > Ok.  Are you seeing error messages like this?
> > [Mon Mar 18 21:06:04 2013] [error] NSS_Initialize failed. Certificate
> > database: /etc/dirsrv/admin-serv.
> > [Mon Mar 18 21:06:04 2013] [error] SSL Library Error: -8038
> > SEC_ERROR_NOT_INITIALIZED
> > [Mon Mar 18 21:06:05 2013] [notice] child pid 5725 exit signal Segmentation
> > fault (11)
> > 
> > That is, do you see a Segmentation fault?
> 
> 
> No.. I dont have this errors !!!
> 
> 
> > > > Have you tried the workaround in
> > > > https://bugzilla.redhat.com/show_bug.cgi?id=712491#c6 ?
> > > 
> > > yes.. I tried it.. but the resulted is the same.
> > > and my problem is not in the console.. is in dirsrv-admin
> > > so, I tried to put "export NSS_STRICT_NOFORK=DISABLED" in
> > > "/etc/sysconfig/dirsrv-admin" too, but the resulted is the same.
> > > 
> > > thanks and antetive.
> > 
> > Can you provide the exact error messages you are seeing in your logs?
> 
> of course:
> 
> ================
> [Fri Apr 19 12:35:35 2013] [notice] SELinux policy enabled; httpd running as
> context unconfined_u:system_r:httpd_t:s0
> [Fri Apr 19 12:35:36 2013] [crit] sslinit: NSS is required to use LDAPS, but
> security initialization failed [-8018:(null)].  Cannot start server
> ================
> 
> that is all the errors in /var/log/dirsrv/admin-serv/error when I restarted
> the service dirsrv-admin.
> 
> thanks

Ok.  Even though the result for you is the same, this is a different problem than the original one - the original problem was error -8038, but your problem is error -8018.  Please file a ticket at https://fedorahosted.org/389/newticket

Comment 21 Slava 2014-04-01 01:04:12 UTC
I am experience exact issue as described above. Work around of export to sysconfig is resolved the issue for right now.



[Mon Mar 31 21:01:05 2014] [error] SSL Library Error: -8038 SEC_ERROR_NOT_INITIALIZED
[Mon Mar 31 21:01:06 2014] [notice] child pid 30007 exit signal Segmentation fault (11)
[Mon Mar 31 21:01:07 2014] [error] NSS_Initialize failed. Certificate database: /etc/dirsrv/admin-serv.
[Mon Mar 31 21:01:07 2014] [error] SSL Library Error: -8038 SEC_ERROR_NOT_INITIALIZED
[Mon Mar 31 21:01:08 2014] [notice] child pid 30008 exit signal Segmentation fault (11)
[Mon Mar 31 21:01:09 2014] [error] NSS_Initialize failed. Certificate database: /etc/dirsrv/admin-serv.
[Mon Mar 31 21:01:09 2014] [error] SSL Library Error: -8038 SEC_ERROR_NOT_INITIALIZED
[Mon Mar 31 21:01:09 2014] [notice] caught SIGTERM, shutting down
[Mon Mar 31 21:01:10 2014] [notice] SELinux policy enabled; httpd running as context system_u:system_r:httpd_t:s0
[Mon Mar 31 21:01:11 2014] [notice] Access Host filter is: *.pythian.com
[Mon Mar 31 21:01:11 2014] [notice] Access Address filter is: *
[Mon Mar 31 21:01:12 2014] [notice] Apache/2.2.15 (Unix) mod_nss/2.2.15 NSS/3.15.1 Basic ECC configured -- resuming normal operations
[Mon Mar 31 21:01:12 2014] [notice] Access Host filter is: *.pythian.com
[Mon Mar 31 21:01:12 2014] [notice] Access Address filter is: *


Note You need to log in before you can comment on or make changes to this bug.