Bug 1263703

Summary: ipa-server-install with externally signed CA fails with NSS error (SEC_ERROR_BUSY)
Product: Red Hat Enterprise Linux 6 Reporter: David Kupka <dkupka>
Component: ipaAssignee: IPA Maintainers <ipa-maint>
Status: CLOSED ERRATA QA Contact: Namita Soman <nsoman>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 6.7CC: charles_sheridan, ekeck, jcholast, ksiddiqu, mkosek, nsoman, pvoborni, rcritten, salmy, spoore, tscherf
Target Milestone: rcKeywords: Regression, ZStream
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: ipa-3.0.0-48.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1280208 (view as bug list) Environment:
Last Closed: 2016-05-11 00:08:29 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1272422, 1280208    

Description David Kupka 2015-09-16 13:05:51 UTC 
Description of problem:
IPA server installation with externally signed CA certificate fails in the second phase (providing the signed certificate and signing CA certificate) with NSS error 


Version-Release number of selected component (if applicable):
ipa-server-3.0.0-47.el6
python-nss-0.16.0-1.el6
pki-ca-9.0.3-43.el6

How reproducible:
Always


Steps to Reproduce:
1. # ipa-server-install -a Secret123 -p Secret123 -r EXAMPLE.TEST --external-ca -U
2. # echo -e "0\n1\n5\n6\n9\ny\ny\n\ny\n" | certutil -C -d /root/nssdb/ -m $RANDOM -a -i /root/ipa.csr -o /root/ipa.crt -1 -2
3. # ipa-server-install --external_cert_file /root/ipa.crt --external_ca_file /root/nssdb/ca1.pem

Actual results:
  [17/38]: configuring ssl for ds instance
Unexpected error - see /var/log/ipaserver-install.log for details:
NetworkError: cannot connect to 'https://vm-196.example.test:9444/ca/ee/ca/profileSubmitSSLClient': (SEC_ERROR_BUSY) NSS could not shutdown. Objects are still in use.


Expected results:
IPA server installs without error and have CA signed by external CA certificate.


Additional info:
from /var/log/ipaserver-install:
[snip]
2015-09-16T13:03:02Z DEBUG https_request 'https://vm-196.example.test:9444/ca/ee/ca/profileSubmitSSLClient'
2015-09-16T13:03:02Z DEBUG https_request post 'profileId=caIPAserviceCert&requestor_name=IPA+Installer&cert_request=MIICejCCAWICAQAwNTEVMBMGA1UEChMMRVhBTVBMRS5URVNUMRwwGgYDVQQDExN2%0D%0AbS0xOTYuZXhhbXBsZS50ZXN0MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKC%0D%0AAQEAt41Bl%2FAgMeVKPj6cS6zasOfktN%2BQw3ahp8hgd0Jca8Ii2fkaV3FUkVlrtxqO%0D%0Ay4xrK5piohdb%2BMsYCQbIFhA69EQX6yknNzCaxDjD9nIdVvboDvdH2rd%2BhWNjJixj%0D%0ACiEHAfGySQveXJP%2BM4bsZeozNDIwJnY08V5bhEPWBTb4Pm3iMBZh4bYwtuuyfDVJ%0D%0AM6AG%2BJrHZ4IIFcab3VZwjMcav4%2FqQ62Clt%2FRFDmA%2BluEAt8%2Ft4b%2FFxfKEtglFdoa%0D%0ARmvDR8SD2%2BPfhVGB4lJ6nfWcGMu1imK3%2FwIujOAQMP3GK%2Bl4cfhOmptGVsIIKw2N%0D%0AHvTS0ySl7ll6qXbXS1ueuaK8%2FQIDAQABoAAwDQYJKoZIhvcNAQEFBQADggEBACuC%0D%0AVTRl1cmIncCd7MP9wOHXhkD0rdk4xQhGs44cZsulj0lgN0ygrgQ%2BzIPTndm7wRBM%0D%0ANSz5dizyc%2BZgFuFKNVZTySDRW3YXQZk9KolJTuYy5%2By5qAXQcSpfybYf2eI%2Br66p%0D%0AYDtcOpB1RVybLcLmDw1cdYGo%2BulNcVAaff%2FSRazNkev3JBMBZIlwrbK9f70s63O4%0D%0A8Lpygc%2BYYJVMlK%2BVOEpkGvufvLDRDF1pOOMFPWmOJdeiYSaerqAlnOEnKN5zXd2x%0D%0ANcvoWgyGCa2uQpNW52xYtBUaJHTA51RQvJyafCEs%2BemrC%2B5B4AHP50Fn8G2UbGRw%0D%0AegV4Ul%2F%2FJpRp12FWCKc%3D%0A&cert_request_type=pkcs10&xmlOutput=true'
2015-09-16T13:03:02Z DEBUG NSSConnection init vm-196.example.test
2015-09-16T13:03:02Z INFO   File "/usr/lib/python2.6/site-packages/ipaserver/install/installutils.py", line 614, in run_script
    return_value = main_function()

  File "/usr/sbin/ipa-server-install", line 995, in main
    hbac_allow=not options.hbac_allow)

  File "/usr/lib/python2.6/site-packages/ipaserver/install/dsinstance.py", line 266, in create_instance
    self.start_creation(runtime=60)

  File "/usr/lib/python2.6/site-packages/ipaserver/install/service.py", line 358, in start_creation
    method()

  File "/usr/lib/python2.6/site-packages/ipaserver/install/dsinstance.py", line 559, in __enable_ssl
    self.dercert = dsdb.create_server_cert("Server-Cert", self.fqdn, cadb)

  File "/usr/lib/python2.6/site-packages/ipaserver/install/certs.py", line 575, in create_server_cert
    cdb.issue_server_cert(self.certreq_fname, self.certder_fname)

  File "/usr/lib/python2.6/site-packages/ipaserver/install/certs.py", line 671, in issue_server_cert
    self.secdir, password, "ipaCert", **params)

  File "/usr/lib/python2.6/site-packages/ipapython/dogtag.py", line 206, in https_request
    raise NetworkError(uri=uri, error=str(e))

2015-09-16T13:03:02Z INFO The ipa-server-install command failed, exception: NetworkError: cannot connect to 'https://vm-196.example.test:9444/ca/ee/ca/profileSubmitSSLClient': (SEC_ERROR_BUSY) NSS could not shutdown. Objects are still in use.
Comment 6 Jan Cholasta 2015-10-13 08:42:43 UTC 
This was fixed upstream during 3.3 development:
https://fedorahosted.org/freeipa/ticket/3773
Comment 10 Scott Poore 2016-01-11 21:47:44 UTC 
Verified.

Version ::

ipa-server-3.0.0-50.el6.x86_64
pki-ca-9.0.3-44.el6.noarch
python-nss-0.16.0-1.el6.x86_64

Results :: 

[root@vm6 ~]# ipa-server-install -a Secret123 -p Secret123 -r EXAMPLE.TEST --external-ca -U

The log file for this installation can be found in /var/log/ipaserver-install.log
==============================================================================
This program will set up the IPA Server.

This includes:
  * Configure a stand-alone CA (dogtag) for certificate management
  * Configure the Network Time Daemon (ntpd)
  * Create and configure an instance of Directory Server
  * Create and configure a Kerberos Key Distribution Center (KDC)
  * Configure Apache (httpd)

To accept the default shown in brackets, press the Enter key.

The domain name has been determined based on the host name.


The IPA Master Server will be configured with:
Hostname:      vm6.example.test
IP address:    192.168.122.156
Domain name:   example.test
Realm name:    EXAMPLE.TEST

Configuring NTP daemon (ntpd)
  [1/4]: stopping ntpd
  [2/4]: writing configuration
  [3/4]: configuring ntpd to start on boot
  [4/4]: starting ntpd
Done configuring NTP daemon (ntpd).
Configuring directory server for the CA (pkids): Estimated time 30 seconds
  [1/3]: creating directory server user
  [2/3]: creating directory server instance
  [3/3]: restarting directory server
Done configuring directory server for the CA (pkids).
Configuring certificate server (pki-cad): Estimated time 3 minutes 30 seconds
  [1/4]: creating certificate server user
  [2/4]: creating pki-ca instance
  [3/4]: configuring certificate server instance
The next step is to get /root/ipa.csr signed by your CA and re-run ipa-server-install as:
ipa-server-install --external_cert_file=/path/to/signed_certificate --external_ca_file=/path/to/external_ca_certificate
[root@vm6 ~]# CADIR=/root/RootCA
[root@vm6 ~]# mkdir $CADIR
[root@vm6 ~]# cd $CADIR
[root@vm6 RootCA]# rm  -f *
[root@vm6 RootCA]# echo Secret123 > $CADIR/mypass1
[root@vm6 RootCA]# certutil -N -d $CADIR -f $CADIR/mypass1
[root@vm6 RootCA]# SERNUM=$RANDOM
[root@vm6 RootCA]# SERNUM=$(( SERNUM += 1 ))
[root@vm6 RootCA]# echo -e "y\n10\ny\n" | \
> certutil -S -d $CADIR \
>     -n RootCA \
>     -s "CN=MyRootCA, O=fakerealm1" \
>     -x \
>     -t "CTu,CTu,CTu" \
>     -g 2048 \
>     -m $SERNUM\
>     -v 60 \
>     -z /etc/group \
>     -2 \
>     --keyUsage certSigning \
>     --nsCertType sslCA,smimeCA,objectSigningCA \
>     -f $CADIR/mypass1


Generating key.  This may take a few moments...

Is this a CA certificate [y/N]?
Enter the path length constraint, enter to skip [<0 for unlimited path]: > Is this a critical extension [y/N]?
Notice: Trust flag u is set automatically if the private key is present.
[root@vm6 RootCA]# 
[root@vm6 RootCA]# 
[root@vm6 RootCA]# certutil -L -d $CADIR -n "RootCA" -a >> /root/ipacacert.asc
[root@vm6 RootCA]# 
[root@vm6 RootCA]# 
[root@vm6 RootCA]# 
[root@vm6 RootCA]# certutil -d $CADIR -C -m $RANDOM -a -i /root/ipa.csr -o /root/ipa.crt -c RootCA -1 -2
		0 - Digital Signature
		1 - Non-repudiation
		2 - Key encipherment
		3 - Data encipherment
		4 - Key agreement
		5 - Cert signing key
		6 - CRL signing key
		Other to finish
 > 0
		0 - Digital Signature
		1 - Non-repudiation
		2 - Key encipherment
		3 - Data encipherment
		4 - Key agreement
		5 - Cert signing key
		6 - CRL signing key
		Other to finish
 > 1
		0 - Digital Signature
		1 - Non-repudiation
		2 - Key encipherment
		3 - Data encipherment
		4 - Key agreement
		5 - Cert signing key
		6 - CRL signing key
		Other to finish
 > 5
		0 - Digital Signature
		1 - Non-repudiation
		2 - Key encipherment
		3 - Data encipherment
		4 - Key agreement
		5 - Cert signing key
		6 - CRL signing key
		Other to finish
 > 6
		0 - Digital Signature
		1 - Non-repudiation
		2 - Key encipherment
		3 - Data encipherment
		4 - Key agreement
		5 - Cert signing key
		6 - CRL signing key
		Other to finish
 > 9
Is this a critical extension [y/N]?
y
Is this a CA certificate [y/N]?
y
Enter the path length constraint, enter to skip [<0 for unlimited path]: > 
Is this a critical extension [y/N]?
y
Enter Password or Pin for "NSS Certificate DB":
[root@vm6 RootCA]# ipa-server-install -p Secret123 --external_cert_file /root/ipa.crt --external_ca_file /root/ipacacert.asc

The log file for this installation can be found in /var/log/ipaserver-install.log
==============================================================================
This program will set up the IPA Server.

This includes:
  * Configure a stand-alone CA (dogtag) for certificate management
  * Configure the Network Time Daemon (ntpd)
  * Create and configure an instance of Directory Server
  * Create and configure a Kerberos Key Distribution Center (KDC)
  * Configure Apache (httpd)

To accept the default shown in brackets, press the Enter key.


The IPA Master Server will be configured with:
Hostname:      vm6.example.test
IP address:    192.168.122.156
Domain name:   example.test
Realm name:    EXAMPLE.TEST

Configuring certificate server (pki-cad): Estimated time 3 minutes 30 seconds
  [1/20]: creating certificate server user
  [2/20]: configuring certificate server instance
  [3/20]: disabling nonces
  [4/20]: creating CA agent PKCS#12 file in /root
  [5/20]: creating RA agent certificate database
  [6/20]: importing CA chain to RA certificate database
  [7/20]: fixing RA database permissions
  [8/20]: setting up signing cert profile
  [9/20]: set up CRL publishing
  [10/20]: set certificate subject base
  [11/20]: enabling Subject Key Identifier
  [12/20]: setting audit signing renewal to 2 years
  [13/20]: configuring certificate server to start on boot
  [14/20]: restarting certificate server
  [15/20]: requesting RA certificate from CA
  [16/20]: issuing RA agent certificate
  [17/20]: adding RA agent as a trusted user
  [18/20]: configure certificate renewals
  [19/20]: configure Server-Cert certificate renewal
  [20/20]: Configure HTTP to proxy connections
Done configuring certificate server (pki-cad).
Configuring directory server (dirsrv): Estimated time 1 minute
  [1/38]: creating directory server user
  [2/38]: creating directory server instance
  [3/38]: adding default schema
  [4/38]: enabling memberof plugin
  [5/38]: enabling winsync plugin
  [6/38]: configuring replication version plugin
  [7/38]: enabling IPA enrollment plugin
  [8/38]: enabling ldapi
  [9/38]: disabling betxn plugins
  [10/38]: configuring uniqueness plugin
  [11/38]: configuring uuid plugin
  [12/38]: configuring modrdn plugin
  [13/38]: enabling entryUSN plugin
  [14/38]: configuring lockout plugin
  [15/38]: creating indices
  [16/38]: enabling referential integrity plugin
  [17/38]: configuring ssl for ds instance
  [18/38]: configuring certmap.conf
  [19/38]: configure autobind for root
  [20/38]: configure new location for managed entries
  [21/38]: restarting directory server
  [22/38]: adding default layout
  [23/38]: adding delegation layout
  [24/38]: adding replication acis
  [25/38]: creating container for managed entries
  [26/38]: configuring user private groups
  [27/38]: configuring netgroups from hostgroups
  [28/38]: creating default Sudo bind user
  [29/38]: creating default Auto Member layout
  [30/38]: adding range check plugin
  [31/38]: creating default HBAC rule allow_all
  [32/38]: Upload CA cert to the directory
  [33/38]: initializing group membership
  [34/38]: adding master entry
  [35/38]: configuring Posix uid/gid generation
  [36/38]: enabling compatibility plugin
  [37/38]: tuning directory server
  [38/38]: configuring directory to start on boot
Done configuring directory server (dirsrv).
Configuring Kerberos KDC (krb5kdc): Estimated time 30 seconds
  [1/10]: adding sasl mappings to the directory
  [2/10]: adding kerberos container to the directory
  [3/10]: configuring KDC
  [4/10]: initialize kerberos container
  [5/10]: adding default ACIs
  [6/10]: creating a keytab for the directory
  [7/10]: creating a keytab for the machine
  [8/10]: adding the password extension to the directory
  [9/10]: starting the KDC
  [10/10]: configuring KDC to start on boot
Done configuring Kerberos KDC (krb5kdc).
Configuring kadmin
  [1/2]: starting kadmin 
  [2/2]: configuring kadmin to start on boot
Done configuring kadmin.
Configuring ipa_memcached
  [1/2]: starting ipa_memcached 
  [2/2]: configuring ipa_memcached to start on boot
Done configuring ipa_memcached.
Configuring the web interface (httpd): Estimated time 1 minute
  [1/14]: setting mod_nss port to 443
  [2/14]: setting mod_nss protocol list to TLSv1.0 - TLSv1.2
  [3/14]: setting mod_nss password file
  [4/14]: enabling mod_nss renegotiate
  [5/14]: adding URL rewriting rules
  [6/14]: configuring httpd
  [7/14]: setting up ssl
  [8/14]: setting up browser autoconfig
  [9/14]: publish CA cert
  [10/14]: creating a keytab for httpd
  [11/14]: clean up any existing httpd ccache
  [12/14]: configuring SELinux for httpd
  [13/14]: restarting httpd
  [14/14]: configuring httpd to start on boot
Done configuring the web interface (httpd).
Applying LDAP updates
Restarting the directory server
Restarting the KDC
Sample zone file for bind has been created in /tmp/sample.zone.C0SFm7.db
Restarting the web server
==============================================================================
Setup complete

Next steps:
	1. You must make sure these network ports are open:
		TCP Ports:
		  * 80, 443: HTTP/HTTPS
		  * 389, 636: LDAP/LDAPS
		  * 88, 464: kerberos
		UDP Ports:
		  * 88, 464: kerberos
		  * 123: ntp

	2. You can now obtain a kerberos ticket using the command: 'kinit admin'
	   This ticket will allow you to use the IPA tools (e.g., ipa user-add)
	   and the web user interface.

Be sure to back up the CA certificate stored in /root/cacert.p12
This file is required to create replicas. The password for this
file is the Directory Manager password
Comment 12 errata-xmlrpc 2016-05-11 00:08:29 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-0874.html