Description of problem: When I install the 4.1 replica server (CentOS 7.1) I get this error on the new replica: ipa : CRITICAL CA DS schema check failed. Make sure the PKI service on the remote master is operational. When I restart IPA on the old 3.0 master (Centos 6.6) I get this: PKI-IPA...[22/Jun/2015:22:50:36 +0200] attr_syntax_create - Error: the EQUALITY matching rule [caseIgnoreIA5Match] is not compatible with the syntax [1.3.6.1.4.1.1466.115.121.1.15] for the attribute [dc] [22/Jun/2015:22:50:36 +0200] attr_syntax_create - Error: the SUBSTR matching rule [caseIgnoreIA5SubstringsMatch] is not compatible with the syntax [1.3.6.1.4.1.1466.115.121.1.15] for the attribute [dc] [ OK ] Version-Release number of selected component (if applicable): 3.0 master 4.1 replica How reproducible: Follow these steps for upgrading: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Linux_Domain_Identity_Authentication_and_Policy_Guide/upgrading.html Actual results: Messes up Master Schemas and cannot add the replica. Master still works OK Expected results: Have a 4.1 replica of the 3.0 master Additional info:
Hello Yamakasi, the error you see in the ds could be a false positive. I want to make sure that you run steps 3. and 4. of https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Linux_Domain_Identity_Authentication_and_Policy_Guide/migrating-ipa-proc.html If yes and you still receive the error please attach: * /var/log/ipareplica/ipareplica-install.log * /var/log
Comment 1 was sent prematurely: the second log should be: /var/log/dirsrv/slapd-YOUR-REALM/errors (of both servers) Besides that, based on your description, this bug is a most likely a duplicate of bug 1167196 , there is also related bug 1224769.
I checked the copy-schema-to-ca.py and it's the right version of what it should be. I don't see any strange errors on the master, but the replica gives me the following: Check SSH connection to remote master Execute check on remote master Check connection from master to remote replica 'ipa-01-71.domain.local': Directory Service: Unsecure port (389): OK Directory Service: Secure port (636): OK Kerberos KDC: TCP (88): OK Kerberos KDC: UDP (88): OK Kerberos Kpasswd: TCP (464): OK Kerberos Kpasswd: UDP (464): OK HTTP Server: Unsecure port (80): OK HTTP Server: Secure port (443): OK Connection from master to replica is OK. Connection check OK Using reverse zone(s) 3.16.172.in-addr.arpa. ipa : CRITICAL CA DS schema check failed. Make sure the PKI service on the remote master is operational. Your system may be partly configured. Run /usr/sbin/ipa-server-install --uninstall to clean up. Invalid password [root@ipa-01-71 ~]# tail -f /var/log/ipareplica-install.log File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 469, in result3 resp_ctrl_classes=resp_ctrl_classes File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 476, in result4 ldap_result = self._ldap_call(self._l.result4,msgid,all,timeout,add_ctrls,add_intermediates,add_extop) File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 99, in _ldap_call result = func(*args,**kwargs) 2015-06-27T18:20:51Z DEBUG The ipa-replica-install command failed, exception: INVALID_CREDENTIALS: {'desc': 'Invalid credentials'} But the credentials are OK.
We need more from the log, this snippet doesn't even contain the full traceback. But the error message indeed indicates that the supplied Directory Manager's password is incorrect. Are you able to do following LDAP search with the password on the 3.0 master: ldapsearch -D "cn=directory manager" -W -s ONE
I don't see any more logging in the logs, I need to recheck but I'm pretty sure there isn't more. I can do an ldap search, that is such strange.
tail with `-f` options prints couple of last lines of the log and then the lines which are appended to the log file while the command is running. I bet that `less /var/log/ipareplica-install.log` will show you much more.
HI Petr, There are more lines, only debug ones and no errors. It really fails on this Critical error. 2015-06-29T11:28:52Z DEBUG /usr/sbin/ipa-replica-install was invoked with argument "/var/lib/ipa/replica-info-ipa-01-71.domain.local.gpg" and options: {'no_forwarders': True, 'conf_ssh': True, 'skip_schema_check': False, 'ui_redirect': True, 'trust_sshfp': False, 'unattended': True, 'ip_addresses': [CheckedIPAddress('172.16.3.151')], 'no_host_dns': False, 'mkhomedir': False, 'no_reverse': False, 'setup_dns': True, 'create_sshfp': True, 'conf_sshd': True, 'forwarders': None, 'debug': False, 'conf_ntp': False, 'setup_ca': True, 'skip_conncheck': False, 'reverse_zones': []} 2015-06-29T11:28:52Z DEBUG IPA version 4.1.0-18.el7.centos.3 2015-06-29T11:28:52Z DEBUG Loading Index file from '/var/lib/ipa-client/sysrestore/sysrestore.index' 2015-06-29T11:28:52Z DEBUG Loading StateFile from '/var/lib/ipa/sysrestore/sysrestore.state' 2015-06-29T11:28:52Z DEBUG Loading Index file from '/var/lib/ipa/sysrestore/sysrestore.index' 2015-06-29T11:28:52Z DEBUG Starting external process 2015-06-29T11:28:52Z DEBUG args='/usr/sbin/httpd' '-t' '-D' 'DUMP_VHOSTS' 2015-06-29T11:28:52Z DEBUG Process finished, return code=0 2015-06-29T11:28:52Z DEBUG stdout=VirtualHost configuration: *:8443 is a NameVirtualHost default server ipa-01-71.domain.local (/etc/httpd/conf.d/nss.conf:86) port 8443 namevhost ipa-01-71.domain.local (/etc/httpd/conf.d/nss.conf:86) port 8443 namevhost ipa-01-71.domain.local (/etc/httpd/conf.d/nss.conf:86) 2015-06-29T11:28:52Z DEBUG stderr= 2015-06-29T11:29:05Z DEBUG Starting external process 2015-06-29T11:29:05Z DEBUG args='/usr/bin/gpg-agent' '--batch' '--homedir' '/tmp/tmpmbCjioipa/ipa-7m8PK5/.gnupg' '--daemon' '/usr/bin/gpg' '--batch' '--homedir' '/tmp/tmpmbCjioipa/ipa-7m8PK5/.gnupg' '--passphrase-fd' '0' '--yes' '--no-tty' '-o' '/tmp/tmpmbCjioipa/files.tar' '-d' '/var/lib/ipa/replica-info-ipa-01-71.domain.local.gpg' 2015-06-29T11:29:05Z DEBUG Process finished, return code=0 2015-06-29T11:29:05Z DEBUG Starting external process 2015-06-29T11:29:05Z DEBUG args='tar' 'xf' '/tmp/tmpmbCjioipa/files.tar' '-C' '/tmp/tmpmbCjioipa' 2015-06-29T11:29:05Z DEBUG Process finished, return code=0 2015-06-29T11:29:05Z DEBUG stdout= 2015-06-29T11:29:05Z DEBUG stderr= 2015-06-29T11:29:05Z DEBUG Installing replica file with version 300 (0 means no version in prepared file). 2015-06-29T11:29:05Z DEBUG Check if ipa-01-71.domain.local is a primary hostname for localhost 2015-06-29T11:29:05Z DEBUG Primary hostname for localhost: ipa-01-71.domain.local 2015-06-29T11:29:05Z DEBUG Search DNS for ipa-01-71.domain.local 2015-06-29T11:29:05Z DEBUG Check if ipa-01-71.domain.local is not a CNAME 2015-06-29T11:29:05Z DEBUG Check reverse address of 172.16.3.151 2015-06-29T11:29:05Z DEBUG Found reverse name: ipa-01-71.domain.local 2015-06-29T11:29:05Z DEBUG Check if ipa-01.domain.local is a primary hostname for localhost 2015-06-29T11:29:05Z DEBUG Primary hostname for localhost: ipa-01.domain.local 2015-06-29T11:29:05Z DEBUG Search DNS for ipa-01.domain.local 2015-06-29T11:29:05Z DEBUG Check if ipa-01.domain.local is not a CNAME 2015-06-29T11:29:05Z DEBUG Check reverse address of 172.16.3.251 2015-06-29T11:29:05Z DEBUG Found reverse name: ipa-01.domain.local 2015-06-29T11:29:05Z DEBUG Starting external process 2015-06-29T11:29:05Z DEBUG args='/usr/sbin/ipa-replica-conncheck' '--master' 'ipa-01.domain.local' '--auto-master-check' '--realm' 'domain.local' '--principal' 'admin' '--hostname' 'ipa-01-71.domain.local' '--check-ca' 2015-06-29T11:29:15Z DEBUG Process finished, return code=0 2015-06-29T11:29:15Z DEBUG Starting external process 2015-06-29T11:29:15Z DEBUG args='/sbin/ip' '-family' 'inet' '-oneline' 'address' 'show' 2015-06-29T11:29:15Z DEBUG Process finished, return code=0 2015-06-29T11:29:15Z DEBUG stdout=1: lo inet 127.0.0.1/8 scope host lo\ valid_lft forever preferred_lft forever 2: eth0 inet 172.16.3.151/24 brd 172.16.3.255 scope global dynamic eth0\ valid_lft 489sec preferred_lft 489sec 2015-06-29T11:29:15Z DEBUG stderr= 2015-06-29T11:29:15Z DEBUG importing all plugin modules in '/usr/lib/python2.7/site-packages/ipalib/plugins'... 2015-06-29T11:29:15Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/aci.py' 2015-06-29T11:29:15Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/automember.py' 2015-06-29T11:29:15Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/automount.py' 2015-06-29T11:29:15Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/baseldap.py' 2015-06-29T11:29:15Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/batch.py' 2015-06-29T11:29:15Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/cert.py' 2015-06-29T11:29:15Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/config.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/delegation.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/dns.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/group.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/hbacrule.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/hbacsvc.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/hbacsvcgroup.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/hbactest.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/host.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/hostgroup.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/idrange.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/idviews.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/internal.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/kerberos.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/krbtpolicy.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/migration.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/misc.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/netgroup.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/otptoken.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/otptoken_yubikey.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/passwd.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/permission.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/ping.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/pkinit.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/privilege.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/pwpolicy.py' 2015-06-29T11:29:16Z DEBUG Starting external process 2015-06-29T11:29:16Z DEBUG args='klist' '-V' 2015-06-29T11:29:16Z DEBUG Process finished, return code=0 2015-06-29T11:29:16Z DEBUG stdout=Kerberos 5 version 1.12.2 2015-06-29T11:29:16Z DEBUG stderr= 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/radiusproxy.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/realmdomains.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/role.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/rpcclient.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/selfservice.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/selinuxusermap.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/service.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/sudocmd.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/sudocmdgroup.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/sudorule.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/trust.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/user.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/virtual.py' 2015-06-29T11:29:16Z DEBUG importing all plugin modules in '/usr/lib/python2.7/site-packages/ipaserver/install/plugins'... 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipaserver/install/plugins/adtrust.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipaserver/install/plugins/baseupdate.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipaserver/install/plugins/ca_renewal_master.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipaserver/install/plugins/dns.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipaserver/install/plugins/fix_replica_agreements.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipaserver/install/plugins/rename_managed.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipaserver/install/plugins/update_idranges.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipaserver/install/plugins/update_managed_permissions.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipaserver/install/plugins/update_pacs.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipaserver/install/plugins/update_passsync.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipaserver/install/plugins/update_referint.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipaserver/install/plugins/update_services.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipaserver/install/plugins/update_uniqueness.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipaserver/install/plugins/updateclient.py' 2015-06-29T11:29:16Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipaserver/install/plugins/upload_cacrt.py' 2015-06-29T11:29:17Z DEBUG group dirsrv exists 2015-06-29T11:29:17Z DEBUG user dirsrv exists 2015-06-29T11:29:17Z DEBUG Created connection context.ldap2_40587920 2015-06-29T11:29:17Z DEBUG flushing ldaps://ipa-01.domain.local from SchemaCache 2015-06-29T11:29:17Z DEBUG retrieving schema for SchemaCache url=ldaps://ipa-01.domain.local conn=<ldap.ldapobject.SimpleLDAPObject instance at 0x3c6b6c8> 2015-06-29T11:29:18Z DEBUG Created connection context.ldap2 2015-06-29T11:29:18Z DEBUG flushing ldaps://ipa-01.domain.local from SchemaCache 2015-06-29T11:29:18Z DEBUG retrieving schema for SchemaCache url=ldaps://ipa-01.domain.local conn=<ldap.ldapobject.SimpleLDAPObject instance at 0x429d488> 2015-06-29T11:29:18Z DEBUG Destroyed connection context.ldap2 2015-06-29T11:29:18Z DEBUG Check forward/reverse DNS resolution 2015-06-29T11:29:18Z DEBUG Search DNS server ipa-01.domain.local (['172.16.3.251', '172.16.3.251', '172.16.3.251']) for ipa-01.domain.local 2015-06-29T11:29:18Z DEBUG Check reverse address 172.16.3.251 (ipa-01.domain.local) 2015-06-29T11:29:18Z DEBUG Address 172.16.3.251 resolves to: ipa-01.domain.local.. 2015-06-29T11:29:18Z DEBUG Search DNS server ipa-01.domain.local (['172.16.3.251', '172.16.3.251', '172.16.3.251']) for ipa-01-71.domain.local 2015-06-29T11:29:18Z DEBUG Check reverse address 172.16.3.151 (ipa-01-71.domain.local) 2015-06-29T11:29:18Z DEBUG Address 172.16.3.151 resolves to: ipa-01-71.domain.local.. 2015-06-29T11:29:18Z DEBUG Destroyed connection context.ldap2_40587920 2015-06-29T11:29:18Z DEBUG Loading StateFile from '/var/lib/ipa/sysrestore/sysrestore.state' 2015-06-29T11:29:18Z DEBUG Checking if IPA schema is present in ldap://ipa-01.domain.local:7389 2015-06-29T11:29:18Z CRITICAL CA DS schema check failed. Make sure the PKI service on the remote master is operational. 2015-06-29T11:29:18Z DEBUG File "/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py", line 646, in run_script return_value = main_function() File "/usr/sbin/ipa-replica-install", line 691, in main cainstance.replica_ca_install_check(config) File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 1798, in replica_ca_install_check config.dirman_password) File "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 663, in simple_bind_s return self.conn.simple_bind_s(who, cred, serverctrls, clientctrls) File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 208, in simple_bind_s resp_type, resp_data, resp_msgid, resp_ctrls = self.result3(msgid,all=1,timeout=self.timeout) File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 469, in result3 resp_ctrl_classes=resp_ctrl_classes File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 476, in result4 ldap_result = self._ldap_call(self._l.result4,msgid,all,timeout,add_ctrls,add_intermediates,add_extop) File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 99, in _ldap_call result = func(*args,**kwargs) 2015-06-29T11:29:18Z DEBUG The ipa-replica-install command failed, exception: INVALID_CREDENTIALS: {'desc': 'Invalid credentials'}
Weird, It still points to wrong credentials. Could you execute following search from 4.1 replica: ldapsearch -H ldap://ipa-01.domain .local:7389 -D "cn=directory manager" -W -s ONE -b o=ipaca Does it return successfully with e.g. "numEntries: 9" on the end of response
This bug appears to have been reported against 'rawhide' during the Fedora 23 development cycle. Changing version to '23'. (As we did not run this process for some time, it could affect also pre-Fedora 23 development cycle bugs. We are very sorry. It will help us with cleanup during Fedora 23 End Of Life. Thank you.) More information and reason for this action is here: https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Fedora23
waiting for answer for question in comment 8
Hi Petr, Sorry I was on something else that had priority. I cannot run that command on the replica as the installation failed. I even tried it with resetting the Directory Manager password to something very simple but that didn't change it also. Same PKI error.
you don't have to have ipa-server installed on the server in order to run ldapsearch against other server. When changing Directory Manager password, make sure to follow: http://www.freeipa.org/page/Howto/Change_Directory_Manager_Password
True, but is there something wrong in that command as I wasn't able to run it on the ipa master itself also. ldapsearch -D "cn=directory manager" -W -s ONE gives me reply, your command gives me indeed "ldap_bind: Invalid credentials (49)"
For the record, I've just discussed another occurrence of this bug and the cause was that REALM dirsrv instance had a different directory manager password than the PKI(CA) instance. Could be checked by comparing nsslapd-rootpw in - /etc/dirsrv/slapd-MY-REALM-TEST/dse.ldif - /etc/dirsrv/slapd-PKI-IPA/dse.ldif
OK, the password issue is solved. This was an issue between the CA and LDAP password. Now the install starts but ends with: 2015-07-20T11:08:44Z DEBUG Saving StateFile to '/var/lib/ipa/sysrestore/sysrestore.state' 2015-07-20T11:08:44Z DEBUG Starting external process 2015-07-20T11:08:44Z DEBUG args='/usr/sbin/pkispawn' '-s' 'CA' '-f' '/tmp/tmp3IqrOU' 2015-07-20T11:12:36Z DEBUG Process finished, return code=1 2015-07-20T11:12:36Z DEBUG stdout=Loading deployment configuration from /tmp/tmp3IqrOU. Installing CA into /var/lib/pki/pki-tomcat. Storing deployment configuration into /etc/sysconfig/pki/tomcat/pki-tomcat/ca/deployment.cfg. Installation failed. 2015-07-20T11:12:36Z DEBUG stderr=pkispawn : WARNING ....... unable to validate security domain user/password through REST interface. Interface not available pkispawn : ERROR ....... Exception from Java Configuration Servlet: Error while updating security domain: java.io.IOException: java.io.IOException: SocketException cannot read on socket 2015-07-20T11:12:36Z CRITICAL failed to configure ca instance Command ''/usr/sbin/pkispawn' '-s' 'CA' '-f' '/tmp/tmp3IqrOU'' returned non-zero exit status 1 2015-07-20T11:12:36Z DEBUG Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 382, in start_creation run_step(full_msg, method) File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 372, in run_step method() File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 673, in __spawn_instance raise RuntimeError('Configuration of CA failed') RuntimeError: Configuration of CA failed 2015-07-20T11:12:36Z DEBUG [error] RuntimeError: Configuration of CA failed 2015-07-20T11:12:36Z DEBUG File "/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py", line 646, in run_script return_value = main_function() File "/usr/sbin/ipa-replica-install", line 703, in main CA = cainstance.install_replica_ca(config) File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 1869, in install_replica_ca subject_base=config.subject_base) File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 520, in configure_instance self.start_creation(runtime=210) File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 382, in start_creation run_step(full_msg, method) File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 372, in run_step method() File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 673, in __spawn_instance raise RuntimeError('Configuration of CA failed') 2015-07-20T11:12:36Z DEBUG The ipa-replica-install command failed, exception: RuntimeError: Configuration of CA failed AND: 2015-07-20 11:08:48 pkispawn : DEBUG ........... chown 995:993 /root/.dogtag/pki-tomcat/ca/pkcs12_password.conf 2015-07-20 11:08:48 pkispawn : INFO ....... executing 'certutil -N -d /tmp/tmp-rbfc7m -f /root/.dogtag/pki-tomcat/ca/password.conf' 2015-07-20 11:08:48 pkispawn : INFO ....... executing 'systemctl daemon-reload' 2015-07-20 11:08:48 pkispawn : INFO ....... executing 'systemctl start pki-tomcatd' 2015-07-20 11:08:49 pkispawn : DEBUG ........... No connection - server may still be down 2015-07-20 11:08:49 pkispawn : DEBUG ........... No connection - exception thrown: [Errno 111] Connection refused 2015-07-20 11:08:50 pkispawn : DEBUG ........... No connection - server may still be down 2015-07-20 11:08:50 pkispawn : DEBUG ........... No connection - exception thrown: [Errno 111] Connection refused 2015-07-20 11:08:51 pkispawn : DEBUG ........... No connection - server may still be down 2015-07-20 11:08:51 pkispawn : DEBUG ........... No connection - exception thrown: [Errno 111] Connection refused 2015-07-20 11:08:52 pkispawn : DEBUG ........... No connection - server may still be down 2015-07-20 11:08:52 pkispawn : DEBUG ........... No connection - exception thrown: [Errno 111] Connection refused 2015-07-20 11:08:53 pkispawn : DEBUG ........... No connection - server may still be down 2015-07-20 11:08:53 pkispawn : DEBUG ........... No connection - exception thrown: [Errno 111] Connection refused 2015-07-20 11:08:54 pkispawn : DEBUG ........... No connection - server may still be down 2015-07-20 11:08:54 pkispawn : DEBUG ........... No connection - exception thrown: [Errno 111] Connection refused 2015-07-20 11:09:06 pkispawn : DEBUG ........... <?xml version="1.0" encoding="UTF-8" standalone="no"?><XMLResponse><State>0</State><Type>CA</Type><St$ 2015-07-20 11:09:07 pkispawn : INFO ....... constructing PKI configuration data. 2015-07-20 11:09:07 pkispawn : INFO ....... configuring PKI configuration data. 2015-07-20 11:12:35 pkispawn : ERROR ....... Exception from Java Configuration Servlet: Error while updating security domain: java.io.IOException: jav$ 2015-07-20 11:12:35 pkispawn : DEBUG ....... Error Type: HTTPError 2015-07-20 11:12:35 pkispawn : DEBUG ....... Error Message: 500 Server Error: Internal Server Error 2015-07-20 11:12:35 pkispawn : DEBUG ....... File "/usr/sbin/pkispawn", line 463, in main rv = instance.spawn(deployer) File "/usr/lib/python2.7/site-packages/pki/server/deployment/scriptlets/configuration.py", line 126, in spawn json.dumps(data, cls=pki.encoder.CustomTypeEncoder)) File "/usr/lib/python2.7/site-packages/pki/server/deployment/pkihelper.py", line 3211, in configure_pki_data response = client.configure(data) File "/usr/lib/python2.7/site-packages/pki/system.py", line 80, in configure r = self.connection.post('/rest/installer/configure', data, headers) File "/usr/lib/python2.7/site-packages/pki/client.py", line 64, in post r.raise_for_status() File "/usr/lib/python2.7/site-packages/requests/models.py", line 638, in raise_for_status raise http_error Might it be that I need to update the cacert.p12 on the master where I generate the replica file ?
Created attachment 1054405 [details] pki-tomcat-ca-debug loggin after replica install from replica
Created attachment 1054409 [details] pki-tomcat-ca-debug loggin after replica install from replica
It seems that when I reset the ldap password something goes wrong. - An ldapsearch can be done with the NEW set password - An ldappasswd can only be done with the OLD password Dogtag not updated in some way ?
All passwords reset and double checked, it goes wrong here after getting the XML from the master. It can receive the XML: 2015-07-22 22:29:21 pkispawn : INFO ....... modifying '/root/.dogtag/pki-tomcat/ca/password.conf' 2015-07-22 22:29:21 pkispawn : DEBUG ........... chmod 660 /root/.dogtag/pki-tomcat/ca/password.conf 2015-07-22 22:29:21 pkispawn : DEBUG ........... chown 0:0 /root/.dogtag/pki-tomcat/ca/password.conf 2015-07-22 22:29:21 pkispawn : INFO ....... generating '/root/.dogtag/pki-tomcat/ca/pkcs12_password.conf' 2015-07-22 22:29:21 pkispawn : INFO ....... modifying '/root/.dogtag/pki-tomcat/ca/pkcs12_password.conf' 2015-07-22 22:29:21 pkispawn : DEBUG ........... chmod 660 /root/.dogtag/pki-tomcat/ca/pkcs12_password.conf 2015-07-22 22:29:21 pkispawn : DEBUG ........... chown 995:993 /root/.dogtag/pki-tomcat/ca/pkcs12_password.conf 2015-07-22 22:29:21 pkispawn : INFO ....... executing 'certutil -N -d /tmp/tmp-NYQvku -f /root/.dogtag/pki-tomcat/ca/password.conf' 2015-07-22 22:29:21 pkispawn : INFO ....... executing 'systemctl daemon-reload' 2015-07-22 22:29:21 pkispawn : INFO ....... executing 'systemctl start pki-tomcatd' 2015-07-22 22:29:22 pkispawn : DEBUG ........... No connection - server may still be down 2015-07-22 22:29:22 pkispawn : DEBUG ........... No connection - exception thrown: [Errno 111] Connection refused 2015-07-22 22:29:23 pkispawn : DEBUG ........... No connection - server may still be down 2015-07-22 22:29:23 pkispawn : DEBUG ........... No connection - exception thrown: [Errno 111] Connection refused 2015-07-22 22:29:24 pkispawn : DEBUG ........... No connection - server may still be down 2015-07-22 22:29:24 pkispawn : DEBUG ........... No connection - exception thrown: [Errno 111] Connection refused 2015-07-22 22:29:37 pkispawn : DEBUG ........... <?xml version="1.0" encoding="UTF-8" standalone="no"?><XMLResponse><State>0 </State><Type>CA</Type><Status>running</Status><Version>10.1.2-7.el7</Version></XMLResponse> 2015-07-22 22:29:38 pkispawn : INFO ....... constructing PKI configuration data. 2015-07-22 22:29:38 pkispawn : INFO ....... configuring PKI configuration data. 2015-07-22 22:29:40 pkispawn : ERROR ....... Exception from Java Configuration Servlet: Failed to obtain installation token from security domain 2015-07-22 22:29:40 pkispawn : DEBUG ....... Error Type: HTTPError 2015-07-22 22:29:40 pkispawn : DEBUG ....... Error Message: 500 Server Error: Internal Server Error 2015-07-22 22:29:40 pkispawn : DEBUG ....... File "/usr/sbin/pkispawn", line 463, in main rv = instance.spawn(deployer) File "/usr/lib/python2.7/site-packages/pki/server/deployment/scriptlets/configuration.py", line 126, in spawn json.dumps(data, cls=pki.encoder.CustomTypeEncoder)) File "/usr/lib/python2.7/site-packages/pki/server/deployment/pkihelper.py", line 3211, in configure_pki_data response = client.configure(data) File "/usr/lib/python2.7/site-packages/pki/system.py", line 80, in configure r = self.connection.post('/rest/installer/configure', data, headers) File "/usr/lib/python2.7/site-packages/pki/client.py", line 64, in post r.raise_for_status() File "/usr/lib/python2.7/site-packages/requests/models.py", line 638, in raise_for_status raise http_error
After some digging and some other bug I found, I think it's the XML which is not OK 23:50 < YamakasY> ok, maybe some xml issue on the format ? 23:52 < YamakasY> kewl found it ! 23:52 < YamakasY> securityDomainName=null, securityDomainUser=admin, securityDomainPassword=XXXX 23:53 < YamakasY> and that is indeed NOT in the XML <XMLResponse><DomainInfo><?xml version="1.0" encoding="UTF-8" standalone="no"?><DomainInfo><Name>IPA</Name><CAList><CA><Host>ipa-01.domain.local</Host><SecurePort>443</SecurePort><SecureAgentPort>443</SecureAgentPort><SecureAdminPort>443</SecureAdminPort><SecureEEClientAuthPort>443</SecureEEClientAuthPort><UnSecurePort>80</UnSecurePort><Clone>FALSE</Clone><SubsystemName>pki-cad</SubsystemName><DomainManager>TRUE</DomainManager></CA><SubsystemCount>1</SubsystemCount></CAList><OCSPList><SubsystemCount>0</SubsystemCount></OCSPList><KRAList><SubsystemCount>0</SubsystemCount></KRAList><RAList><SubsystemCount>0</SubsystemCount></RAList><TKSList><SubsystemCount>0</SubsystemCount></TKSList><TPSList><SubsystemCount>0</SubsystemCount></TPSList></DomainInfo></DomainInfo><Status>0</Status></XMLResponse> Refering to this: https://fedorahosted.org/pki/ticket/1235 Where I came across because of some 3-topic mailinglist item which had the same issue only on 3.3 I don't see a REALM or Domain ?
OK, password was still not updated well using the docs, this is covered now with help from Ade. Now the replica fails on: [23/Jul/2015:18:13:11][http-bio-8443-exec-3]: Cloning a domain master [23/Jul/2015:18:13:11][http-bio-8443-exec-3]: WizardPanelBase updateDomainXML start hostname=ipa-01.domain.local port=443 [23/Jul/2015:18:13:11][http-bio-8443-exec-3]: updateSecurityDomain: failed to update security domain using admin port 443:org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 50; White spaces are required between publicId and systemId. [23/Jul/2015:18:13:11][http-bio-8443-exec-3]: updateSecurityDomain: now trying agent port with client auth [23/Jul/2015:18:13:11][http-bio-8443-exec-3]: WizardPanelBase updateDomainXML start hostname=ipa-01.domain.local port=443 [23/Jul/2015:18:13:11][http-bio-8443-exec-3]: updateDomainXML() nickname=subsystemCert cert-pki-ca [23/Jul/2015:18:13:12][http-bio-8443-exec-3]: WizardPanelBase updateDomainXML: status=1
The replica is almost complete, but I meed a few things to diagnose where it went wrong. Specifically .. debug log on the master /var/log/pki-ca/debug catalina.out on the master: /var/log/pki-ca/catalina,out debug log pn replica /var/log/pki/pki-tomcat/ca/debug journal on the replica : journalctl -u pki-tomcatd IPA replica instal log -- so I can see the exception thrown. Ade
OK, as we know I need to update the certificate but I get an error on this ldapmodify: dn: uid=CA-ipa.example.com-9443,ou=people,o=ipaca changetype: modify replace: usercertificate usercertificate:< file:///tmp/subsystem.der - replace: description description: 2;123;CN=Certificate Authority,O=EXAMPLE.COM;CN=CA Subsystem,O=EXAMPLE.COM ldapmodify: wrong attributeType at line 4, entry "uid=CA-ipa.example.com-9443,ou=people,o=ipaca"
What is the result of this bug? Can it be closed? If others experience it: if the 6.7 was a master with externally signed CA cert, there is a bug 1256039 which says that there the cause is missing PKI subsystem user. Workaround text from the bug: """ 3. Based on master's system log it looks like the subsystem certificate provided by the replica cannot be mapped into a user on the master: 1850.TP-Processor8 - [23/Oct/2015:12:47:30 CDT] [6] [3] Cannot authenticate agent with certificate Serial 0x3 Subject DN CN=CA Subsystem,O=EXAMPLE.COM. Error: User not found 1850.TP-Processor8 - [23/Oct/2015:12:47:30 CDT] [3] [3] Servlet caUpdateDomainXML: Failed to authorize: Invalid Credential.. To help troubleshooting ideally this message should be logged in the debug log as well in the next RHEL 6 update. 4. The authentication problem might be caused by a missing subsystem user (bug #1225589) and there's already a tool to restore it. However, before running the script, please run this command on the master to verify the problem: $ pki-server ca-group-member-find "Subsystem Group" Ideally it should return a user ID "CA-<hostname>-9443" and the description attribute should contain the subsystem certificate in this format "<version>;<serial>;<issuer DN>;<subject DN>". If that's not the case, please run this tool to restore the subsystem user: $ python /usr/share/pki/scripts/restore-subsystem-user.py Then run this command again to verify the fix: $ pki-server ca-group-member-find "Subsystem Group" If everything works well, please try installing the replica again. """
RHEL 6.7 recently received update which fixes some issues with migration procedure from 3.0 to 4.x. See the bugs in comment 24. Closing this bug due to lack of activity.