RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1650155 - [DOC] IPA Replica installation fails as part of IPA-mIgration process
Summary: [DOC] IPA Replica installation fails as part of IPA-mIgration process
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: pki-core
Version: 7.6
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: pre-dev-freeze
: ---
Assignee: Marc Muehlfeld
QA Contact: Asha Akkiangady
Marc Muehlfeld
URL:
Whiteboard:
: 1667434 (view as bug list)
Depends On:
Blocks: 1672180
TreeView+ depends on / blocked
 
Reported: 2018-11-15 13:14 UTC by Nikhil Dehadrai
Modified: 2019-03-11 23:36 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: Known Issue
Doc Text:
Installing an IdM replica on RHEL 7.6 fails if the IdM master runs on RHEL 6 With the update of the _pki-core_ package provided in Red Hat Enterprise Linux (RHEL) 7.6, certain ciphers that are not supported by hardware security modules (HSM) are no longer enabled by default in Certificate System. As a consequence, setting up Identity Management (IdM) on RHEL 7.6 as a replica with a master running on RHEL 6 fails with a "CRITICAL Failed to configure CA instance" error. To work around the problem, append the following entry to the end of the "NSSCipherSuite" parameter in the `/etc/httpd/conf.d/nss.conf` file: +ecdhe_rsa_aes_128_sha,+ecdhe_rsa_aes_256_sha As a result, the IdM installation on RHEL 7.6 no longer fails.
Clone Of:
Environment:
Last Closed: 2019-02-28 12:57:37 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)

Description Nikhil Dehadrai 2018-11-15 13:14:19 UTC 
Description of problem:
Replica installation fails against DL0 IPA master, causing the IPA-migration test suite to fail.

Version-Release number of selected component (if applicable):
RHEL 7.6up1

How reproducible:
Always

Steps to Reproduce:
1. Setup IPA master on RHEL 6.10
2. Setup IPA-REPLICA on RHEL 7.6up1
# prepare gpg file for replica on master
# copy gpg filr to replica
# run ipa-replica install comand using 
/usr/sbin/ipa-replica-install -U --setup-ca --setup-dns --forwarder=10.x.x.x -w Secret123 -p Secret123 /opt/rhqa_ipa/replica-info-intel-piketon-02.testrelm.test.gpg


Actual results:
Replica install fails

:: [ 02:35:41 ] :: [  BEGIN   ] :: Running ' /usr/sbin/ipa-replica-install -U --setup-ca --setup-dns --forwarder=10.x.x.x -w Secret123 -p Secret123 /opt/rhqa_ipa/replica-info-intel-piketon-02.testrelm.test.gpg'
..
..
..
..
..
Configuring ipa-custodia
  [1/5]: Making sure custodia container exists
  [2/5]: Generating ipa-custodia config file
  [3/5]: Generating ipa-custodia keys
  [4/5]: starting ipa-custodia 
  [5/5]: configuring ipa-custodia to start on boot
Done configuring ipa-custodia.
Configuring certificate server (pki-tomcatd). Estimated time: 3 minutes
  [1/28]: configuring certificate server instance
ipaserver.install.dogtaginstance: CRITICAL Failed to configure CA instance: Command '/usr/sbin/pkispawn -s CA -f /tmp/tmpxQv9bc' returned non-zero exit status 1
ipaserver.install.dogtaginstance: CRITICAL See the installation logs and the following files/directories for more information:
ipaserver.install.dogtaginstance: CRITICAL   /var/log/pki/pki-tomcat
  [error] RuntimeError: CA configuration failed.
ipapython.admintool: ERROR    CA configuration failed.
ipapython.admintool: ERROR    The ipa-replica-install command failed. See /var/log/ipareplica-install.log for more information
Your system may be partly configured.
Run /usr/sbin/ipa-server-install --uninstall to clean up.


Expected results:
Ipa replica installation against DL0 master should be successful, as a result ipa-migration should be successful
Comment 5 Alexander Bokovoy 2018-11-15 16:38:57 UTC 
It looks like the failure is at pki side where a request to create a server certificate fails due to inability to write to SSL socket internally:

[15/Nov/2018:06:21:32][http-bio-8443-exec-3]: SystemConfigService: request: ConfigurationRequest [pin=XXXX, token=Internal Key Storage Token, tokenPassword=XXXX, securityDomainType=existingdomain, securityDomainUri=https://ibm-x3500m4-01.testrelm.test:443, securityDomainName=null, securityDomainUser=admin, securityDomainPassword=XXXX, securityDomainPostLoginSleepSeconds=null, isClone=true, cloneUri=https://ibm-x3500m4-01.testrelm.test:443, subsystemName=CA mgmt3.testrelm.test 8443, p12File=/tmp/ca.p12, p12Password=XXXX, hierarchy=root, dsHost=mgmt3.testrelm.test, dsPort=389, baseDN=o=ipaca, bindDN=cn=Directory Manager, bindpwd=XXXX, database=ipaca, secureConn=false, removeData=true, replicateSchema=false, masterReplicationPort=7389, cloneReplicationPort=389, replicationSecurity=TLS, systemCertsImported=false, systemCerts=[com.netscape.certsrv.system.SystemCertData@2977e3a5], issuingCA=https://ibm-x3500m4-01.testrelm.test:443, backupKeys=true, backupPassword=XXXX, backupFile=/etc/pki/pki-tomcat/alias/ca_backup_keys.p12, adminUID=null, adminPassword=XXXX, adminEmail=null, adminCertRequest=null, adminCertRequestType=null, adminSubjectDN=null, adminName=null, adminProfileID=null, adminCert=null, importAdminCert=false, generateServerCert=true, external=false, standAlone=false, authdbBaseDN=null, authdbHost=null, authdbPort=null, authdbSecureConn=null, caUri=null, kraUri=null, tksUri=null, enableServerSideKeyGen=null, importSharedSecret=null, generateSubsystemCert=null, sharedDB=false, sharedDBUserDN=null, createNewDB=true, setupReplication=True, subordinateSecurityDomainName=null, reindexData=False, startingCrlNumber=0, createSigningCertRecord=true, signingCertSerialNumber=1]
[15/Nov/2018:06:21:32][http-bio-8443-exec-3]: === Token Authentication ===
[15/Nov/2018:06:21:32][http-bio-8443-exec-3]: === Security Domain Configuration ===
[15/Nov/2018:06:21:32][http-bio-8443-exec-3]: Joining existing security domain
[15/Nov/2018:06:21:32][http-bio-8443-exec-3]: Resolving security domain URL https://ibm-x3500m4-01.testrelm.test:443
[15/Nov/2018:06:21:32][http-bio-8443-exec-3]: Getting security domain cert chain
[15/Nov/2018:06:21:32][http-bio-8443-exec-3]: ConfigurationUtils.importCertChain()
[15/Nov/2018:06:21:33][http-bio-8443-exec-3]: ConfigurationUtils: GET https://ibm-x3500m4-01.testrelm.test:443/ca/admin/ca/getCertChain
javax.ws.rs.ProcessingException: Unable to invoke request
	at org.jboss.resteasy.client.jaxrs.engines.ApacheHttpClient4Engine.invoke(ApacheHttpClient4Engine.java:287)
	at org.jboss.resteasy.client.jaxrs.internal.ClientInvocation.invoke(ClientInvocation.java:407)
	at org.jboss.resteasy.client.jaxrs.internal.ClientInvocation.invoke(ClientInvocation.java:442)
	at org.jboss.resteasy.client.jaxrs.internal.ClientInvocationBuilder.get(ClientInvocationBuilder.java:165)
	at com.netscape.certsrv.client.PKIConnection.get(PKIConnection.java:500)
	at com.netscape.cms.servlet.csadmin.ConfigurationUtils.get(ConfigurationUtils.java:259)
	at com.netscape.cms.servlet.csadmin.ConfigurationUtils.importCertChain(ConfigurationUtils.java:289)
	at org.dogtagpki.server.rest.SystemConfigService.logIntoSecurityDomain(SystemConfigService.java:918)
	at org.dogtagpki.server.rest.SystemConfigService.configureSecurityDomain(SystemConfigService.java:879)
	at org.dogtagpki.server.rest.SystemConfigService.configure(SystemConfigService.java:140)
	at org.dogtagpki.server.rest.SystemConfigService.configure(SystemConfigService.java:101)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:137)
	at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:280)
	at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:234)
	at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:221)
	at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:356)
	at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:179)
	at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:220)
	at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
	at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
	at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)
	at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:175)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:297)
	at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:55)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:191)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:187)
	at java.security.AccessController.doPrivileged(Native Method)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:186)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
	at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)
	at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:260)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:237)
	at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:55)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:191)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:187)
	at java.security.AccessController.doPrivileged(Native Method)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:186)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:218)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:506)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)
	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1087)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
	at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: SocketException cannot write on socket
	at org.mozilla.jss.ssl.SSLSocket.write(SSLSocket.java:1503)
	at org.mozilla.jss.ssl.SSLOutputStream.write(SSLOutputStream.java:24)
	at org.apache.http.impl.io.AbstractSessionOutputBuffer.flushBuffer(AbstractSessionOutputBuffer.java:147)
	at org.apache.http.impl.io.AbstractSessionOutputBuffer.flush(AbstractSessionOutputBuffer.java:154)
	at org.apache.http.impl.AbstractHttpClientConnection.doFlush(AbstractHttpClientConnection.java:278)
	at org.apache.http.impl.AbstractHttpClientConnection.flush(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.flush(ManagedClientConnectionImpl.java:175)
	at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:260)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:715)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:520)
	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
	at org.jboss.resteasy.client.jaxrs.engines.ApacheHttpClient4Engine.invoke(ApacheHttpClient4Engine.java:283)
	... 72 more
[15/Nov/2018:06:21:33][http-bio-8443-exec-3]: SignedAuditLogger: event ACCESS_SESSION_TERMINATED
Comment 6 Endi Sukma Dewata 2018-11-15 19:19:17 UTC 
This is a very generic SSL failure without any indication of what's causing it.

Has any other replication test been done using the same RHEL 7.6up1 build? Are they failing as well with the same error message?

Could you manually check if the proxy URL is accessible using the replica's NSS database (i.e. /etc/pki/pki-tomcat/alias)?
https://ibm-x3500m4-01.testrelm.test:443/ca/admin/ca/getCertChain

Could you perform the same test against the Tomcat URL directly?
https://ibm-x3500m4-01.testrelm.test:8443/ca/admin/ca/getCertChain

Please list the certs and keys in the master's NSS database and in the replica's NSS database. Thanks!
Comment 7 Endi Sukma Dewata 2018-11-15 19:23:19 UTC 
Also, does the master has any logs (e.g. access log) associated with the failed operation? Does it show the reason for the SSL connection failure?
Comment 9 Endi Sukma Dewata 2018-11-15 21:46:48 UTC 
Per cfu's request, could you also provide the cipher settings in the server.xml of both master and replica (i.e. strictCiphers, sslVersionRangeStream, sslVersionRangeDatagram, sslRangeCiphers)? Thanks.
Comment 22 Christina Fu 2018-11-19 18:24:53 UTC 
first, fyi, the TLS_RSA_* ciphers were removed due to
https://bugzilla.redhat.com/show_bug.cgi?id=1578389
where after much investigation by both CS and NSS teams, it was determined that those ciphers are not supported by HSM in the FIPS environment (will cause BAD_RECORD_MAC error).
Also according to Bob Relyea, "We can basically say we don't support  RSA_ algorithms in FIPS mode with a HSM.
   - This may be fine because the RSA_ algorithms are deprecated in TLS 1.3 anyway (and currently not preferred in TLS 1.2)."
They are hence removed from the default list of ciphers.

As to the remedy for the cross platform issue, I'd prefer not making permanent changes to forward releases due to reason stated above. If temporary changes are acceptable, that'd be more ideal.
Comment 25 Rob Crittenden 2018-11-20 12:34:09 UTC 
Based on the ciphers in #10 the following need to be added to the end of NSSCipherSuite line in /etc/httpd/conf.d/nss.conf:

+ecdhe_rsa_aes_128_sha,+ecdhe_rsa_aes_256_sha

This enables TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA and TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA which appear to be the only overlapping ciphers.

It worked in my test. Nikhil, can you confirm?
Comment 26 Matthew Harmsen 2018-11-26 17:44:02 UTC 
As discussed in comments above, this bug is being re-assigned to the RHEL 6 ipa component for configuration changes to allow migration.

The test listed above still needs confirmation.
Comment 43 Endi Sukma Dewata 2019-03-11 23:36:23 UTC 
*** Bug 1667434 has been marked as a duplicate of this bug. ***
Note You need to log in before you can comment on or make changes to this bug.