Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 961257

Summary: No available certificate or key corresponds to the SSL cipher suites which are enabled when 2 SSL connector configured
Product: [JBoss] JBoss Enterprise Application Platform 6 Reporter: Hisanobu Okuda <hokuda>
Component: WebAssignee: Rémy Maucherat <rmaucher>
Status: CLOSED CURRENTRELEASE QA Contact: Jan Stefl <jstefl>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 6.0.1CC: aogburn, jbliznak, jclere, kkhan, pslavice, rlucente
Target Milestone: DR1   
Target Release: EAP 6.4.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1138672    
Bug Blocks: 981516, 1022777    
Attachments:
Description Flags
test fix none

Description Hisanobu Okuda 2013-05-09 09:14:47 UTC 
Description of problem:

We are configuring 2 SSL connectors in JBoss 6. When JBoss starts up, we sometimes get an error initializing one of the SSL web connectors. this occurs like 1 out of 20 times on average and appears randomly.

The error message:
10:20:28,755 ERROR [org.apache.coyote.http11.Http11Protocol] (MSC service thread 1-1) Error initializing endpoint: java.io.IOException: SSL configuration is invalid due to No available certificate or key corresponds to the SSL cipher suites which are enabled.
	at org.apache.tomcat.util.net.jsse.JSSESocketFactory.checkConfig(JSSESocketFactory.java:788) [jbossweb-7.0.17.Final-redhat-1.jar:]
	at org.apache.tomcat.util.net.jsse.JSSESocketFactory.init(JSSESocketFactory.java:493) [jbossweb-7.0.17.Final-redhat-1.jar:]
	at org.apache.tomcat.util.net.jsse.JSSESocketFactory.createSocket(JSSESocketFactory.java:168) [jbossweb-7.0.17.Final-redhat-1.jar:]
	at org.apache.tomcat.util.net.JIoEndpoint.init(JIoEndpoint.java:978) [jbossweb-7.0.17.Final-redhat-1.jar:]
	at org.apache.coyote.http11.Http11Protocol.init(Http11Protocol.java:190) [jbossweb-7.0.17.Final-redhat-1.jar:]
	at org.apache.catalina.connector.Connector.init(Connector.java:1001) [jbossweb-7.0.17.Final-redhat-1.jar:]
	at org.jboss.as.web.WebConnectorService.start(WebConnectorService.java:268) [jboss-as-web-7.1.3.Final-redhat-4.jar:7.1.3.Final-redhat-4]
	at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) [jboss-msc-1.0.2.GA-redhat-2.jar:1.0.2.GA-redhat-2]
	at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) [jboss-msc-1.0.2.GA-redhat-2.jar:1.0.2.GA-redhat-2]
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_38]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_38]
	at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_38]
Caused by: javax.net.ssl.SSLException: No available certificate or key corresponds to the SSL cipher suites which are enabled.
	at com.sun.net.ssl.internal.ssl.SSLServerSocketImpl.checkEnabledSuites(SSLServerSocketImpl.java:310) [jsse.jar:1.6]
	at com.sun.net.ssl.internal.ssl.SSLServerSocketImpl.accept(SSLServerSocketImpl.java:255) [jsse.jar:1.6]
	at org.apache.tomcat.util.net.jsse.JSSESocketFactory.checkConfig(JSSESocketFactory.java:784) [jbossweb-7.0.17.Final-redhat-1.jar:]
	... 11 more


Version-Release number of selected component (if applicable):


How reproducible:

1. Configure 2 connection as follow:-

<connector name="https" protocol="HTTP/1.1" scheme="https" socket-binding="https" secure="true" max-connections="100">
<ssl name="ssl" password="password" cipher-suite="TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_RSA_WITH_3DES_EDE_CBC_SHA, TLS_RSA_WITH_RC4_128_MD5, TLS_RSA_WITH_RC4_128_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_RC4_128_MD5, SSL_RSA_WITH_RC4_128_SHA"  verify-client="true" keystore-type="PKCS11"/>
</connector> 
<connector name="common-login"  protocol="HTTP/1.1" scheme="https" socket-binding="common-login" secure="true">
<ssl name="common-login" password="password" verify-client="false" keystore-type="PKCS11"/>
</connector>

2. Start EAP6 20+ times

Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:
Comment 3 Aaron Ogburn 2013-07-02 21:20:44 UTC 
I implemented a fix in a test build on EAP 6.0.1/JBossWeb 7.0.17.GA.  This pretty much just allows for synchronized connector initialization through a system property.  If that property is set to true, then Http11Protocol.init operations will be done synchronized on a static object.  Testing shows the issue is successfully avoided altogether here with multiple https connector set up with that fix applied.  I've attached the modified Http11Protocol class for a reference.  What would you guys think about implementing this or something similar to it in future releases as an option for protection against this issue in the future?
Comment 4 Aaron Ogburn 2013-07-02 21:21:41 UTC 
Created attachment 767982 [details]
test fix
Comment 7 Martin Velas 2014-07-29 15:34:14 UTC 
For EAP 6.3.0.ER10 configured as described above, the following error still occurs randomly (approx. for 1 startup of 10):

17:20:39,281 ERROR [org.apache.coyote.http11.Http11Protocol] (MSC service thread 1-6) JBWEB003043: Error initializing endpoint: java.io.IOException: java.security.KeyStoreException: invalid KeyStore state: found 2 private keys sharing CKA_ID 0x9d1b9651c5675fbcee9b91a10c06e6118379fbc1
	at org.apache.tomcat.util.net.jsse.JSSESocketFactory.init(JSSESocketFactory.java:440) [jbossweb-7.4.8.Final-redhat-4.jar:7.4.8.Final-redhat-4]
	at org.apache.tomcat.util.net.jsse.JSSESocketFactory.createSocket(JSSESocketFactory.java:162) [jbossweb-7.4.8.Final-redhat-4.jar:7.4.8.Final-redhat-4]
	at org.apache.tomcat.util.net.JIoEndpoint.init(JIoEndpoint.java:973) [jbossweb-7.4.8.Final-redhat-4.jar:7.4.8.Final-redhat-4]
	at org.apache.coyote.http11.Http11Protocol.init(Http11Protocol.java:174) [jbossweb-7.4.8.Final-redhat-4.jar:7.4.8.Final-redhat-4]
	at org.apache.catalina.connector.Connector.init(Connector.java:985) [jbossweb-7.4.8.Final-redhat-4.jar:7.4.8.Final-redhat-4]
	at org.jboss.as.web.WebConnectorService.start(WebConnectorService.java:318) [jboss-as-web-7.4.0.Final-redhat-19.jar:7.4.0.Final-redhat-19]
	at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1980) [jboss-msc-1.1.5.Final-redhat-1.jar:1.1.5.Final-redhat-1]
	at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1913) [jboss-msc-1.1.5.Final-redhat-1.jar:1.1.5.Final-redhat-1]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_55]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_55]
	at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_55]

17:20:39,298 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-6) MSC000001: Failed to start service jboss.web.connector.https: org.jboss.msc.service.StartException in service jboss.web.connector.https: JBAS018007: Error starting web connector
	at org.jboss.as.web.WebConnectorService.start(WebConnectorService.java:376)
	at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1980) [jboss-msc-1.1.5.Final-redhat-1.jar:1.1.5.Final-redhat-1]
	at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1913) [jboss-msc-1.1.5.Final-redhat-1.jar:1.1.5.Final-redhat-1]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_55]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_55]
	at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_55]
Caused by: LifecycleException:  JBWEB000023: Protocol handler initialization failed
	at org.apache.catalina.connector.Connector.init(Connector.java:987)
	at org.jboss.as.web.WebConnectorService.start(WebConnectorService.java:318)
	... 5 more

In my opinion (also confirmed by Radim), this error message is related to the same problem as described above.
Comment 8 Rémy Maucherat 2014-08-26 14:31:52 UTC 
After reviewing, I think it is more generic to sync the connector init. r2496 in web.
Comment 9 Kabir Khan 2014-09-08 11:53:37 UTC 
Fixed by component upgrade to JBoss Web 7.5.0.Beta1: https://bugzilla.redhat.com/show_bug.cgi?id=1138672
Comment 10 Martin Velas 2014-09-30 13:23:10 UTC 
Verified that this issue is fixed in EAP 6.4.0.DR3.