There is a race condition in Red Hat JBoss Enterprise Application Platform that is exhibited when running the product under the security manager. The CXF codebase handles OneWay invocations by processing the request on a separate worker thread, allowing the original request to return to the client. The request is not marked as asynchronous. The web service, and any handlers associated with the invocation, have access to a WebServiceContext and, through this, a facade which references the original ServletRequest. The problem arises when, after processing the original request, the first thread cleans up by invoking recycle() on the org.apache.catalina.connector.Request.
When the security manager is enabled this code will also break the connection between the ServletRequest and its facade.
Meanwhile the CXF OneWay request continues to execute in a separate thread and will attempt to access the ServletRequest through the facade. This race will then cause the SwitchYard codebase to fail when executing the soap InboundHandler as it then attempts to extract the security credential (SOAPBindingData, ServletRequestCredentialExtractor) using the ServletRequest facade that has now lost its connection to the underlying ServletRequest.
As a result of this, users cannot invoke BPEL processes through WSDL. if they try to do so, they will see no response to their submitted request and a warning will be recorded in the server log.
A workaround is to make the end point robust. This, however, will change the behaviour as faults could now be returned to the client.
Created attachment 851596[details]
BPEL quickstart
Description of problem:
Can't invoke BPEL process using WSDL. There is no response to submitted request and a warning shows up in server log.
Steps to Reproduce:
1. unzip attached BPELdatabaseTest.jar into ~/jboss-eap-6.1/quickstarts/switchyard/bpel-service/
2. mvn clean install
3. run server in debug mode; in ~/jboss-eap-6.1/standalone/configuration/standalone.xml change:
in <console-handler name="CONSOLE">
<level name="INFO"/> -> <level name="DEBUG"/>
in <logger category="org.jboss.as.config">
<level name="INFO"/> -> <level name="DEBUG"/>
in <root-logger>
<level name="INFO"/> -> <level name="DEBUG"/>
4. deploy switchyard-quickstart-bpel-database-test.jar from ~/jboss-eap-6.1/quickstarts/switchyard/bpel-service/BPELdatabaseTest/target
5. create new project with SoapUI with initial wsdl : http://localhost:8080/soap/LoggingService?wsdl
6. submit request
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:urn="urn:switchyard-soa:bpel_simple_process:0.1.0">
<soapenv:Header/>
<soapenv:Body>
<urn:logMessage>
<!--Optional:-->
<urn:string>a</urn:string>
</urn:logMessage>
</soapenv:Body>
</soapenv:Envelope>
7. no response
8. submit request again
9. no response and there is the warning in server.log:
WARNING [org.apache.cxf.phase.PhaseInterceptorChain] (default-workqueue-1) Application {urn:switchyard-soa:bpel_simple_process:0.1.0}LoggingService#{urn:switchyard-soa:bpel_simple_process:0.1.0}logMessage has thrown exception, unwinding now: org.apache.cxf.interceptor.Fault: JBWEB000057: The request object has been recycled and is no longer associated with this facade
at org.apache.cxf.service.invoker.AbstractInvoker.createFault(AbstractInvoker.java:162) [cxf-api-2.6.8.redhat-7.jar:2.6.8.redhat-7]
at org.apache.cxf.jaxws.AbstractJAXWSMethodInvoker.createFault(AbstractJAXWSMethodInvoker.java:213) [cxf-rt-frontend-jaxws-2.6.8.redhat-7.jar:2.6.8.redhat-7]
at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:128) [cxf-api-2.6.8.redhat-7.jar:2.6.8.redhat-7]
at org.apache.cxf.jaxws.AbstractJAXWSMethodInvoker.invoke(AbstractJAXWSMethodInvoker.java:178) [cxf-rt-frontend-jaxws-2.6.8.redhat-7.jar:2.6.8.redhat-7]
at org.apache.cxf.jaxws.JAXWSMethodInvoker.invoke(JAXWSMethodInvoker.java:66) [cxf-rt-frontend-jaxws-2.6.8.redhat-7.jar:2.6.8.redhat-7]
at org.jboss.wsf.stack.cxf.JBossWSInvoker.invoke(JBossWSInvoker.java:129) [jbossws-cxf-server-4.1.4.Final-redhat-7.jar:4.1.4.Final-redhat-7]
at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:58) [cxf-api-2.6.8.redhat-7.jar:2.6.8.redhat-7]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_17]
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_17]
at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_17]
at org.apache.cxf.workqueue.SynchronousExecutor.execute(SynchronousExecutor.java:37) [cxf-api-2.6.8.redhat-7.jar:2.6.8.redhat-7]
at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:107) [cxf-api-2.6.8.redhat-7.jar:2.6.8.redhat-7]
at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:262) [cxf-api-2.6.8.redhat-7.jar:2.6.8.redhat-7]
at org.apache.cxf.phase.PhaseInterceptorChain.resume(PhaseInterceptorChain.java:231) [cxf-api-2.6.8.redhat-7.jar:2.6.8.redhat-7]
at org.apache.cxf.interceptor.OneWayProcessorInterceptor$1.run(OneWayProcessorInterceptor.java:144) [cxf-api-2.6.8.redhat-7.jar:2.6.8.redhat-7]
at org.apache.cxf.workqueue.AutomaticWorkQueueImpl$3.run(AutomaticWorkQueueImpl.java:426) [cxf-api-2.6.8.redhat-7.jar:2.6.8.redhat-7]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_17]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_17]
at org.apache.cxf.workqueue.AutomaticWorkQueueImpl$AWQThreadFactory$1.run(AutomaticWorkQueueImpl.java:351) [cxf-api-2.6.8.redhat-7.jar:2.6.8.redhat-7]
at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_17]
Caused by: java.lang.IllegalStateException: JBWEB000057: The request object has been recycled and is no longer associated with this facade
at org.apache.catalina.connector.RequestFacade.isSecure(RequestFacade.java:533)
at org.switchyard.security.credential.extractor.ServletRequestCredentialExtractor.extract(ServletRequestCredentialExtractor.java:47) [switchyard-security-1.1.1-p4-redhat-1.jar:1.1.1-p4-redhat-1]
at org.switchyard.component.soap.composer.SOAPBindingData.extractCredentials(SOAPBindingData.java:119) [switchyard-component-soap-1.1.1-p4-redhat-1.jar:1.1.1-p4-redhat-1]
at org.switchyard.component.soap.InboundHandler.invoke(InboundHandler.java:296) [switchyard-component-soap-1.1.1-p4-redhat-1.jar:1.1.1-p4-redhat-1]
at org.switchyard.component.soap.endpoint.BaseWebService.invoke(BaseWebService.java:112) [switchyard-component-soap-1.1.1-p4-redhat-1.jar:1.1.1-p4-redhat-1]
at org.switchyard.component.soap.endpoint.BaseWebService.invoke(BaseWebService.java:41) [switchyard-component-soap-1.1.1-p4-redhat-1.jar:1.1.1-p4-redhat-1]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_17]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_17]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_17]
at java.lang.reflect.Method.invoke(Method.java:601) [rt.jar:1.7.0_17]
at org.jboss.ws.common.invocation.AbstractInvocationHandlerJSE.invoke(AbstractInvocationHandlerJSE.java:108)
at org.jboss.wsf.stack.cxf.JBossWSInvoker.performInvocation(JBossWSInvoker.java:149) [jbossws-cxf-server-4.1.4.Final-redhat-7.jar:4.1.4.Final-redhat-7]
at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:96) [cxf-api-2.6.8.redhat-7.jar:2.6.8.redhat-7]
... 17 more
Expected results:
Invoked BPEL process, response to request, no warning
This is a race condition in EAP, exhibited when running under the security manager.
The CXF codebase handles OneWay invocations by processing the request on a separate worker thread, allowing the original request to return to the client. The request is not marked as async.
The webservice, and any handlers associated with the invocation, have access to a WebServiceContext and, through this, a facade which references the original ServletRequest.
The problem arises when, after processing the original request, the first thread cleans up by invoking recycle() on the org.apache.catalina.connector.Request. When the security manager is enabled this code will also break the connection between the ServletRequest and its facade. Meanwhile the CXF OneWay request continues to execute in a separate thread and will attempt to access the ServletRequest through the facade.
(http://grepcode.com/file/repository.jboss.org/nexus/content/repositories/releases/org.jboss.web/jbossweb/7.2.2.Final/org/apache/catalina/connector/Request.java#572)
This race will then cause the SwitchYard codebase to fail when executing the soap InboundHandler as it then attempts to extract the security credential (SOAPBindingData, ServletRequestCredentialExtractor) using the ServletRequest facade that has now lost its connection to the underlying ServletRequest.
A possible workaround would be to make the endpoint robust however this will change the behaviour as faults could now be returned to the client.
(http://grepcode.com/file/repo1.maven.org/maven2/org.apache.cxf/cxf-api/2.6.8/org/apache/cxf/interceptor/OneWayProcessorInterceptor.java#OneWayProcessorInterceptor.handleMessage%28org.apache.cxf.message.Message%29)