Bug 1384856 - [GSS](6.4.z) NullPointerException at org.apache.cxf.common.jaxb.JAXBUtils.createJAXBContextProxy
Summary: [GSS](6.4.z) NullPointerException at org.apache.cxf.common.jaxb.JAXBUtils.cre...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: JBoss Enterprise Application Platform 6
Classification: JBoss
Component: Web Services
Version: 6.4.10,6.4.3
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: CR1
: EAP 6.4.14
Assignee: Fedor Gavrilov
QA Contact: Peter Mackay
URL:
Whiteboard:
Depends On:
Blocks: eap6414-payload 1403751
TreeView+ depends on / blocked
 
Reported: 2016-10-14 09:36 UTC by Masafumi Miura
Modified: 2019-12-16 07:06 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-03-23 08:21:42 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
cxf-npe-reproducer.tar.gz (46.23 KB, application/x-gzip)
2016-10-14 09:37 UTC, Masafumi Miura
no flags Details
fix suggestion (1.40 KB, patch)
2016-10-24 13:11 UTC, Fedor Gavrilov
no flags Details | Diff


Links
System ID Private Priority Status Summary Last Updated
Apache JIRA CXF-7166 0 None None None 2016-12-06 13:54:12 UTC
Red Hat Issue Tracker JBEAP-6432 0 Major Verified [GSS](7.0.z) NullPointerException at org.apache.cxf.common.jaxb.JAXBUtils.createJAXBContextProxy 2018-10-19 01:09:08 UTC
Red Hat Issue Tracker JBEAP-6433 0 Major Verified [GSS](7.1.0) NullPointerException at org.apache.cxf.common.jaxb.JAXBUtils.createJAXBContextProxy 2018-10-19 01:09:08 UTC
Red Hat Knowledge Base (Solution) 2773951 0 None None None 2016-11-21 10:17:36 UTC

Description Masafumi Miura 2016-10-14 09:36:21 UTC
### Description of problem:

The following NullPointerException rarely happens in Web Service application.

This phenomenon occurs when Full GC happens before calling CachedContextAndSchemasInternal#getContext() in a constructor of org.apache.cxf.common.jaxb.JAXBContextCache$CachedContextAndSchemas. Because "WeakReference<JAXBContext> context" inside org.apache.cxf.common.jaxb.JAXBContextCache$CachedContextAndSchemasInternal is cleared at the Full GC and CachedContextAndSchemasInternal#getContext()  returns null.

As far as I tested, this issue can not be reproducible with OpenJDK/Oracle JDK but it was reproducible with IBM JDK. 

~~~
java.lang.NullPointerException
	at org.apache.cxf.common.jaxb.JAXBUtils.createJAXBContextProxy(JAXBUtils.java:1425)
	at org.apache.cxf.jaxb.JAXBDataBinding.checkForJAXBAnnotations(JAXBDataBinding.java:433)
	at org.apache.cxf.jaxb.JAXBDataBinding.justCheckForJAXBAnnotations(JAXBDataBinding.java:427)
	at org.apache.cxf.jaxb.JAXBDataBinding.initialize(JAXBDataBinding.java:363)
	at org.apache.cxf.service.factory.AbstractServiceFactoryBean.initializeDataBindings(AbstractServiceFactoryBean.java:86)
	at org.apache.cxf.service.factory.ReflectionServiceFactoryBean.buildServiceFromWSDL(ReflectionServiceFactoryBean.java:446)
	at org.apache.cxf.service.factory.ReflectionServiceFactoryBean.initializeServiceModel(ReflectionServiceFactoryBean.java:548)
	at org.apache.cxf.service.factory.ReflectionServiceFactoryBean.create(ReflectionServiceFactoryBean.java:265)
	at org.apache.cxf.jaxws.support.JaxWsServiceFactoryBean.create(JaxWsServiceFactoryBean.java:215)
	at org.apache.cxf.frontend.AbstractWSDLBasedEndpointFactory.createEndpoint(AbstractWSDLBasedEndpointFactory.java:102)
	at org.apache.cxf.frontend.ClientFactoryBean.create(ClientFactoryBean.java:91)
	at org.apache.cxf.frontend.ClientProxyFactoryBean.create(ClientProxyFactoryBean.java:157)
	at org.apache.cxf.jaxws.JaxWsProxyFactoryBean.create(JaxWsProxyFactoryBean.java:142)
	at org.apache.cxf.jaxws.ServiceImpl.createPort(ServiceImpl.java:478)
	at org.jboss.wsf.stack.cxf.client.ProviderImpl$JBossWSServiceImpl.createPort(ProviderImpl.java:558)
	at org.apache.cxf.jaxws.ServiceImpl.getPort(ServiceImpl.java:329)
	at org.apache.cxf.jaxws.ServiceImpl.getPort(ServiceImpl.java:324)
	at javax.xml.ws.Service.getPort(Service.java:168) [jboss-jaxws-api_2.2_spec-2.0.2.Final-redhat-1.jar:2.0.2.Final-redhat-1]
	at org.jboss.as.quickstarts.wshelloworld.Client.<init>(Client.java:44) [classes:]
	at org.jboss.as.quickstarts.wshelloworld.WSClientServlet.doGet(WSClientServlet.java:97) [classes:]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:734) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-2.jar:1.0.2.Final-redhat-2]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-2.jar:1.0.2.Final-redhat-2]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:295) [jbossweb-7.5.12.Final-redhat-1.jar:7.5.12.Final-redhat-1]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214) [jbossweb-7.5.12.Final-redhat-1.jar:7.5.12.Final-redhat-1]
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:231) [jbossweb-7.5.12.Final-redhat-1.jar:7.5.12.Final-redhat-1]
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) [jbossweb-7.5.12.Final-redhat-1.jar:7.5.12.Final-redhat-1]
	at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) [jboss-as-web-7.5.6.Final-redhat-2.jar:7.5.6.Final-redhat-2]
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:150) [jbossweb-7.5.12.Final-redhat-1.jar:7.5.12.Final-redhat-1]
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) [jbossweb-7.5.12.Final-redhat-1.jar:7.5.12.Final-redhat-1]
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) [jbossweb-7.5.12.Final-redhat-1.jar:7.5.12.Final-redhat-1]
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344) [jbossweb-7.5.12.Final-redhat-1.jar:7.5.12.Final-redhat-1]
	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:854) [jbossweb-7.5.12.Final-redhat-1.jar:7.5.12.Final-redhat-1]
	at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) [jbossweb-7.5.12.Final-redhat-1.jar:7.5.12.Final-redhat-1]
	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:926) [jbossweb-7.5.12.Final-redhat-1.jar:7.5.12.Final-redhat-1]
	at java.lang.Thread.run(Thread.java:785) [vm.jar:1.8.0]
~~~


### Version-Release number of selected component (if applicable):

CXF 2.7.18, CXF 2.7.18.SP1 
The above stack trace is from EAP 6.4.6 (CXF 2.7.18).
Same issue is reproducible with the latest EAP 6.4.10 (CXF 2.7.18.SP1).


### How reproducible:

You can reproduce this issue anytime when you use byteman to invoke Full GC (System.gc()) before calling CachedContextAndSchemasInternal#getContext() in a constructor of org.apache.cxf.common.jaxb.JAXBContextCache$CachedContextAndSchemas.


### Steps to Reproduce:

0.  Extract the attached reproducer cxf-npe-reproducer.tar.gz

1.  Build and deploy two applications

    - helloworld-ws : Hello World JAX-WS Web service Application (based on JBoss EAP QuickStarts)
    - servlet-wsclient : Web Services Client on Servlet 


2.  Place the following byteman rule as /tmp/byteman-script.btm
    You can use byteman-script.btm in the cxf-npe-reproducer.tar.gz

~~~
RULE org.apache.cxf.common.jaxb.JAXBContextCache$CachedContextAndSchemas_init_ENTRY
CLASS org.apache.cxf.common.jaxb.JAXBContextCache$CachedContextAndSchemas
METHOD <init>
AT ENTRY
IF TRUE
DO
    System.gc();
ENDRULE
~~~


3.  Add the following to $JBOSS_HOME/bin/standalone.conf
    Note: Make sure to tune BYTEMAN_HOME to a byteman install location

~~~
BYTEMAN_RULE=/tmp/byteman-script.btm
BYTEMAN_HOME=/path/to/your/byteman
BYTEMAN_OPTS="-javaagent:$BYTEMAN_HOME/lib/byteman.jar=listener:false,boot:$BYTEMAN_HOME/lib/byteman.jar"
if [ "x$BYTEMAN_RULE" != "x" ]; then
   BYTEMAN_OPTS="${BYTEMAN_OPTS},script:$BYTEMAN_RULE"
fi
BYTEMAN_OPTS="$BYTEMAN_OPTS -Dorg.jboss.byteman.transform.all -Dorg.jboss.byteman.debug"
# BYTEMAN_OPTS="$BYTEMAN_OPTS -Djboss.modules.system.pkgs=org.jboss.byteman"
JAVA_OPTS="$BYTEMAN_OPTS $JAVA_OPTS"
~~~


4.  Set IBM JDK 7 or IBM JDK 8 as JAVA_HOME. For example:

    export JAVA_HOME=/usr/lib/jvm/java-1.7.0-ibm.x86_64"

    or  

    export JAVA_HOME=/usr/lib/jvm/java-1.8.0-ibm.x86_64"


5.  Start JBoss EAP 6 

    ./bin/standalone.sh


6.  Access Web Services Client Servlet 

    curl -v http://localhost:8080/jboss-servlet-wsclient/client



### Actual results:

Application fails with NPE 


### Expected results:

Application succeeds without NPE 


### Additional info:

A similar issue was reported in https://issues.apache.org/jira/browse/CXF-4947 but this does not look a perfect fix.

Comment 1 Masafumi Miura 2016-10-14 09:37:04 UTC
Created attachment 1210446 [details]
cxf-npe-reproducer.tar.gz

Comment 3 Masafumi Miura 2016-10-14 09:45:02 UTC
Sorry, I forgot to mention the version of JBoss which a customer originally reported. 

This issue was originally reported by the customer using JBoss EAP 6.4.3 in the support case 01704042.

Comment 4 Masafumi Miura 2016-10-18 07:51:02 UTC
I got an update from the customer. They said this NPE occurred on OpenJDK in their environment although I have not yet reproduced this on OpenJDK in my environment.

Anyway I think the attached reproducer can be used to reproduce this issue on IBM JDK reliably. Please utilize it to investigate a root cause and how to fix it.

FYI: Here's the version information they are using:

~~~
# rpm -qa | grep java
java-1.7.0-openjdk-devel-1.7.0.95-2.6.4.0.el6_7.x86_64
tzdata-java-2014g-1.el6.noarch
java-1.7.0-openjdk-1.7.0.95-2.6.4.0.el6_7.x86_64
libvirt-java-0.4.9-1.el6.noarch
libvirt-java-devel-0.4.9-1.el6.noarch
java-1.7.0-openjdk-debuginfo-1.7.0.95-2.6.4.0.el6_7.x86_64
~~~

~~~
# java -version
java version "1.7.0_95"
OpenJDK Runtime Environment (rhel-2.6.4.0.el6_7-x86_64 u95-b00)
OpenJDK 64-Bit Server VM (build 24.95-b01, mixed mode)
~~~

Comment 6 Fedor Gavrilov 2016-10-24 13:11:20 UTC
Created attachment 1213467 [details]
fix suggestion

Comment 12 Peter Mackay 2017-02-28 11:25:15 UTC
Verified with EAP 6.4.14.CP.CR1 using the provided reproducer.

Comment 13 Petr Penicka 2017-03-23 08:21:42 UTC
Released with EAP 6.4.14 on March 14 (ZIPs) and March 22 (RPMs).


Note You need to log in before you can comment on or make changes to this bug.