1031413 – Call time widget fails with 'null'

Bug 1031413 - Call time widget fails with 'null'

Summary: Call time widget fails with 'null'

Keywords:
Status:	CLOSED CURRENTRELEASE
Alias:	None
Product:	JBoss Fuse Service Works 6
Classification:	JBoss
Component:	RT Governance
Sub Component:
Version:	6.0.0 GA
Hardware:	Unspecified
OS:	Unspecified
Priority:	unspecified
Severity:	high
Target Milestone:	ER7
Target Release:	6.0.0
Assignee:	Thomas Hauser
QA Contact:	Jiri Pechanec
Docs Contact:
URL:
Whiteboard:
Duplicates (1):	1032008 (view as bug list)
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2013-11-17 17:55 UTC by Alan Santos
Modified:	2015-02-06 01:19 UTC (History)
CC List:	6 users (show)
Fixed In Version:
Doc Type:	Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed:
Type:	Bug
Embargoed:

Attachments	(Terms of Use)
sample h2 database which can be used to reproduce the exception (663.94 KB, application/x-bzip2) 2013-11-18 14:51 UTC, Alan Santos	no flags	Details
View All

Description Alan Santos 2013-11-17 17:55:52 UTC

Running the RTGov examples over a period of time has broken the call trace service and/or widget.  

using 1 or 3 as the value for an orderID results in 'null' being displayed in the widget and the following stack trace in the server log


18:48:07,635 INFO  [org.jboss.resteasy.spi.ResteasyDeployment] (http-/127.0.0.1:8080-13) Adding singleton resource org.overlord.rtgov.internal.service.dependency.rest.RESTServiceDependencyServer from Application javax.ws.rs.core.Application
18:48:13,085 SEVERE [org.jboss.resteasy.core.SynchronousDispatcher] (http-/127.0.0.1:8080-9) Failed executing GET /call/trace/instance: org.jboss.resteasy.spi.WriterException: ClientAbortException:  java.net.SocketException: Broken pipe
	at org.jboss.resteasy.core.ServerResponse.writeTo(ServerResponse.java:262) [resteasy-jaxrs-2.3.6.Final-redhat-1.jar:2.3.6.Final-redhat-1]
	at org.jboss.resteasy.core.SynchronousDispatcher.writeJaxrsResponse(SynchronousDispatcher.java:602) [resteasy-jaxrs-2.3.6.Final-redhat-1.jar:2.3.6.Final-redhat-1]
	at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:528) [resteasy-jaxrs-2.3.6.Final-redhat-1.jar:2.3.6.Final-redhat-1]
	at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:126) [resteasy-jaxrs-2.3.6.Final-redhat-1.jar:2.3.6.Final-redhat-1]
	at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:208) [resteasy-jaxrs-2.3.6.Final-redhat-1.jar:2.3.6.Final-redhat-1]
	at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:55) [resteasy-jaxrs-2.3.6.Final-redhat-1.jar:2.3.6.Final-redhat-1]
	at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:50) [resteasy-jaxrs-2.3.6.Final-redhat-1.jar:2.3.6.Final-redhat-1]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:295) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:481) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.0.Final-redhat-8.jar:7.2.0.Final-redhat-8]
	at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.0.Final-redhat-8.jar:7.2.0.Final-redhat-8]
	at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) [jboss-as-web-7.2.0.Final-redhat-8.jar:7.2.0.Final-redhat-8]
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_45]
Caused by: ClientAbortException:  java.net.SocketException: Broken pipe
	at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:403) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:356) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.apache.catalina.connector.OutputBuffer.writeBytes(OutputBuffer.java:426) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:415) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:89) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:83) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.jboss.resteasy.plugins.server.servlet.HttpServletResponseWrapper$DeferredOutputStream.write(HttpServletResponseWrapper.java:40) [resteasy-jaxrs-2.3.6.Final-redhat-1.jar:2.3.6.Final-redhat-1]
	at org.jboss.resteasy.util.CommitHeaderOutputStream.write(CommitHeaderOutputStream.java:45) [resteasy-jaxrs-2.3.6.Final-redhat-1.jar:2.3.6.Final-redhat-1]
	at org.jboss.resteasy.plugins.providers.StringTextStar.writeTo(StringTextStar.java:60) [resteasy-jaxrs-2.3.6.Final-redhat-1.jar:2.3.6.Final-redhat-1]
	at org.jboss.resteasy.plugins.providers.StringTextStar.writeTo(StringTextStar.java:20) [resteasy-jaxrs-2.3.6.Final-redhat-1.jar:2.3.6.Final-redhat-1]
	at org.jboss.resteasy.core.interception.MessageBodyWriterContextImpl.proceed(MessageBodyWriterContextImpl.java:117) [resteasy-jaxrs-2.3.6.Final-redhat-1.jar:2.3.6.Final-redhat-1]
	at org.jboss.resteasy.plugins.providers.jackson.JacksonJsonpInterceptor.write(JacksonJsonpInterceptor.java:112) [resteasy-jackson-provider-2.3.6.Final-redhat-1.jar:2.3.6.Final-redhat-1]
	at org.jboss.resteasy.core.interception.MessageBodyWriterContextImpl.proceed(MessageBodyWriterContextImpl.java:123) [resteasy-jaxrs-2.3.6.Final-redhat-1.jar:2.3.6.Final-redhat-1]
	at org.jboss.resteasy.plugins.interceptors.encoding.GZIPEncodingInterceptor.write(GZIPEncodingInterceptor.java:104) [resteasy-jaxrs-2.3.6.Final-redhat-1.jar:2.3.6.Final-redhat-1]
	at org.jboss.resteasy.core.interception.MessageBodyWriterContextImpl.proceed(MessageBodyWriterContextImpl.java:123) [resteasy-jaxrs-2.3.6.Final-redhat-1.jar:2.3.6.Final-redhat-1]
	at org.jboss.resteasy.core.ServerResponse.writeTo(ServerResponse.java:250) [resteasy-jaxrs-2.3.6.Final-redhat-1.jar:2.3.6.Final-redhat-1]
	... 23 more
Caused by: java.net.SocketException: Broken pipe
	at java.net.SocketOutputStream.socketWrite0(Native Method) [rt.jar:1.7.0_45]
	at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:113) [rt.jar:1.7.0_45]
	at java.net.SocketOutputStream.write(SocketOutputStream.java:159) [rt.jar:1.7.0_45]
	at org.apache.coyote.http11.InternalOutputBuffer.realWriteBytes(InternalOutputBuffer.java:711) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:450) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:351) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.apache.coyote.http11.InternalOutputBuffer$OutputStreamOutputBuffer.doWrite(InternalOutputBuffer.java:735) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.apache.coyote.http11.filters.ChunkedOutputFilter.doWrite(ChunkedOutputFilter.java:126) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.apache.coyote.http11.InternalOutputBuffer.doWrite(InternalOutputBuffer.java:548) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.apache.coyote.Response.doWrite(Response.java:594) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:398) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	... 38 more

18:48:13,091 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/overlord-rtgov].[CallTrace]] (http-/127.0.0.1:8080-9) JBWEB000236: Servlet.service() for servlet CallTrace threw exception: java.lang.IllegalStateException
	at org.apache.catalina.connector.ResponseFacade.sendError(ResponseFacade.java:392) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.jboss.resteasy.plugins.server.servlet.HttpServletResponseWrapper.sendError(HttpServletResponseWrapper.java:109) [resteasy-jaxrs-2.3.6.Final-redhat-1.jar:2.3.6.Final-redhat-1]
	at org.jboss.resteasy.core.SynchronousDispatcher.handleFailure(SynchronousDispatcher.java:284) [resteasy-jaxrs-2.3.6.Final-redhat-1.jar:2.3.6.Final-redhat-1]
	at org.jboss.resteasy.core.SynchronousDispatcher.handleWriterException(SynchronousDispatcher.java:418) [resteasy-jaxrs-2.3.6.Final-redhat-1.jar:2.3.6.Final-redhat-1]
	at org.jboss.resteasy.core.SynchronousDispatcher.handleException(SynchronousDispatcher.java:238) [resteasy-jaxrs-2.3.6.Final-redhat-1.jar:2.3.6.Final-redhat-1]
	at org.jboss.resteasy.core.SynchronousDispatcher.handleWriteResponseException(SynchronousDispatcher.java:222) [resteasy-jaxrs-2.3.6.Final-redhat-1.jar:2.3.6.Final-redhat-1]
	at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:532) [resteasy-jaxrs-2.3.6.Final-redhat-1.jar:2.3.6.Final-redhat-1]
	at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:126) [resteasy-jaxrs-2.3.6.Final-redhat-1.jar:2.3.6.Final-redhat-1]
	at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:208) [resteasy-jaxrs-2.3.6.Final-redhat-1.jar:2.3.6.Final-redhat-1]
	at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:55) [resteasy-jaxrs-2.3.6.Final-redhat-1.jar:2.3.6.Final-redhat-1]
	at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:50) [resteasy-jaxrs-2.3.6.Final-redhat-1.jar:2.3.6.Final-redhat-1]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:295) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:481) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.0.Final-redhat-8.jar:7.2.0.Final-redhat-8]
	at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.0.Final-redhat-8.jar:7.2.0.Final-redhat-8]
	at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) [jboss-as-web-7.2.0.Final-redhat-8.jar:7.2.0.Final-redhat-8]
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_45]

Comment 2 Gary Brown 2013-11-18 08:34:48 UTC

Hi Alan

Can you describe the steps to reproduce these problems from a clean start (i.e. install).

Regards
Gary

Comment 3 Alan Santos 2013-11-18 14:51:43 UTC

Created attachment 825699 [details]
sample h2 database which can be used to reproduce the exception

Comment 4 Gary Brown 2013-11-19 11:33:09 UTC

This issue is being caused by a large json response (177kb). The service has the response incorrectly marked as producing "image/svg+xml" - wondered if this was the problem, but the exceptions occur even if changed to "application/json".

Will assign to Eric to see whether this could be a gadget server issue, as the exception stack indicates the client is closing the connection.

Comment 5 Eric Wittmann 2013-11-19 18:16:23 UTC

Shindig (the gadget api framework we use) had a default http connection timeout for REST calls of 5s and an http connection read timeout of 5s.  So this was presumably hitting the 5s mark and then failing.

I have fixed this by making these two timeouts configurable via gadget-server.properties.  The properties that drive this behavior are:

gadget-server.config.auth.connection-timeout=10000
gadget-server.config.auth.read-timeout=60000

These properties are now required to be in gadget-server.properties - there are no defaults.  The values are in milliseconds, so the above uses a 10s connection timeout and a 60s read timeout.  These seem like reasonable values.

I'm going to assign this to thauser in case there are installer implications.

Comment 6 Thomas Hauser 2013-11-19 18:26:32 UTC

As long as these changes are present in the gadget-server.properties file at build time, there should be no installer changes required for this issue.

Comment 7 Gary Brown 2013-11-20 13:12:20 UTC

*** Bug 1032008 has been marked as a duplicate of this bug. ***

Comment 8 Jiri Pechanec 2013-12-16 11:35:21 UTC

Verified in ER7 that increasing timeout mitigates the issue

Note You need to log in before you can comment on or make changes to this bug.