Bug 1014908

Summary:	IllegalStateException: JBAS018058: Session is already expired at server shutdown
Product:	[JBoss] JBoss Enterprise Application Platform 6	Reporter:	Jitka Kozana <jkudrnac>
Component:	Clustering	Assignee:	Paul Ferraro <paul.ferraro>
Status:	CLOSED NOTABUG	QA Contact:	Jitka Kozana <jkudrnac>
Severity:	medium	Docs Contact:	Russell Dickenson <rdickens>
Priority:	unspecified
Version:	6.2.0	CC:	jkudrnac, lthon, rhusar, rjanik
Target Milestone:	---
Target Release:	EAP 6.2.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2013-12-12 16:24:48 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:

Description Jitka Kozana 2013-10-03 05:42:00 UTC

EAP 6.2.0 ER3.
At the end of a test, when nodes were shutting down and leaving the cluster, one node started to log this exception extensively (hundreds of log entries):

19:54:13,450 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/clusterbench].[HttpSessionServlet]] (ajp-perf20/10.16.90.58:8009-454) JBWEB000236: Servlet.service() for servlet HttpSessionServlet threw exception: java.lang.IllegalStateException: JBAS018058: Session is already expired
	at org.jboss.as.web.session.ClusteredSession.isNew(ClusteredSession.java:582) [jboss-as-web-7.3.0.Final-redhat-6.jar:7.3.0.Final-redhat-6]
	at org.apache.catalina.session.StandardSessionFacade.isNew(StandardSessionFacade.java:155) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1]
	at org.jboss.test.clusterbench.common.session.CommonHttpSessionServlet.doGet(CommonHttpSessionServlet.java:23) [clusterbench-common.jar:]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:734) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1]
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1]
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1]
	at org.jboss.modcluster.container.jbossweb.JBossWebContext$RequestListenerValve.event(JBossWebContext.java:91)
	at org.jboss.modcluster.container.jbossweb.JBossWebContext$RequestListenerValve.invoke(JBossWebContext.java:72)
	at org.jboss.as.web.session.ClusteredSessionValve.handleRequest(ClusteredSessionValve.java:134) [jboss-as-web-7.3.0.Final-redhat-6.jar:7.3.0.Final-redhat-6]
	at org.jboss.as.web.session.ClusteredSessionValve.invoke(ClusteredSessionValve.java:99) [jboss-as-web-7.3.0.Final-redhat-6.jar:7.3.0.Final-redhat-6]
	at org.jboss.as.web.session.JvmRouteValve.invoke(JvmRouteValve.java:92) [jboss-as-web-7.3.0.Final-redhat-6.jar:7.3.0.Final-redhat-6]
	at org.jboss.as.web.session.LockingValve.invoke(LockingValve.java:64) [jboss-as-web-7.3.0.Final-redhat-6.jar:7.3.0.Final-redhat-6]
	at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) [jboss-as-web-7.3.0.Final-redhat-6.jar:7.3.0.Final-redhat-6]
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1]
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1]
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1]
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1]
	at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:488) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1]
	at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:420) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1]
	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1]
	at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45]

Cache setup: dist async

Link to server log:
https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-failover-http-session-netDown-dist-async/32/artifact/report/config/jboss-perf20/server.log

Comment 1 Paul Ferraro 2013-11-05 21:04:16 UTC

I don't understand. The bug description says that this occurred at the end of the test when the nodes were shutting down, yet the stacktrace indicates an exception originating from a request.

Comment 2 Jitka Kozana 2013-11-20 14:24:20 UTC

The clients are accessing the cluster, as long as there is a node in the cluster that is still alive. 

So clients can access server, that is currently in the middle of shutdown. This results in this exception.

Comment 3 Paul Ferraro 2013-12-05 00:28:43 UTC

What is the configured session timeout for this test?

Comment 4 Jitka Kozana 2013-12-05 09:39:52 UTC

5minutes.

Comment 5 Paul Ferraro 2013-12-12 16:24:48 UTC

This is acceptable/expected behavior given the test scenario.

Comment 6 Radoslav Husar 2013-12-16 23:30:57 UTC

Graceful shutdown is not support in the affected version of EAP 6. The 
described behavior is to be expected.

The RFE to implement graceful shutdown is 
https://issues.jboss.org/browse/EAP6-7

Issues relating to HTTP can be avoided using mod_cluster and session 
draining prior to shutdown/undeploy.