Bug 849924 - Tomcat6 with mod_cluster refuses to shut down
Tomcat6 with mod_cluster refuses to shut down
Status: CLOSED CURRENTRELEASE
Product: JBoss Enterprise Web Server 2
Classification: JBoss
Component: mod_cluster (Show other bugs)
2.0.0
Unspecified Unspecified
urgent Severity urgent
: ---
: 2.0.1
Assigned To: Jean-frederic Clere
Michal Karm Babacek
: Reopened
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-08-21 05:00 EDT by Michal Karm Babacek
Modified: 2014-01-03 07:58 EST (History)
1 user (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-01-03 07:58:34 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Tomcat log describing slow shut down process that does not end. (18.84 KB, text/x-log)
2012-08-21 05:00 EDT, Michal Karm Babacek
no flags Details
Thread dump at the end of catalina.out. (34.40 KB, text/plain)
2012-08-23 04:56 EDT, Michal Karm Babacek
no flags Details

  None (edit)
Description Michal Karm Babacek 2012-08-21 05:00:48 EDT
Created attachment 605887 [details]
Tomcat log describing slow shut down process that does not end.

Scenario
========
 * 1 Tomcat node:
   server.xml
   ----------
   <Listener
      className="org.jboss.modcluster.container.catalina.standalone.ModClusterListener"
      loadMetricClass="org.jboss.modcluster.load.metric.impl.BusyConnectorsLoadMetric"
      loadMetricCapacity="1"
      loadHistory="9"
      loadDecayFactor="2"
      stickySession="true"
      stickySessionForce="false"
      stickySessionRemove="true"
      advertise="true"
      advertiseGroupAddress="224.0.1.105"
      advertisePort="23364"
      excludedContexts="ROOT,manager,docs,examples,host-manager"
   />
 * 1 httpd balancer
   mod_cluster.conf
   ----------------
   MemManagerFile "/root/workspace/jboss-ews-2.0/httpd/cache/mod_cluster"
   
   # MOD_CLUSTER_ADDS
   # Adjust to you hostname and subnet.
   <IfModule manager_module>
     Listen 6666
     <VirtualHost dell-pe1950-05.rhts.eng.brq.redhat.com:6666>
       <Directory />
         Order deny,allow
         Deny from all
         Allow from all
       </Directory>
       KeepAliveTimeout 60
       MaxKeepAliveRequests 0
       ServerAdvertise on
       AdvertiseFrequency 5
       ManagerBalancerName qacluster
       AdvertiseGroup 224.0.1.105:23364
       EnableMCPMReceive
       <Location /mcm>
         SetHandler mod_cluster-manager
         Order deny,allow
         Deny from all
         Allow from all
       </Location>
     </VirtualHost>
   </IfModule>

* *100* simple JSP applications are being deployed as 100 different contexts.OK.
* Every single one of these contexts is accessed by a client via balancer exactly once. Response code is HTTP 200, OK.
* Tomcat is being shut down via *shutdown.sh*
* Tomcat does not shut down, NOT OK :-(

Furthermore, as you may see at the bottom of the attached log, connectors are being stopped, yet, the tomcat process does not shut down...
Comment 1 Michal Karm Babacek 2012-08-21 09:26:33 EDT
Default:
stopContextTimeout	10
stopContextTimeoutUnit	TimeUnit.SECONDS

Docs:
"The amount of time, measured in units specified by stopContextTimeoutUnit, indicating how long to wait for a clean shutdown of a context (completion of pending requests for a distributable context or destruction/expiration of active sessions for a non-distributable context)."

 
So..if the client destroyed each session, tomcat would not have been waiting 10 seconds for each of the 100 contexts :-)

Anyhow, the problem holds: 
   Even after this 100x10 seconds time, the Tomcat does not shut down...
Comment 2 Jean-frederic Clere 2012-08-22 04:37:32 EDT
Get a thread dump of the cat when it is hanging.
Comment 3 Michal Karm Babacek 2012-08-23 04:56:47 EDT
Created attachment 606483 [details]
Thread dump at the end of catalina.out.

Adding a thread dump.
1)start
2)deploy 100 contexts
3)access contexts
4)shutdown.sh
5)process hangs forever
Comment 4 Michal Karm Babacek 2012-08-24 08:02:41 EDT
EDIT: FYI - the amount of contexts deployed affects only connectors. The Tomcat6 itself does not shutdown even with 1 context deployed.

I am escalating this bug to URGENT.
Comment 5 Michal Karm Babacek 2012-08-30 03:40:39 EDT
JIRA: https://issues.jboss.org/browse/JBPAPP-9788
Comment 6 Libor Fuka 2012-11-05 08:12:58 EST
Closed, because https://issues.jboss.org/browse/JBPAPP-9788 is closed now

Note You need to log in before you can comment on or make changes to this bug.