Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
First Last Prev Next    This bug is not in your last search results.
Bug 917010 - CacheException: Failure while unregistering mbeans at server shutdown
CacheException: Failure while unregistering mbeans at server shutdown
Status: CLOSED CURRENTRELEASE
Product: JBoss Enterprise Application Platform 6
Classification: JBoss
Component: Clustering (Show other bugs)
6.1.1
Unspecified Unspecified
unspecified Severity medium
: DR6
: EAP 6.3.0
Assigned To: Dominik Pospisil
Jitka Kozana
Russell Dickenson
:
Depends On: 1078158
Blocks:
  Show dependency tree / graph
 
Reported: 2013-03-01 08:04 EST by Jitka Kozana
Modified: 2014-06-28 11:29 EDT (History)
9 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Previous versions of JBoss EAP 6 contained a bug in the Infinispan component that could cause the following exception at server shut down: ---- WARN [org.infinispan.jmx.CacheJmxRegistration] (MSC service thread 1-1) ISPN000032: Problems un-registering MBeans: org.infinispan.CacheException: Failure while unregistering mbeans ---- The error did not prevent successful shut down of the server and was caused by multiple stop requests (from `CacheService` and `EmbeddedCacheManagerService`) being sent to single cache instances. This issue was resolved with an upgrade to the Infinispan component.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-06-28 11:29:26 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
Add an attachment (proposed patch, testcase, etc.)


External Trackers
Tracker ID Priority Status Summary Last Updated
JBoss Issue Tracker AS7-4247 Minor Closed Server shutdown: Problems un-registering MBeans: CacheException: Failure while unregistering mbeans 2018-07-30 03:40 EDT
JBoss Issue Tracker ISPN-4127 Major Resolved CacheJmxRegistration.unregisterCacheMBean unregisters cache mbean of all caches 2018-07-30 03:40 EDT

  None (edit)
Description Jitka Kozana 2013-03-01 08:04:01 EST 
EAP 6.1.0.ER1.
Seen in pure ipv6 environment, though this may be just coincidence. 

At server shutdown, were are seeing this exception:
01:48:51,766 WARN  [org.infinispan.jmx.CacheJmxRegistration] (MSC service thread 1-1) ISPN000032: Problems un-registering MBeans: org.infinispan.CacheException: Failure while unregistering mbeans
	at org.infinispan.jmx.ComponentsJmxRegistration.unregisterMBeans(ComponentsJmxRegistration.java:105)
	at org.infinispan.jmx.AbstractJmxRegistration.unregisterMBeans(AbstractJmxRegistration.java:70)
	at org.infinispan.jmx.CacheJmxRegistration.stop(CacheJmxRegistration.java:107)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.6.0_38]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [rt.jar:1.6.0_38]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_38]
	at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_38]
	at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:203)
	at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:883)
	at org.infinispan.factories.AbstractComponentRegistry.internalStop(AbstractComponentRegistry.java:690)
	at org.infinispan.factories.AbstractComponentRegistry.stop(AbstractComponentRegistry.java:568)
	at org.infinispan.factories.ComponentRegistry.stop(ComponentRegistry.java:224)
	at org.infinispan.CacheImpl.stop(CacheImpl.java:581)
	at org.infinispan.CacheImpl.stop(CacheImpl.java:576)
	at org.infinispan.AbstractDelegatingCache.stop(AbstractDelegatingCache.java:348)
	at org.jboss.as.clustering.infinispan.subsystem.CacheService.stop(CacheService.java:96)
	at org.jboss.as.clustering.msc.AsynchronousService.stop(AsynchronousService.java:114) [jboss-as-clustering-common-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]
	at org.jboss.msc.service.ServiceControllerImpl$StopTask.stopService(ServiceControllerImpl.java:1911) [jboss-msc-1.0.4.GA-redhat-1.jar:1.0.4.GA-redhat-1]
	at org.jboss.msc.service.ServiceControllerImpl$StopTask.run(ServiceControllerImpl.java:1874) [jboss-msc-1.0.4.GA-redhat-1.jar:1.0.4.GA-redhat-1]
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_38]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_38]
	at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_38]
Caused by: javax.management.InstanceNotFoundException: jboss.infinispan:type=Cache,name="default-host/clusterbench(repl_async)",manager="web",component=Activation
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1094) [rt.jar:1.6.0_38]
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:415) [rt.jar:1.6.0_38]
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:403) [rt.jar:1.6.0_38]
	at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:506) [rt.jar:1.6.0_38]
	at org.jboss.as.jmx.PluggableMBeanServerImpl$TcclMBeanServer.unregisterMBean(PluggableMBeanServerImpl.java:613)
	at org.jboss.as.jmx.PluggableMBeanServerImpl.unregisterMBean(PluggableMBeanServerImpl.java:360)
	at org.infinispan.jmx.JmxUtil.unregisterMBean(JmxUtil.java:112)
	at org.infinispan.jmx.ComponentsJmxRegistration.unregisterMBeans(ComponentsJmxRegistration.java:101)
	... 21 more


Cache: repl async. 
whole server log is here: 
https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-failover-http-session-jvmkill-repl-async-modcluster-ipv6/4/artifact/report/config/jboss-vmg36.mw.lab.eng.bos.redhat.com/server.log

Can this be AS7-4247?
Comment 2 Ladislav Thon 2013-03-25 05:49:13 EDT 
It also happens in EAP 6.1.0.ER3 and indeed, it is NOT specific to IPv6: https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-failover-ejb-ejbstateless-shutdown-repl-async/14/console-perf21/
Comment 4 Ladislav Thon 2013-08-26 07:12:58 EDT 
Still seeing this with EAP 6.1.1.ER7. For example:

https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-failover-ejb-ejbstateless-jvmkill-repl-async/25/
Comment 5 JBoss JIRA Server 2013-12-09 08:38:50 EST 
Radoslav Husar <rhusar@redhat.com> updated the status of jira AS7-4247 to Closed
Comment 6 Paul Ferraro 2013-12-09 08:44:46 EST 
This should no longer be an issue.
Comment 8 Dominik Pospisil 2014-03-14 04:34:20 EDT 
The cause seems to be that we are getting multiple cache stop request on a single cache instance during server shutdown:

08:56:18,692 DEBUG [org.infinispan.CacheImpl] (ServerService Thread Pool -- 5) Stopping cache repl on localhost/web: java.lang.RuntimeException: StackTrace
        at org.infinispan.CacheImpl.stop(CacheImpl.java:603)
        at org.infinispan.CacheImpl.stop(CacheImpl.java:599)
        at org.infinispan.AbstractDelegatingCache.stop(AbstractDelegatingCache.java:348)
        at org.jboss.as.clustering.infinispan.subsystem.CacheService.stop(CacheService.java:97)
        at org.jboss.as.clustering.msc.AsynchronousService$2.run(AsynchronousService.java:113)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_29]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_29]
        at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_29]
        at org.jboss.threads.JBossThread.run(JBossThread.java:122) [jboss-threads-2.1.1.Final-redhat-1.jar:2.1.1.Final-redhat-1]

08:56:18,790 DEBUG [org.infinispan.CacheImpl] (MSC service thread 1-7) Stopping cache repl on localhost/web: java.lang.RuntimeException: StackTrace
        at org.infinispan.CacheImpl.stop(CacheImpl.java:603) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]
        at org.infinispan.CacheImpl.stop(CacheImpl.java:599) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]
        at org.infinispan.manager.DefaultCacheManager.stop(DefaultCacheManager.java:732) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]
        at org.infinispan.manager.AbstractDelegatingEmbeddedCacheManager.stop(AbstractDelegatingEmbeddedCacheManager.java:179) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]
        at org.jboss.as.clustering.infinispan.subsystem.EmbeddedCacheManagerService.stop(EmbeddedCacheManagerService.java:76) [jboss-as-clustering-infinispan-7.4.0.Final-redhat-SNAPSHOT.jar:7.4.0.Final-redhat-SNAPSHOT]
        at org.jboss.msc.service.ServiceControllerImpl$StopTask.stopService(ServiceControllerImpl.java:2088) [jboss-msc-1.1.5.Final.jar:1.1.5.Final]
        at org.jboss.msc.service.ServiceControllerImpl$StopTask.run(ServiceControllerImpl.java:2049) [jboss-msc-1.1.5.Final.jar:1.1.5.Final]
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_29]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_29]
        at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_29]

The first comes from CacheService instance running the cache stop method and the second comes from EmbeddedCacheManagerService containing the cache stop method. It seems that Infinispan cachemanager and MSC service management is interleaving. The CacheImpl.stop is not synchronized so there is potential conflict.
Comment 9 JBoss JIRA Server 2014-03-18 04:58:21 EDT 
Dominik Pospisil <dpospisi@redhat.com> updated the status of jira ISPN-4127 to Coding In Progress
Comment 10 JBoss JIRA Server 2014-03-18 11:21:26 EDT 
Mircea Markus <mmarkus@redhat.com> updated the status of jira ISPN-4127 to Resolved
Comment 11 Radoslav Husar 2014-03-18 11:39:39 EDT 
We will need to revisit https://bugzilla.redhat.com/show_bug.cgi?id=1036889 for Final to pick up this and other possible fixes.
Comment 12 Ladislav Thon 2014-04-03 08:58:08 EDT 
This was seen in 6.3.0.DR4 and was NOT seen in 6.3.0.DR6. I will keep this ON_QA for at least one more build and if we don't see it again, we will mark VERIFIED soon.
Comment 13 Radoslav Husar 2014-04-03 11:39:21 EDT 
No problem, sounds like a good approach for issues that can be rarely hit. You can also verify the code changes if you want.
Comment 14 Ladislav Thon 2014-04-29 07:26:50 EDT 
This wasn't seen in the last 3 builds (DR6, ER1 and ER2), marking as verified.
Comment 15 sgilda 2014-05-12 15:10:11 EDT 
Changed <literal></literal> tags in Doc Text to ticks (`) and removed <programlisting> tag to fix Bug 1096865
Note You need to log in before you can comment on or make changes to this bug.
First Last Prev Next    This bug is not in your last search results.