Bug 1108035
| Summary: | Tests for suppressing state transfer with jmx fail | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [JBoss] JBoss Data Grid 6 | Reporter: | Jakub Markos <jmarkos> | ||||
| Component: | Server | Assignee: | Tristan Tarrant <ttarrant> | ||||
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Martin Gencur <mgencur> | ||||
| Severity: | medium | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 6.3.0 | CC: | jawilson, jdg-bugs, wburns, wfink | ||||
| Target Milestone: | ER3 | ||||||
| Target Release: | 6.3.0, 6.4.0 | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2015-01-26 14:03:33 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
The trace logs are from the run of this class https://github.com/infinispan/infinispan/blob/master/server/integration/testsuite/src/test/java/org/infinispan/server/test/jmx/suppress/statetransfer/StateTransferSuppressForHotrodIT.java and this test case only: https://github.com/infinispan/infinispan/blob/master/server/integration/testsuite/src/test/java/org/infinispan/server/test/jmx/suppress/statetransfer/AbstractStateTransferSuppressIT.java#L133 (fails on line 196, when starting the 3rd node) This has been verified a long time ago. |
Created attachment 907573 [details] Server TRACE logs This started failing with 6.3.0.ER6, ER5 was OK. When the 3rd node joins the cluster, it hangs for ~ 4 minutes, after which this exception occurs on the 3rd node: 12:00:43,839 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-5) MSC000001: Failed to start service jboss.infinispan.clustered.default: org.jboss.msc.service.StartException in service jboss.infinispan.clustered.default: Failed to start service at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1767) [jboss-msc-1.0.4.GA-redhat-1.jar:1.0.4.GA-redhat-1] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [rt.jar:1.7.0_06] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [rt.jar:1.7.0_06] at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_06] Caused by: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete() throws java.lang.InterruptedException on object of type StateTransferManagerImpl at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:170) at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869) at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638) at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627) at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530) at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:216) at org.infinispan.CacheImpl.start(CacheImpl.java:696) at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:574) at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:529) at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:409) at org.infinispan.registry.ClusterRegistryImpl.startRegistryCache(ClusterRegistryImpl.java:136) at org.infinispan.registry.ClusterRegistryImpl.addListener(ClusterRegistryImpl.java:97) at org.infinispan.query.remote.ProtobufMetadataManager.ensureInit(ProtobufMetadataManager.java:65) at org.infinispan.query.remote.ProtobufMetadataManager.getSerializationContext(ProtobufMetadataManager.java:132) at org.infinispan.query.remote.LifecycleManager.cacheStarting(LifecycleManager.java:114) at org.infinispan.factories.ComponentRegistry.notifyCacheStarting(ComponentRegistry.java:228) at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:214) at org.infinispan.CacheImpl.start(CacheImpl.java:696) at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:574) at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:529) at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:409) at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:423) at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:89) at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:80) at org.jboss.as.clustering.infinispan.subsystem.SecurityActions$5.run(SecurityActions.java:101) at org.jboss.as.clustering.infinispan.subsystem.SecurityActions$5.run(SecurityActions.java:98) at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_06] at org.jboss.as.clustering.infinispan.subsystem.SecurityActions.startCache(SecurityActions.java:109) at org.jboss.as.clustering.infinispan.subsystem.CacheService.start(CacheService.java:78) at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) [jboss-msc-1.0.4.GA-redhat-1.jar:1.0.4.GA-redhat-1] at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) [jboss-msc-1.0.4.GA-redhat-1.jar:1.0.4.GA-redhat-1] ... 3 more Caused by: org.infinispan.commons.CacheException: Initial state transfer timed out for cache __cluster_registry_cache__ on node2/clustered at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:209) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_06] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_06] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_06] at java.lang.reflect.Method.invoke(Method.java:601) [rt.jar:1.7.0_06] at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:168) ... 33 more The tests fail with arquillian error though, because arquillian gives the server only 60s to start. TRACE logs attached.