Bug 1177228
| Summary: | JDG 6.4.0.ER8: Topology hint breaks cross site replication in CacheTopologyControlCommand | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [JBoss] JBoss Data Grid 6 | Reporter: | Osamu Nagano <onagano> | ||||
| Component: | Infinispan | Assignee: | Pedro Ruivo <pruivo> | ||||
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Martin Gencur <mgencur> | ||||
| Severity: | unspecified | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | unspecified | CC: | bban, dmehra, dstahl, jdg-bugs, jpallich, ksuzumur, mcimbora, tkimura | ||||
| Target Milestone: | CR1 | ||||||
| Target Release: | 6.4.0 | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | Type: | Bug | |||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | |||||||
| Bug Blocks: | 1153111 | ||||||
| Attachments: |
|
||||||
Created attachment 972927 [details]
clustered.xml for site A
The issue seems to be JGroups-related and is caused by setting relay.enable_address_tagging property to true. In that case, RELAY2 protocol adds a new address generator, which influences resulting address (ExtendedUUID instead of TopologyUUID). JGroupsTransport.fromJGroupsAddress then returns wrong address wrapper class (JGroupsAddress instead of JGroupsTopologyAwareAddress), resulting in aforementioned ClassCastException. Possible workaround (which however ignores can_become_site_master=false setting, if used): Replace <property name="enable_address_tagging">true</property> with <property name="enable_address_tagging">false</property>. |
Description of problem: When topology hints like site, rack and machine are set under cross site replication configuration, server will keep generating the following WARN log messages. Version-Release number of selected component (if applicable): JDG 6.4.0.ER8 (Infinispan 6.2.0.ER8) How reproducible: Always Steps to Reproduce: 1. Just start a server with the attached xml file. Actual results: ~~~ 17:51:03,037 WARN [org.infinispan.topology.CacheTopologyControlCommand] (MSC service thread 1-3) ISPN000071: Caught exception when handling command CacheTopologyControlCommand{cache=namedCache, type=JOIN, sender=svr01/xSite-A ( flags=2 (can_be_sm), site-id=s1, rack-id=r1, machine-id=m1), joinInfo=CacheJoinInfo{consistentHashFactory=org.infinispan.distribution.ch.TopologyAwareConsistentHashFactory@f0d, hashFunction=MurmurHash3, numSegments=80, numOwners=2, timeout=60000, totalOrder=false, distributed=true}, topologyId=0, rebalanceId=0, currentCH=null, pendingCH=null, availabilityMode=null, actualMembers=null, throwable=null, viewId=0}: java.lang.ClassCastException: org.infinispan.remoting.transport.jgroups.JGroupsAddress cannot be cast to org.infinispan.remoting.transport.TopologyAwareAddress at org.infinispan.distribution.topologyaware.TopologyInfo.addTopology(TopologyInfo.java:56) [infinispan-core-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at org.infinispan.distribution.topologyaware.TopologyInfo.<init>(TopologyInfo.java:37) [infinispan-core-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at org.infinispan.distribution.ch.TopologyAwareConsistentHashFactory.addBackupOwners(TopologyAwareConsistentHashFactory.java:27) [infinispan-core-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at org.infinispan.distribution.ch.DefaultConsistentHashFactory.rebalanceBuilder(DefaultConsistentHashFactory.java:126) [infinispan-core-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at org.infinispan.distribution.ch.DefaultConsistentHashFactory.create(DefaultConsistentHashFactory.java:41) [infinispan-core-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at org.infinispan.distribution.ch.DefaultConsistentHashFactory.create(DefaultConsistentHashFactory.java:25) [infinispan-core-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at org.infinispan.topology.ClusterCacheStatus.createInitialCacheTopology(ClusterCacheStatus.java:567) [infinispan-core-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at org.infinispan.topology.ClusterCacheStatus.doJoin(ClusterCacheStatus.java:551) [infinispan-core-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at org.infinispan.topology.ClusterTopologyManagerImpl.handleJoin(ClusterTopologyManagerImpl.java:131) [infinispan-core-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at org.infinispan.topology.CacheTopologyControlCommand.doPerform(CacheTopologyControlCommand.java:162) [infinispan-core-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at org.infinispan.topology.CacheTopologyControlCommand.perform(CacheTopologyControlCommand.java:144) [infinispan-core-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at org.infinispan.topology.LocalTopologyManagerImpl.executeOnCoordinator(LocalTopologyManagerImpl.java:377) [infinispan-core-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at org.infinispan.topology.LocalTopologyManagerImpl.join(LocalTopologyManagerImpl.java:103) [infinispan-core-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at org.infinispan.statetransfer.StateTransferManagerImpl.start(StateTransferManagerImpl.java:109) [infinispan-core-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_71] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_71] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_71] at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_71] at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:168) [infinispan-commons-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869) [infinispan-core-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638) [infinispan-core-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627) [infinispan-core-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530) [infinispan-core-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:216) [infinispan-core-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at org.infinispan.CacheImpl.start(CacheImpl.java:756) [infinispan-core-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:581) [infinispan-core-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:536) [infinispan-core-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:414) [infinispan-core-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:428) [infinispan-core-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:89) [infinispan-server-infinispan-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:80) [infinispan-server-infinispan-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at org.infinispan.server.infinispan.SecurityActions$4.run(SecurityActions.java:104) [infinispan-server-infinispan-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at org.infinispan.server.infinispan.SecurityActions$4.run(SecurityActions.java:101) [infinispan-server-infinispan-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at org.infinispan.security.Security.doPrivileged(Security.java:76) [infinispan-core-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at org.infinispan.server.infinispan.SecurityActions.doPrivileged(SecurityActions.java:48) [infinispan-server-infinispan-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at org.infinispan.server.infinispan.SecurityActions.startCache(SecurityActions.java:109) [infinispan-server-infinispan-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at org.jboss.as.clustering.infinispan.subsystem.CacheService.start(CacheService.java:79) [infinispan-server-infinispan-6.2.0.ER8-redhat-1.jar:6.2.0.ER8-redhat-1] at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1980) [jboss-msc-1.1.5.Final-redhat-1.jar:1.1.5.Final-redhat-1] at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1913) [jboss-msc-1.1.5.Final-redhat-1.jar:1.1.5.Final-redhat-1] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_71] at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_71] ~~~ Expected results: No WARN logs and stack traces. Additional info: If the topology hints are removed from the blow, no issues happen. ~~~ <transport type="UDP" socket-binding="jgroups-udp" site="${jboss.jgroups.transport.site:s1}" rack="${jboss.jgroups.transport.rack:r1}" machine="${jboss.jgroups.transport.machine:m1}"/> ~~~