Bug 986307

Tomas Sykora <tsykora> made a comment on jira ISPN-3183

I've tried to remove checking for accessing old data from new node (old data = data stored into remote cache store even before starting new node which connects to it) to find out what will happen then.

RecordKnownGlobalKeyset operation issued on source node *looks* ok but I'm really not sure.

Then test will try to perform synchronizeData on target node with this result:

testRollingUpgrades(org.infinispan.test.rollingupdates.IspnRollingUpdatesTest)  Time elapsed: 36.699 sec  <<< ERROR!
javax.management.MBeanException
        at org.infinispan.jmx.ResourceDMBean.invoke(ResourceDMBean.java:271)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836)
        at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
        at org.jboss.as.jmx.PluggableMBeanServerImpl$TcclMBeanServer.invoke(PluggableMBeanServerImpl.java:527)
        at org.jboss.as.jmx.PluggableMBeanServerImpl.invoke(PluggableMBeanServerImpl.java:263)
        at org.jboss.remotingjmx.protocol.v1.ServerProxy$InvokeHandler.handle(ServerProxy.java:1058)
        at org.jboss.remotingjmx.protocol.v1.ServerProxy$MessageReciever$1.run(ServerProxy.java:225)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.infinispan.jmx.ResourceDMBean.invoke(ResourceDMBean.java:269)
        ... 9 more
Caused by: org.infinispan.commons.CacheException: ISPN020004: Could not find migration data in cache default
        at org.infinispan.upgrade.hotrod.HotRodTargetMigrator.synchronizeData(HotRodTargetMigrator.java:94)
        at org.infinispan.upgrade.RollingUpgradeManager.synchronizeData(RollingUpgradeManager.java:59)
        ... 14 more

Maybe this can help a little bit.

Tomas Sykora <tsykora> made a comment on jira ISPN-3183

I've tried to remove checking for accessing old data from new node (old data = data stored into remote cache store even before starting new node which connects to it) to find out what will happen then.

RecordKnownGlobalKeyset operation issued on source node *looks* ok but I'm really not sure because of exception below.

Then test will try to perform synchronizeData on target node with this result:

testRollingUpgrades(org.infinispan.test.rollingupdates.IspnRollingUpdatesTest)  Time elapsed: 36.699 sec  <<< ERROR!
javax.management.MBeanException
        at org.infinispan.jmx.ResourceDMBean.invoke(ResourceDMBean.java:271)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836)
        at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
        at org.jboss.as.jmx.PluggableMBeanServerImpl$TcclMBeanServer.invoke(PluggableMBeanServerImpl.java:527)
        at org.jboss.as.jmx.PluggableMBeanServerImpl.invoke(PluggableMBeanServerImpl.java:263)
        at org.jboss.remotingjmx.protocol.v1.ServerProxy$InvokeHandler.handle(ServerProxy.java:1058)
        at org.jboss.remotingjmx.protocol.v1.ServerProxy$MessageReciever$1.run(ServerProxy.java:225)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.infinispan.jmx.ResourceDMBean.invoke(ResourceDMBean.java:269)
        ... 9 more
Caused by: org.infinispan.commons.CacheException: ISPN020004: Could not find migration data in cache default
        at org.infinispan.upgrade.hotrod.HotRodTargetMigrator.synchronizeData(HotRodTargetMigrator.java:94)
        at org.infinispan.upgrade.RollingUpgradeManager.synchronizeData(RollingUpgradeManager.java:59)
        ... 14 more

Maybe this can help a little bit.

Tomas Sykora <tsykora> made a comment on jira ISPN-3183

Not absolutely sure whether this is the proper way of testing this... but I consider this scenario should work
2 JDG 6.1.GA as source cluster, 2 latest ispn-servers with this PR as a target cluster:

08:17:19,579 WARN  [org.jgroups.protocols.UDP] (multicast receiver,shared=udp) [JGRP00010] packe08:17:19,579 WARN  [org.jgroups.protocols.UDP] (multicast receiver,shared=udp) [JGRP00010] packet from 192.168.2.103:45688 has different version (3.4.0) than ours (3.3.4); packet is discarded (received 13 identical messages from 192.168.2.103:45688 in the last 66549 ms)
t from 192.168.2.103:45688 has different version (3.4.0) than ours (3.3.4); packet is discarded (received 13 identical messages from 192.168.2.103:45688 in the last 66549 ms)

Looks like we need to deal somehow with different version of jgroups as well. What do you thing? Any simple workaround for that?

Thanks!

Tomas Sykora <tsykora> made a comment on jira ISPN-3183

Tristan's PR should fix this problem.

I got it work and successfully migrated data using hotrod migrator from jdg 6.1 (ispn schema 5.2) to the latest ispn server (ispn schema 6.0).

Nice work!
Thank you

Shouldn't this be ON_QA?

I'm pretty sure that this should be ON_QA.

Setting ON_QA + target milestone back to ER2, and immediately setting as VERIFIED (in for 6.2 ER2 build) because this seems to be no longer an issue since this build.