More information in linked JIRA. Setting severity as High, because this issue prevents the whole process of rolling upgrades to be finished as specified in specification DOCs.
Tomas Sykora <tsykora> made a comment on jira ISPN-3552 *New observations:* I found this while testing HotRod rolling upgrades from 2 nodes -- JDG 6.1 GA to 2 nodes -- JDG 6.2 ER3: ------------------------------------------------------------------------------- Test set: org.infinispan.test.rollingupdatesdist.HotRodRollUpsDistTest ------------------------------------------------------------------------------- Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 90.917 sec <<< FAILURE! testRollingUpgrades(org.infinispan.test.rollingupdatesdist.HotRodRollUpsDistTest) Time elapsed: 86.902 sec <<< ERROR! java.io.IOException: java.lang.ClassNotFoundException: org.infinispan.CacheException at org.jboss.remotingjmx.protocol.v1.ClientConnection$BaseResponseHandler.handle(ClientConnection.java:1775) at org.jboss.remotingjmx.protocol.v1.ClientConnection$MessageReceiver$1.run(ClientConnection.java:455) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) Caused by: java.lang.ClassNotFoundException: org.infinispan.CacheException at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:423) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:356) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.jboss.marshalling.AbstractClassResolver.loadClass(AbstractClassResolver.java:135) at org.jboss.marshalling.AbstractClassResolver.resolveClass(AbstractClassResolver.java:116) at org.jboss.marshalling.river.RiverUnmarshaller.doReadClassDescriptor(RiverUnmarshaller.java:947) at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1259) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:276) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:213) at org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1732) at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1648) at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1290) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:276) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:213) at org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1732) at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1648) at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1290) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:276) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:213) at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:72) at org.jboss.remotingjmx.protocol.v1.ClientConnection$BaseResponseHandler.handle(ClientConnection.java:1766) ... 4 more
Pedro Ruivo <pedroruivo2> updated the status of jira ISPN-3552 to Coding In Progress
This issue depends on: https://bugzilla.redhat.com/show_bug.cgi?id=1026862 Formerly, I thought 1026862 exception is related to disabling remote cache store. This is not true and this issue blocks even the first of rolling upgrades operations -- recordKnownGlobalKeyset. Once https://bugzilla.redhat.com/show_bug.cgi?id=1026862 is fixed and verified, this issue can be consequently verified as well. We should remove this (1011499) bugzilla from beta release notes as it is superseded by 1026862 for now.
The whole smooth process is still blocked by https://bugzilla.redhat.com/show_bug.cgi?id=1030485 (HotRod Rolling Upgrades from Delirium 5.2.4 (JDG-6.1) to latest server needs dropped org.infinispan.util.ByteArrayKey) which prevents us from synchronization of data. But disabling stores looks not to be dependant on it. Therefore, we verified disconnectSource operation without issuing sychnonizeData to see, whether this is working properly. And YES, this is working now. Thank you for the fix. VERIFIED