Bug 1324030 - ClassCastException at cache.getWithMetadata(key) in HotRod client during JDG rolling upgrade
Summary: ClassCastException at cache.getWithMetadata(key) in HotRod client during JDG ...
Keywords:
Status: CLOSED UPSTREAM
Alias: None
Product: JBoss Data Grid 6
Classification: JBoss
Component: Server
Version: 6.6.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ER1
: 6.6.1
Assignee: Tristan Tarrant
QA Contact: Martin Gencur
URL:
Whiteboard:
Depends On:
Blocks: 1309749 1328307
TreeView+ depends on / blocked
 
Reported: 2016-04-05 11:38 UTC by Masafumi Miura
Modified: 2025-02-10 03:48 UTC (History)
4 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2025-02-10 03:48:55 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 2302971 0 None None None 2016-05-09 10:25:54 UTC

Description Masafumi Miura 2016-04-05 11:38:45 UTC
### Description of problem:

ClassCastException happened at cache.getWithMetadata(key) in HotRod client during JDG rolling upgrade

~~~
WARN: ISPN004005: Error received from the server:
java.lang.ClassCastException:
org.infinispan.container.entries.ReadCommittedEntry cannot be cast to
org.infinispan.container.entries.InternalCacheEntry
org.infinispan.client.hotrod.exceptions.HotRodClientException:Request
for message id[5] returned server error (status=0x85):
java.lang.ClassCastException:
org.infinispan.container.entries.ReadCommittedEntry cannot be cast to
org.infinispan.container.entries.InternalCacheEntry
    at org.infinispan.client.hotrod.impl.protocol.Codec20.checkForErrorsInResponseStatus(Codec20.java:336)
    at org.infinispan.client.hotrod.impl.protocol.Codec20.readPartialHeader(Codec20.java:126)
    at org.infinispan.client.hotrod.impl.protocol.Codec20.readHeader(Codec20.java:112)
    at org.infinispan.client.hotrod.impl.operations.HotRodOperation.readHeaderAndValidate(HotRodOperation.java:56)
    at org.infinispan.client.hotrod.impl.operations.AbstractKeyOperation.sendKeyOperation(AbstractKeyOperation.java:53)
    at org.infinispan.client.hotrod.impl.operations.GetWithMetadataOperation.executeOperation(GetWithMetadataOperation.java:35)
    at org.infinispan.client.hotrod.impl.operations.GetWithMetadataOperation.executeOperation(GetWithMetadataOperation.java:23)
    at org.infinispan.client.hotrod.impl.operations.RetryOnFailureOperation.execute(RetryOnFailureOperation.java:52)
    at org.infinispan.client.hotrod.impl.RemoteCacheImpl.getWithMetadata(RemoteCacheImpl.java:166)
    ...(snip)...
~~~


### Version-Release number of selected component (if applicable):

  * Target cluster: JDG 6.6.0
  * Source cluster: JDG 6.5.1


### How reproducible:

You will see ClassCastException when you follow steps to Reproduce.


### Steps to Reproduce:

1. Put some data on source cluster
2. Configure rolling upgrade (<remote-store> setting) and start target cluster
3. Call "cache.getWithMetadata(key)" against target cluster

Note:
 - ClassCastException does not happen if "cache.get(key)" is called before "cache.getWithMetadata(key)".
 - ClassCastException does not happen after completing "upgrade --dumpkeys" (or recordKnownGlobalKeyset() in JMX)on source cluster and "upgrade --synchronize=hotrod" (or synchronizeData() in JMX) on target cluster.


### Actual results:

HotRod client failed at "cache.getWithMetadata(key)" with ClassCastException


### Expected results:

HotRod client successfully obtains the result from "cache.getWithMetadata(key)" without ClassCastException


### Additional info:

ClassCastException happened at infinispan-6.4.0.Final-redhat-4-src/server/hotrod/src/main/scala/org/infinispan/server/hotrod/Decoder2x.scala:

~~~
422    def getKeyMetadata(h: HotRodHeader, k: Array[Byte], cache: Cache): GetWithMetadataResponse = {
423       val ce = cache.getCacheEntry(k)
424       if (ce != null) {
425          val ice = ce.asInstanceOf[InternalCacheEntry]
~~~

You can also see ClassCastException on JDG Server when you enable DEBUG level of "org.infinispan.server.hotrod.CacheDecodeContext" on JDG target cluster with RemoteStore:

~~~
DEBUG [org.infinispan.server.hotrod.CacheDecodeContext] (HotRodServerWorker-1) Exception caught: io.netty.handler.codec.DecoderException: org.infinispan.server.hotrod.HotRodException: java.lang.ClassCastException: org.
infinispan.container.entries.ReadCommittedEntry cannot be cast to org.infinispan.container.entries.InternalCacheEntry
        at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:417) [netty-all-4.0.18.Final-redhat-1.jar:4.0.18.Final-redhat-1]
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:141) [netty-all-4.0.18.Final-redhat-1.jar:4.0.18.Final-redhat-1]
        at org.infinispan.server.hotrod.HotRodDecoder.org$infinispan$server$core$transport$StatsChannelHandler$$super$channelRead(HotRodDecoder.scala:32) [infinispan-server-hotrod-6.4.0.Final-redhat-4.jar:6.4.0.Final-redhat-4]
        at org.infinispan.server.core.transport.StatsChannelHandler$class.channelRead(StatsChannelHandler.scala:32) [infinispan-server-core-6.4.0.Final-redhat-4.jar:6.4.0.Final-redhat-4]
        at org.infinispan.server.hotrod.HotRodDecoder.channelRead(HotRodDecoder.scala:32) [infinispan-server-hotrod-6.4.0.Final-redhat-4.jar:6.4.0.Final-redhat-4]
        at io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:341) [netty-all-4.0.18.Final-redhat-1.jar:4.0.18.Final-redhat-1]
        at io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:327) [netty-all-4.0.18.Final-redhat-1.jar:4.0.18.Final-redhat-1]
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:785) [netty-all-4.0.18.Final-redhat-1.jar:4.0.18.Final-redhat-1]
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:116) [netty-all-4.0.18.Final-redhat-1.jar:4.0.18.Final-redhat-1]
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:494) [netty-all-4.0.18.Final-redhat-1.jar:4.0.18.Final-redhat-1]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:461) [netty-all-4.0.18.Final-redhat-1.jar:4.0.18.Final-redhat-1]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:378) [netty-all-4.0.18.Final-redhat-1.jar:4.0.18.Final-redhat-1]
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:350) [netty-all-4.0.18.Final-redhat-1.jar:4.0.18.Final-redhat-1]
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116) [netty-all-4.0.18.Final-redhat-1.jar:4.0.18.Final-redhat-1]
        at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_31]
Caused by: org.infinispan.server.hotrod.HotRodException: java.lang.ClassCastException: org.infinispan.container.entries.ReadCommittedEntry cannot be cast to org.infinispan.container.entries.InternalCacheEntry
        at org.infinispan.server.hotrod.CacheDecodeContext.createServerException(CacheDecodeContext.scala:96) [infinispan-server-hotrod-6.4.0.Final-redhat-4.jar:6.4.0.Final-redhat-4]
        at org.infinispan.server.hotrod.HotRodDecoder.decode(HotRodDecoder.scala:65) [infinispan-server-hotrod-6.4.0.Final-redhat-4.jar:6.4.0.Final-redhat-4]
        at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:362) [netty-all-4.0.18.Final-redhat-1.jar:4.0.18.Final-redhat-1]
        ... 14 more
Caused by: java.lang.ClassCastException: org.infinispan.container.entries.ReadCommittedEntry cannot be cast to org.infinispan.container.entries.InternalCacheEntry
        at org.infinispan.server.hotrod.Decoder2x$.getKeyMetadata(Decoder2x.scala:425) [infinispan-server-hotrod-6.4.0.Final-redhat-4.jar:6.4.0.Final-redhat-4]
        at org.infinispan.server.hotrod.Decoder2x$.customReadKey(Decoder2x.scala:353) [infinispan-server-hotrod-6.4.0.Final-redhat-4.jar:6.4.0.Final-redhat-4]
        at org.infinispan.server.hotrod.HotRodDecoder.customDecodeKey(HotRodDecoder.scala:207) [infinispan-server-hotrod-6.4.0.Final-redhat-4.jar:6.4.0.Final-redhat-4]
        at org.infinispan.server.hotrod.HotRodDecoder.org$infinispan$server$hotrod$HotRodDecoder$$decodeKey(HotRodDecoder.scala:114) [infinispan-server-hotrod-6.4.0.Final-redhat-4.jar:6.4.0.Final-redhat-4]
        at org.infinispan.server.hotrod.HotRodDecoder$$anonfun$decode$1.apply$mcV$sp(HotRodDecoder.scala:52) [infinispan-server-hotrod-6.4.0.Final-redhat-4.jar:6.4.0.Final-redhat-4]
        at org.infinispan.server.hotrod.HotRodDecoder.wrapSecurity(HotRodDecoder.scala:219) [infinispan-server-hotrod-6.4.0.Final-redhat-4.jar:6.4.0.Final-redhat-4]
        at org.infinispan.server.hotrod.HotRodDecoder.decode(HotRodDecoder.scala:49) [infinispan-server-hotrod-6.4.0.Final-redhat-4.jar:6.4.0.Final-redhat-4]
        ... 15 more
~~~

Comment 3 Osamu Nagano 2016-04-07 03:32:13 UTC
It seems that the ClassCastException happens for the very first getWithMetadata() for every key.  ISPN-3330 and ISPN-4583 report similar error messages of "can not be cast to org.infinispan.container.entries.InternalCacheEntry" but their fixes are included in JDG 6.6.0 already.

Comment 16 Red Hat Bugzilla 2025-02-10 03:48:55 UTC
This product has been discontinued or is no longer tracked in Red Hat Bugzilla.


Note You need to log in before you can comment on or make changes to this bug.