Martin Gencur <mgencur@redhat.com> made a comment on jira ISPN-3398

Are you sure you're using the same version of HotRod client and HotRod server? The server is trying to unmarshall the data sent by HotRod client. This issue appears only if HotRod server is trying to retrieve an object from byte array that was not created as a result of marshalling or the versions of marshallers were different.

Martin Gencur <mgencur@redhat.com> made a comment on jira ISPN-3398

Ah, I overlooked that the version in the error message changes. This signifies that the byte array that is being unmarshalled is not a product of marshalling (e.g. byte array created in some other way or already unmarshalled)

Martin Gencur <mgencur@redhat.com> made a comment on jira ISPN-3398

I found the root cause. It is similar to ISPN-3406 where boxValue/boxKey/unboxValue is called twice. First on the node where the Put command originated and then on the other node where the data is replicated. Since the data was already unmarshalled on the originating node, another attempt to unmarshall the data from the byte array fails with the exception above. 

This is only replicable in dist/repl mode and with byte array values. The HotRodTypeConverter handles byte arrays in a special way and hence this problem appears.

However, this problem is more general. Not only put command is affected but IMO also replace and remove commands. And not only with bytearray values but also with bytearray keys. Also, I think that if we specified a marshaller for EmbeddedTypeConverter, it would not be possible to store byte array values either, and that also in single-node deployment. That's because the converter is automatically trying to unmarshall the byte array (but of course it depends how the marshaller is implemented).

Here you can find tests to reproduce the issues for all the operations: 
 https://github.com/mgencur/infinispan/tree/ISPN-3398/unmarshall_compat

Still present in 6.2.0.DR4

Verified for 6.2.0.ER2