Description of problem: A HotRod client fail over silent if one instance is shutting down if there is no Security enabled. With role based authentication the client fail sporadically with a WARN message WARN: ISPN004005: Error received from the server: java.security.PrivilegedActionException: org.infinispan.IllegalLifecycleStateException: ISPN000324: Cache 'testCache' is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container. and the following Exception: org.infinispan.client.hotrod.exceptions.HotRodClientException:Request for message id[2825] returned server error (status=0x85): java.security.PrivilegedActionException: org.infinispan.IllegalLifecycleStateException: ISPN000324: Cache 'testCache' is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container. at org.infinispan.client.hotrod.impl.protocol.Codec20.checkForErrorsInResponseStatus(Codec20.java:336) at org.infinispan.client.hotrod.impl.protocol.Codec20.readPartialHeader(Codec20.java:126) at org.infinispan.client.hotrod.impl.protocol.Codec20.readHeader(Codec20.java:112) at org.infinispan.client.hotrod.impl.operations.HotRodOperation.readHeaderAndValidate(HotRodOperation.java:56) at org.infinispan.client.hotrod.impl.operations.AbstractKeyValueOperation.sendPutOperation(AbstractKeyValueOperation.java:57) at org.infinispan.client.hotrod.impl.operations.PutIfAbsentOperation.executeOperation(PutIfAbsentOperation.java:36) at org.infinispan.client.hotrod.impl.operations.PutIfAbsentOperation.executeOperation(PutIfAbsentOperation.java:23) at org.infinispan.client.hotrod.impl.operations.RetryOnFailureOperation.execute(RetryOnFailureOperation.java:52) at org.infinispan.client.hotrod.impl.RemoteCacheImpl.putIfAbsent(RemoteCacheImpl.java:257) at org.infinispan.client.hotrod.impl.RemoteCacheSupport.putIfAbsent(RemoteCacheSupport.java:54) at HotRodTestClient.updateCache(HotRodTestClient.java:69) at HotRodTestClient.lambda$queuePut$0(HotRodTestClient.java:88) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
Created attachment 1084471 [details] reproducer.zip Steps to reproduce: - create two servers as copy - copy jdg-test1/* to one and jdg-test2/* to the other - start the nodes bin/clustered.sh -b localhost -c=custom-clustered.xml -Djboss.node.name=server0 bin/clustered.sh -b localhost -c=custom-clustered.xml -Djboss.node.name=server1 -Djboss.socket.binding.port-offset=100 - run the client from the hotrodclient directory java -cp /..../jboss-datagrid-6.5.1-remote-java-client/*:target/hotrodtestclient-1.0-SNAPSHOT.jar -Dinfinispan.client.hotrod.server_list="localhost:11222;localhost:11322" -Dinfinispan.client.hotrod.trust_store_file_name=localhost.jks -Dinfinispan.client.hotrod.trust_store_password=localhost.test HotRodTestClient testwriter Test1Writer! - stop one of the servers (maybe retry)
Galder Zamarreño <galder.zamarreno> updated the status of jira ISPN-5875 to Coding In Progress