Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
Bug 881080 - Silence SuspectExceptions
Silence SuspectExceptions
Status: ASSIGNED
Product: JBoss Data Grid 6
Classification: JBoss
Component: Infinispan (Show other bugs)
6.1.0
Unspecified Unspecified
medium Severity medium
: ---
: 6.2.0
Assigned To: Tristan Tarrant
nobody nobody
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-11-28 10:32 EST by Michal Linhard
Modified: 2015-05-17 19:06 EDT (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Known Issue
Doc Text:
In Red Hat JBoss Data Grid, <literal>SuspectExceptions</literal> are routinely raised when nodes shut down because they are unresponsive as they shut down. As a result, a <literal>SuspectException</literal> error is added to the logs. The <literal>SuspectExceptions</literal> do not affect data integrity. This is a known issue in JBoss Data Grid 6.4 and no workaround is currently available for this issue.
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
JBoss Issue Tracker ISPN-2577 Critical Resolved Silence SuspectExceptions 2018-01-22 00:24 EST

  None (edit)
Description Michal Linhard 2012-11-28 10:32:22 EST
There are lots of SuspectExceptions in resilience test run:

https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/EDG6/view/EDG-REPORTS-RESILIENCE/job/edg-60-resilience-dist-4-3/54/artifact/report/serverlogs.zip

We managed to keep these under cover for 5.1.x branch.
This is of course a cosmetic problem, but it polutes the log with unnecessary ERRORs.

Parsed logs here:
http://www.qa.jboss.com/~mlinhard/test_results/run54-parsed/
Comment 1 Michal Linhard 2012-12-03 03:37:01 EST
Created ISPN JIRA
Comment 2 JBoss JIRA Server 2013-01-15 10:46:59 EST
Mircea Markus <mmarkus@redhat.com> made a comment on jira ISPN-2577

This has more of a cosmetic impact, but nice to have for the final.
Comment 3 JBoss JIRA Server 2013-01-25 06:10:37 EST
Michal Linhard <mlinhard@redhat.com> made a comment on jira ISPN-2577

I didn't want to create a new JIRA this kind of cosmetic task, but if we're doing this SuspectException silencing, let's sync it with behaviour on hotrod client. If we have a RetryOnFailureOperation should it log an error (in the retry mode) ? My proposal would be switching this to warning.

{code}
09:25:19,154 ERROR [org.jboss.smartfrog.jdg.loaddriver.DriverThread] (DriverThread-378) Error doing: PUT key655878 to node node0003, took 1395 ms
org.infinispan.client.hotrod.exceptions.RemoteNodeSuspecException:Request for message id[1981607] returned server error (status=0x85): org.infinispan.remoting.transport.jgroups.SuspectException: One or more nodes have left the cluster while replicating command SingleRpcCommand{cacheName='testCache', command=PutKeyValueCommand{key=ByteArrayKey{data=ByteArray{size=12, hashCode=727c8fa, array=0x033e096b65793635..}}, value=CacheValue{data=ByteArray{size=1029, array=0x034304002106020a..}, version=9007207845383422}, flags=[IGNORE_RETURN_VALUES], putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1, successful=true}}
	at org.infinispan.client.hotrod.impl.protocol.Codec10.checkForErrorsInResponseStatus(Codec10.java:153)
	at org.infinispan.client.hotrod.impl.protocol.Codec10.readHeader(Codec10.java:110)
	at org.infinispan.client.hotrod.impl.operations.HotRodOperation.readHeaderAndValidate(HotRodOperation.java:78)
	at org.infinispan.client.hotrod.impl.operations.AbstractKeyValueOperation.sendPutOperation(AbstractKeyValueOperation.java:72)
	at org.infinispan.client.hotrod.impl.operations.PutOperation.executeOperation(PutOperation.java:52)
	at org.infinispan.client.hotrod.impl.operations.PutOperation.executeOperation(PutOperation.java:41)
	at org.infinispan.client.hotrod.impl.operations.RetryOnFailureOperation.execute(RetryOnFailureOperation.java:68)
	at org.infinispan.client.hotrod.impl.RemoteCacheImpl.put(RemoteCacheImpl.java:231)
	at org.infinispan.CacheSupport.put(CacheSupport.java:53)
	at org.jboss.qa.jdg.adapter.HotRodAdapter$HotRodRemoteCacheAdapter.put(HotRodAdapter.java:247)
	at org.jboss.qa.jdg.adapter.HotRodAdapter$HotRodRemoteCacheAdapter.put(HotRodAdapter.java:232)
	at org.jboss.smartfrog.jdg.loaddriver.DriverThreadImpl.makeRequest(DriverThreadImpl.java:236)
	at org.jboss.smartfrog.jdg.loaddriver.DriverThreadImpl.run(DriverThreadImpl.java:331)

{code}
Comment 4 JBoss JIRA Server 2013-01-28 11:35:56 EST
Galder Zamarreño <galder.zamarreno@redhat.com> made a comment on jira ISPN-2577

We should also silence situations like ISPN-2752. In that JIRA, a node is trying to establish a new view sending a cache topology control view, but the node is stopping.
Comment 8 Misha H. Ali 2013-05-06 23:25:22 EDT
Nominated for 6.2 release notes.
Comment 10 Michal Linhard 2013-08-27 03:05:18 EDT
Still present in JDG 6.2.0.DR3
Comment 11 JBoss JIRA Server 2013-11-21 04:47:47 EST
Anuj Shah <anujshahwork@gmail.com> made a comment on jira ISPN-2577

There's a discussion here:
https://community.jboss.org/message/846155

This issue may be more than cosmetic
Comment 12 JBoss JIRA Server 2013-11-23 05:07:50 EST
Michal Linhard <mlinhard@redhat.com> made a comment on jira ISPN-2577

This issue is about silencing the SuspectExceptions - not displaying them as error level message, since it's not an exceptional situation that a node is suspected during topology changes.

Of course currently SuspectExceptions may accompany other serious errors, but those should be logged separately.
Comment 13 Michal Linhard 2013-11-23 05:08:20 EST
Still present in 6.2.0.ER4
Comment 16 JBoss JIRA Server 2015-05-15 05:24:10 EDT
Dan Berindei <dberinde@redhat.com> updated the status of jira ISPN-2577 to Resolved

Note You need to log in before you can comment on or make changes to this bug.