Bug 1000450

Summary:	JGroups: failed handling incoming message: NullPointerException
Product:	[JBoss] JBoss Enterprise Application Platform 6	Reporter:	Ladislav Thon <lthon>
Component:	Clustering	Assignee:	jboss-set
Status:	CLOSED EOL	QA Contact:	Michal Vinkler <mvinkler>
Severity:	low	Docs Contact:
Priority:	low
Version:	6.1.1, 6.2.0	CC:	jkudrnac, paul.ferraro, rhusar, rjanik
Target Milestone:	---
Target Release:	EAP 6.4.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-08-19 12:48:43 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:

Description Ladislav Thon 2013-08-23 13:21:22 UTC

During EAP 6.1.1.ER7 testing, I've seen several occurences of this NPE:

11:53:25,992 ERROR [org.jgroups.protocols.UDP] (OOB-19,shared=udp) failed handling incoming message: java.lang.NullPointerException
	at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768) [rt.jar:1.6.0_45]
	at org.jgroups.protocols.UNICAST2.up(UNICAST2.java:432)
	at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:645)
	at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:147)
	at org.jgroups.protocols.FD.up(FD.java:253)
	at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:288)
	at org.jgroups.protocols.MERGE3.up(MERGE3.java:290)
	at org.jgroups.protocols.Discovery.up(Discovery.java:359)
	at org.jgroups.protocols.TP$ProtocolAdapter.up(TP.java:2610)
	at org.jgroups.protocols.TP.passMessageUp(TP.java:1263)
	at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1825)
	at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1798)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) [rt.jar:1.6.0_45]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) [rt.jar:1.6.0_45]
	at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45]

In the log, this NPE is surrounded by exceptions from bug 923818, but this totally looks like a problem in JGroups.

See: https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-failover-ejb-ejbservlet-shutdown-dist-async/15/console-perf20/ (so that's graceful shutdown, DIST/ASYNC).

Comment 1 Radoslav Husar 2013-08-27 14:19:08 UTC

Upstream Jira: 
https://issues.jboss.org/browse/JGRP-1684

Comment 2 Radoslav Husar 2013-09-03 13:12:49 UTC

Can you guys provide more information about the NPE? What are the circumstances, whether you have seen it in other jobs, etc. I wonder if this couldn't have been another instance messing up with the test cluster.

I am setting also severity/priority.

Comment 3 Ladislav Thon 2013-09-03 13:31:21 UTC

Only seen this in a single job. Other than that, we have no additional information besides what can be seen from the logs.

Comment 7 Paul Ferraro 2013-10-16 12:54:28 UTC

This is no longer addressable in the time remaining for release.

Comment 9 Richard Janík 2013-12-09 10:00:06 UTC

Just an update: present in EAP 6.2.0.

Comment 10 Radoslav Husar 2013-12-09 11:45:00 UTC

Please add a link to the job where it was seen, need to look again at what was the exact timing of the issue.

Comment 11 Richard Janík 2013-12-09 12:17:40 UTC

Logs from 6.2.0.CR3 are here (both from the same test):

https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-failover-ejb-ejbservlet-undeploy-dist-sync/lastSuccessfulBuild/artifact/report/config/jboss-perf21/server.log

https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-failover-ejb-ejbservlet-undeploy-dist-sync/lastSuccessfulBuild/artifact/report/config/jboss-perf19/server.log

and the build (with all the logs) is here:

https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-failover-ejb-ejbservlet-undeploy-dist-sync/32/