Bug 1014300

Summary:	java.util.NoSuchElementException thrown when deleting RHQ plugins
Product:	[Other] RHQ Project	Reporter:	Thomas Segismont <tsegismo>
Component:	Core Server	Assignee:	Thomas Segismont <tsegismo>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Mike Foley <mfoley>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	4.9	CC:	hrupp, jshaughn, lkrejci
Target Milestone:	---
Target Release:	RHQ 4.10
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:
Clones:	1014518 (view as bug list)		Environment:
Last Closed:	2014-04-23 12:30:40 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1014518

Description Thomas Segismont 2013-10-01 16:50:13 UTC

Description of problem:
When an admin tries to purge a plugin with a lot of measurement definitions (Oracle plugin), the server fails to delete them (throws java.util.NoSuchElementException)

Version-Release number of selected component (if applicable):
4.9

How reproducible:
Always

Steps to Reproduce:
1. Delete Oracle plugin
2. Purge Oracle plugin
3. Wait for the background jobs to run

Actual results:
The server throws java.util.NoSuchElementException
http://pastebin.com/gjNbywA5

Expected results:
The server purges the plugin

Comment 1 Thomas Segismont 2013-10-01 16:54:04 UTC

Fixed in bug branch bug/1014300

commit 514fef0f46c9d5e0c4546a7a08f27ca6b629614e
Author: Thomas Segismont <tsegismo>
Date:   Tue Oct 1 18:51:20 2013 +0200

Do not use the criteria query iterator remove function

https://git.fedorahosted.org/cgit/rhq/rhq.git/commit/?h=bug/1014300&id=514fef0f46c9d5e0c4546a7a08f27ca6b629614e

Comment 2 Lukas Krejci 2013-10-01 20:02:12 UTC

I think the error is in the CriteriaQuery iterator implementation rather than in the MeasurementMetadataManagerBean.

The NSEE is thrown on line CriteriaQuery.java#145, which is a call to the underlying iterator.next(). This is directly preceeded by an "if (!iterator.hasNext())" branch.

So the only way (apart from a bug in JDK's ArrayList's iterator impl) line 145 can throw an exception is when the iterator has no more elements, the test for that on line 113 suceeds and the iterator is reinitialized on line 142.

The code therefore assumes that it never gets an empty collection as a result of running a criteria query, which I think is wrong.

Comment 3 Lukas Krejci 2013-10-01 20:14:53 UTC

Well, an empty collection as a last page is not technically a wrong assumption, but it is no longer a correct assumption in the light of possibility of inconsistent results obtained from the criteria query.

The fact that you're only seeing this while deleting stuff en masse is an indicator that in fact that is what you might have uncovered.

Comment 4 Thomas Segismont 2013-10-01 21:02:38 UTC

Thanks for the quick and excellent analysis Lukas.

IIUC, fixing CriteriaQuery iterator implementation will not be a piece of cake. While I think it should be fixed, I also believe that we should commit a workaround for plugin purge very soon.

Besides, I don't understand the real value of using an iterator and iterator#remove here (considering we have to call measurementDefinitionMgr.removeMeasurementDefinition)

Unless I'm wrong and we can fix CriteriaQuery iterator quickly, I'd like to create another BZ to track the iterator problem, and fix the plugin purge with the changes in branch bug/1014300

What do you think?

Comment 5 Lukas Krejci 2013-10-02 08:35:30 UTC

+1, I don't understand the need for the iterator#remove in that method either.

Comment 6 Thomas Segismont 2013-10-02 09:23:37 UTC

Fixed in master

commit 9118dcdb89cbb9cf7a67802b9890f222849579db
Author: Thomas Segismont <tsegismo>
Date:   Tue Oct 1 18:51:20 2013 +0200

Do not use the criteria query iterator which is buggy (besides it adds no value here: more lines and less readable code)

Comment 7 Heiko W. Rupp 2014-04-23 12:30:40 UTC

Bulk closing of 4.10 issues.

If an issue is not solved for you, please open a new BZ (or clone the existing one) with a version designator of 4.10.