749642 – a single group definition with a short recalculation interval can result in many duplicate/redundant dynamic compat groups, with the same name and same membership

Bug 749642 - a single group definition with a short recalculation interval can result in many duplicate/redundant dynamic compat groups, with the same name and same membership

Summary: a single group definition with a short recalculation interval can result in m...

Keywords:
Status:	CLOSED WORKSFORME
Alias:	None
Product:	RHQ Project
Classification:	Other
Component:	Core Server
Sub Component:
Version:	4.2
Hardware:	Unspecified
OS:	Unspecified
Priority:	medium
Severity:	urgent
Target Milestone:	---
Target Release:	---
Assignee:	Lukas Krejci
QA Contact:	Mike Foley
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:	jon3
TreeView+	depends on / blocked

Reported:	2011-10-27 19:01 UTC by Ian Springer
Modified:	2013-12-10 13:55 UTC (History)
CC List:	3 users (show)
Fixed In Version:
Clone Of:
Environment:
Last Closed:	2013-12-10 13:55:41 UTC
Embargoed:

Attachments	(Terms of Use)

Description Ian Springer 2011-10-27 19:01:15 UTC

I have two group definitions defined:

name: compats
expr: groupby resource.type.plugin
      groupby resource.type.name
recalc interval: 9999ms

and:

name: namedthesame
expr: groupby resource.type.plugin
      groupby resource.type.name
      groupby resource.name
recalc interval: 9999ms

Both end up resulting in many duplicates for *some* of the resulting compat groups. The other resulting compat groups do not have duplicates.

Here are the 62 'Linux' compat groups created based off the 'compats' group definition. Notice they only differ in id:

ResourceGroup[id=10102, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=10104, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=10103, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=10014, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=10072, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=10049, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=10013, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=10456, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=10926, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=11095, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=11264, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=11602, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=11433, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=11940, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=11771, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=12109, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=12278, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=12447, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=12616, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=12785, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=12954, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=13123, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=13292, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=13461, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=13630, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=13968, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=13799, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=14306, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=14137, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=14475, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=14982, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=15996, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=18024, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=16334, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=16165, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=15151, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=16503, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=15320, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=14644, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=16841, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=15489, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=14813, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=15658, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=17686, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=16672, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=17517, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=19038, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=19207, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=18531, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=18869, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=19376, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=17010, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=18362, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=19545, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=17179, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=17855, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=17348, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=18700, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=15827, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=18193, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=19670, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=19839, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]]


Note, I manually recalculate both group a number of times, and they are also both set to auto-recalculate every 10 seconds. I'm guessing the frequent recalculation is somehow causing of the duplicate groups. If so, we could potentially workaround the issue by:

1) disallowing a recalc of a groupdef if a recalc is already in progress for that groupdef

and.or

2) setting the minimum recalc interval to 1m, or even 5m.

Comment 1 Charles Crouch 2011-10-31 15:32:13 UTC

Lets just set the min recalc period to be 2mins

Comment 2 Ian Springer 2011-10-31 16:07:57 UTC

Note, that without also doing 1), there is still potential for duplicate groups to be created in the following scenarios:

1) dynagroup recalculation takes longer than 2 mins, so a second auto-recalc gets kicked off before the first auto-recalc has completed

2) user repeatedly clicks the Recalculate button while a recalc is already in progress

Comment 3 Elias Ross 2012-08-08 23:21:41 UTC

I have been bitten by this in RHQ 4.2. I have never intentionally set the interval too low but with a busy server with lots of database locking I have seen hundreds of groups created.

Comment 5 Heiko W. Rupp 2013-12-07 15:02:25 UTC

Lukas, could you please have a look if this is still an issue?

Comment 6 Lukas Krejci 2013-12-10 13:55:41 UTC

I could not reproduce this in 4.10.0-SNAPSHOT.

Since 4.8, the minimum recalculation interval for dynagroups is 1 minute, but even when I concurrently logged on from up to 20 CLI sessions and triggered group recalculation manually (calling GroupDefinitionManager.calculateGroupMembership(int)), I could not reproduce the issue.

I also tried to put the server under load by importing large inventories - even during this time, the recalculation worked OK.

Thus, I am lead to believe this has been addressed some time between RHQ 4.2 and 4.10 and I'm closing this BZ.

Note You need to log in before you can comment on or make changes to this bug.