Bug 749642 - a single group definition with a short recalculation interval can result in many duplicate/redundant dynamic compat groups, with the same name and same membership
Summary: a single group definition with a short recalculation interval can result in m...
Status: CLOSED WORKSFORME
Alias: None
Product: RHQ Project
Classification: Other
Component: Core Server
Version: 4.2
Hardware: Unspecified
OS: Unspecified
medium
urgent vote
Target Milestone: ---
: ---
Assignee: Lukas Krejci
QA Contact: Mike Foley
URL:
Whiteboard:
Keywords:
Depends On:
Blocks: jon3
TreeView+ depends on / blocked
 
Reported: 2011-10-27 19:01 UTC by Ian Springer
Modified: 2013-12-10 13:55 UTC (History)
3 users (show)

(edit)
Clone Of:
(edit)
Last Closed: 2013-12-10 13:55:41 UTC


Attachments (Terms of Use)

Description Ian Springer 2011-10-27 19:01:15 UTC
I have two group definitions defined:

name: compats
expr: groupby resource.type.plugin
      groupby resource.type.name
recalc interval: 9999ms

and:

name: namedthesame
expr: groupby resource.type.plugin
      groupby resource.type.name
      groupby resource.name
recalc interval: 9999ms

Both end up resulting in many duplicates for *some* of the resulting compat groups. The other resulting compat groups do not have duplicates.

Here are the 62 'Linux' compat groups created based off the 'compats' group definition. Notice they only differ in id:

ResourceGroup[id=10102, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=10104, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=10103, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=10014, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=10072, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=10049, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=10013, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=10456, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=10926, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=11095, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=11264, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=11602, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=11433, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=11940, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=11771, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=12109, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=12278, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=12447, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=12616, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=12785, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=12954, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=13123, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=13292, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=13461, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=13630, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=13968, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=13799, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=14306, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=14137, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=14475, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=14982, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=15996, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=18024, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=16334, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=16165, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=15151, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=16503, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=15320, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=14644, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=16841, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=15489, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=14813, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=15658, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=17686, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=16672, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=17517, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=19038, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=19207, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=18531, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=18869, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=19376, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=17010, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=18362, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=19545, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=17179, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=17855, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=17348, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=18700, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=15827, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=18193, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=19670, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]
ResourceGroup[id=19839, name=DynaGroup - compats ( Platforms,Linux ), category=COMPATIBLE, type=Linux, isDynaGroup=true, isClusterGroup=false]]


Note, I manually recalculate both group a number of times, and they are also both set to auto-recalculate every 10 seconds. I'm guessing the frequent recalculation is somehow causing of the duplicate groups. If so, we could potentially workaround the issue by:

1) disallowing a recalc of a groupdef if a recalc is already in progress for that groupdef

and.or

2) setting the minimum recalc interval to 1m, or even 5m.

Comment 1 Charles Crouch 2011-10-31 15:32:13 UTC
Lets just set the min recalc period to be 2mins

Comment 2 Ian Springer 2011-10-31 16:07:57 UTC
Note, that without also doing 1), there is still potential for duplicate groups to be created in the following scenarios:

1) dynagroup recalculation takes longer than 2 mins, so a second auto-recalc gets kicked off before the first auto-recalc has completed

2) user repeatedly clicks the Recalculate button while a recalc is already in progress

Comment 3 Elias Ross 2012-08-08 23:21:41 UTC
I have been bitten by this in RHQ 4.2. I have never intentionally set the interval too low but with a busy server with lots of database locking I have seen hundreds of groups created.

Comment 5 Heiko W. Rupp 2013-12-07 15:02:25 UTC
Lukas, could you please have a look if this is still an issue?

Comment 6 Lukas Krejci 2013-12-10 13:55:41 UTC
I could not reproduce this in 4.10.0-SNAPSHOT.

Since 4.8, the minimum recalculation interval for dynagroups is 1 minute, but even when I concurrently logged on from up to 20 CLI sessions and triggered group recalculation manually (calling GroupDefinitionManager.calculateGroupMembership(int)), I could not reproduce the issue.

I also tried to put the server under load by importing large inventories - even during this time, the recalculation worked OK.

Thus, I am lead to believe this has been addressed some time between RHQ 4.2 and 4.10 and I'm closing this BZ.


Note You need to log in before you can comment on or make changes to this bug.