Description of problem: When metric "[CacheStore] Number of cache store stores" is enabled in JBoss ON, WARN message like ***************** 2014-10-16 14:06:09,998 WARN [ResourceContainer.invoker.daemon-6451] (com.jboss.datagrid.rhq.IspnCacheComponent)- Getting metric [cacheLoaderStores] at [ Address{path: subsystem=infinispan,cache-container=clustered,distributed-cache=Item}] failed: JBAS014792: Unknown attribute cacheLoaderStores, rolled-back=true ***************** is logged in the agent.log file every X minutes where X is the collection period for above metric. Version-Release number of selected component (if applicable): JBoss ON 3.2.2 JDK 6.2 How reproducible: Always Steps to Reproduce: 1. JDG started invoking $JDG-HOME/bin/clustered.sh script 2. enabled [CacheStore] Number of cache store stores metric: JBoss ON UI -> JBossAS7 Standalone Servers -> JDG ... -> infinispan -> Cache Containers -> clustered -> Caches. Expected results: No WARN is logged and enabled metric is properly shown in the JBoss ON UI. Additional info:
JDG does have a "cache-loader-stores" attribute available in the management subsystem. But its rhq-plugin.xml has: <metric property="cacheLoaderStores" displayName="[CacheStore] Number of cache store stores" displayType="detail" units="none" dataType="measurement" description="Number of entries stored in the cache stores"/> I'm assuming the properties in rhq-plugin.xml are supposed to match what's in the JDG management subsystem? A lot of the other properties in rhq-plugin.xml are similar (do not match the actual subsystem).
Moving to Data Grid as this issue is in the JDG management plug-in. As indicated in comment 1 the issue here seems to be that the plug-in descriptor is using the wrong property name for these metrics. The plug-in should either be updated to have its getValues method properly map the legacy metric property names to the attribute name exposed by the cache module in the management API or the plug-in descriptor should be updated to use the correct attribute names as the property name. Normally you can not change a metric name in an existing plug-in descriptor without breaking the plug-in. However, if these properties never existed in previous versions of JDG, then there will be no data corruption if the property name is changed. However, if these metrics ever used the name that is defined in the current plug-in, changing them will corrupt existing management inventory that used the "legacy" name. For example, if cache-loader-stores used to be, cacheLoaderStores, changing the metric property from cacheLoaderStores to cache-loader-stores will result in loss of data and possibly failure to reconcile any monitoring data for the plug-in. In this case, the only option is to map the metric names within the getValues method of the resource component.
I can confirm this is an issue that the MetricsRemappingComponent class (performs the remapping that Larry was alluding to) is missing a single line to map cache-loader-stores metric of the cache to the cacheLoaderStores metric on the RHQ plugin.xml.
I am verifying that warning messages of a given type are not present in agent.log file while monitoring JDG 6.4.1 ER1. Thank you Will for a fix!