Bug 1072543

Summary: Unable to collect metrics from EAP 6 datasources when a system properties is used in configuration
Product: [JBoss] JBoss Operations Network Reporter: Larry O'Leary <loleary>
Component: Plugin -- JBoss EAP 6Assignee: Libor Zoubek <lzoubek>
Status: CLOSED CURRENTRELEASE QA Contact: Filip Brychta <fbrychta>
Severity: high Docs Contact:
Priority: unspecified    
Version: JON 3.2CC: fbrychta, lkrejci, lzoubek, myarboro, theute
Target Milestone: DR01   
Target Release: JON 3.3.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Collecting metrics from an EAP 6 datasource failed due to improper handling of returned data types from the DMR, specifically the minimum and maximum pool size settings of a datasource configuration. If a user declared a system property as the value of min-pool-size or max-pool-size, the DMR could not parse the properties. The fix now handles system properties in the datasource configuration file, which fixes the originally reported issue.
Story Points: ---
Clone Of:
: 1119513 (view as bug list) Environment:
Last Closed: 2014-12-11 14:01:45 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1119513    

Description Larry O'Leary 2014-03-04 18:19:52 UTC
Description of problem:
Collection of metrics from a JBoss EAP 6 datasource fails due to in proper handling of returned data types from the DMR.

This issue had already been identified in Web components in upstream RHQ bug 976291 and can now be seen in other components. Specifically, in the case of a datasource configuration and its minimum and maximum pool size settings.

This issue results in the inability to monitor JBoss EAP datasources.

Version-Release number of selected component (if applicable):
3.2.0

How reproducible:
Always

Steps to Reproduce:
1.  Install JBoss ON 3.2 system.
2.  Change `${RHQ_SERVER_HOME}/jbossas/standalone/configuration/standalone-full.xml` 

        sed -i.orig 's/<min-pool-size>2<\/min-pool-size>/<min-pool-size>${rhq.server.datasource.nonxa.minpool:2}<\/min-pool-size>/' "${RHQ_SERVER_HOME}"'/jbossas/standalone/configuration/standalone-full.xml'
        sed -i 's/<max-pool-size>5<\/max-pool-size>/<max-pool-size>${rhq.server.datasource.nonxa.maxpool:5}<\/max-pool-size>/' "${RHQ_SERVER_HOME}"'/jbossas/standalone/configuration/standalone-full.xml'

3.  Start JBoss ON 3.2 system.
4.  Import RHQ Server resource into inventory.
5.  Navigate to the monitoring schedules page of the EAP (127.0.0.1:6990) RHQ Server / datasources / DataSource / NoTxRHQDS resource.
6.  Select the **Max Pool Size setting** and **Min Pool Size setting** metrics and set their collection interval to 30 seconds.

Actual results:
Metric collection fails for all metrics that are part of this collection schedule and the following error is logged in the agent log:

    ERROR [WorkerThread#0[127.0.0.1:36143]] (rhq.core.pc.measurement.MeasurementManager)- Could not get measurement values
    java.lang.ClassCastException: java.util.LinkedHashMap cannot be cast to java.lang.Integer
        at org.rhq.modules.plugins.jbossas7.DatasourceComponent.getRCAsMetric(DatasourceComponent.java:395)
        at org.rhq.modules.plugins.jbossas7.DatasourceComponent.getValues(DatasourceComponent.java:206)
        ...

Expected results:
Metrics should be collected with no errors, warnings, or exceptions at debug level.

Additional info:
At the time of this error, the agent log -- at debug level -- contains the following debug, warn and error messages:

    DEBUG [WorkerThread#0[127.0.0.1:36143]] (rhq.core.pc.inventory.ResourceContainer$ResourceComponentInvocationHandler)- Call to [org.rhq.modules.plugins.jbossas7.DatasourceComponent.getValues()] with args [[org.rhq.core.domain.measurement.MeasurementReport@19af3bb9, [MeasurementScheduleRequest[scheduleId=11796, name=TotalBlockingTime, interval=2400000, enabled=true, dataType=MEASUREMENT, rawNumericType=null], MeasurementScheduleRequest[scheduleId=11800, name=max-pool-size, interval=14400000, enabled=true, dataType=MEASUREMENT, rawNumericType=null], MeasurementScheduleRequest[scheduleId=11789, name=MaxCreationTime, interval=2400000, enabled=true, dataType=MEASUREMENT, rawNumericType=null], MeasurementScheduleRequest[scheduleId=11783, name=AverageBlockingTime, interval=2400000, enabled=true, dataType=MEASUREMENT, rawNumericType=null], MeasurementScheduleRequest[scheduleId=11764, name=PreparedStatementCacheMissCount, interval=2400000, enabled=true, dataType=MEASUREMENT, rawNumericType=null], MeasurementScheduleRequest[scheduleId=11765, name=PreparedStatementCacheMissCount, interval=2400000, enabled=true, dataType=MEASUREMENT, rawNumericType=TRENDSUP], MeasurementScheduleRequest[scheduleId=11791, name=MaxWaitTime, interval=1200000, enabled=true, dataType=MEASUREMENT, rawNumericType=null], MeasurementScheduleRequest[scheduleId=11790, name=MaxUsedCount, interval=2400000, enabled=true, dataType=MEASUREMENT, rawNumericType=null], MeasurementScheduleRequest[scheduleId=11769, name=PreparedStatementCacheDeleteCount, interval=2400000, enabled=true, dataType=MEASUREMENT, rawNumericType=null], MeasurementScheduleRequest[scheduleId=11784, name=AverageCreationTime, interval=2400000, enabled=true, dataType=MEASUREMENT, rawNumericType=null], MeasurementScheduleRequest[scheduleId=11766, name=PreparedStatementCacheAddCount, interval=2400000, enabled=true, dataType=MEASUREMENT, rawNumericType=null], MeasurementScheduleRequest[scheduleId=11770, name=PreparedStatementCacheDeleteCount, interval=2400000, enabled=true, dataType=MEASUREMENT, rawNumericType=TRENDSUP], MeasurementScheduleRequest[scheduleId=11793, name=TimedOut, interval=1200000, enabled=true, dataType=MEASUREMENT, rawNumericType=TRENDSUP], MeasurementScheduleRequest[scheduleId=11792, name=TimedOut, interval=1200000, enabled=true, dataType=MEASUREMENT, rawNumericType=null], MeasurementScheduleRequest[scheduleId=11797, name=TotalBlockingTime, interval=2400000, enabled=true, dataType=MEASUREMENT, rawNumericType=TRENDSUP], MeasurementScheduleRequest[scheduleId=11795, name=TotalCreationTime, interval=2400000, enabled=true, dataType=MEASUREMENT, rawNumericType=TRENDSUP], MeasurementScheduleRequest[scheduleId=11786, name=CreatedCount, interval=2400000, enabled=true, dataType=MEASUREMENT, rawNumericType=TRENDSUP], MeasurementScheduleRequest[scheduleId=11763, name=PreparedStatementCacheCurrentSize, interval=2400000, enabled=true, dataType=MEASUREMENT, rawNumericType=null], MeasurementScheduleRequest[scheduleId=11782, name=AvailableCount, interval=2400000, enabled=true, dataType=MEASUREMENT, rawNumericType=null], MeasurementScheduleRequest[scheduleId=11785, name=CreatedCount, interval=2400000, enabled=true, dataType=MEASUREMENT, rawNumericType=null], MeasurementScheduleRequest[scheduleId=11794, name=TotalCreationTime, interval=2400000, enabled=true, dataType=MEASUREMENT, rawNumericType=null], MeasurementScheduleRequest[scheduleId=11799, name=min-pool-size, interval=14400000, enabled=true, dataType=MEASUREMENT, rawNumericType=null], MeasurementScheduleRequest[scheduleId=11768, name=PreparedStatementCacheAccessCount, interval=2400000, enabled=true, dataType=MEASUREMENT, rawNumericType=TRENDSUP], MeasurementScheduleRequest[scheduleId=11767, name=PreparedStatementCacheAccessCount, interval=2400000, enabled=true, dataType=MEASUREMENT, rawNumericType=null], MeasurementScheduleRequest[scheduleId=11772, name=PreparedStatementCacheHitCount, interval=2400000, enabled=true, dataType=MEASUREMENT, rawNumericType=TRENDSUP], MeasurementScheduleRequest[scheduleId=11771, name=PreparedStatementCacheHitCount, interval=2400000, enabled=true, dataType=MEASUREMENT, rawNumericType=null], MeasurementScheduleRequest[scheduleId=11787, name=DestroyedCount, interval=2400000, enabled=true, dataType=MEASUREMENT, rawNumericType=null], MeasurementScheduleRequest[scheduleId=11788, name=DestroyedCount, interval=2400000, enabled=true, dataType=MEASUREMENT, rawNumericType=TRENDSUP], MeasurementScheduleRequest[scheduleId=11707, name=ActiveCount, interval=2400000, enabled=true, dataType=MEASUREMENT, rawNumericType=null]]]] failed.
    java.util.concurrent.ExecutionException: java.lang.ClassCastException: java.util.LinkedHashMap cannot be cast to java.lang.Integer
        at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:262)
        at java.util.concurrent.FutureTask.get(FutureTask.java:119)
        at org.rhq.core.pc.inventory.ResourceContainer$ResourceComponentInvocationHandler.invokeInNewThreadWithLock(ResourceContainer.java:561)
        at org.rhq.core.pc.inventory.ResourceContainer$ResourceComponentInvocationHandler.invoke(ResourceContainer.java:549)
        at $Proxy65.getValues(Unknown Source)
        at org.rhq.core.pc.measurement.MeasurementManager.getRealTimeMeasurementValue(MeasurementManager.java:488)
        ...
    Caused by: java.lang.ClassCastException: java.util.LinkedHashMap cannot be cast to java.lang.Integer
        at org.rhq.modules.plugins.jbossas7.DatasourceComponent.getRCAsMetric(DatasourceComponent.java:395)
        at org.rhq.modules.plugins.jbossas7.DatasourceComponent.getValues(DatasourceComponent.java:206)
        at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:616)
        at org.rhq.core.pc.inventory.ResourceContainer$ComponentInvocation.call(ResourceContainer.java:654)
        ...

    ERROR [WorkerThread#0[127.0.0.1:36143]] (rhq.core.pc.measurement.MeasurementManager)- Could not get measurement values
    java.lang.ClassCastException: java.util.LinkedHashMap cannot be cast to java.lang.Integer
        at org.rhq.modules.plugins.jbossas7.DatasourceComponent.getRCAsMetric(DatasourceComponent.java:395)
        at org.rhq.modules.plugins.jbossas7.DatasourceComponent.getValues(DatasourceComponent.java:206)
        ...
    WARN  [MeasurementManager.collector-1] (rhq.core.pc.measurement.MeasurementCollectorRunner)- Failure to collect measurement data for Resource[id=10153, uuid=2c96de45-00f1-4310-91ac-1b9d7806254b, type={JBossAS7}DataSource (Standalone), key=subsystem=datasources,data-source=NoTxRHQDS, name=NoTxRHQDS, parent=datasources], requests=[ScheduledMeasurementInfo[res=10153, name=min-pool-size, sched=11799], ScheduledMeasurementInfo[res=10153, name=max-pool-size, sched=11800]], report.size()=3
    java.lang.ClassCastException: java.util.LinkedHashMap cannot be cast to java.lang.Integer
        at org.rhq.modules.plugins.jbossas7.DatasourceComponent.getRCAsMetric(DatasourceComponent.java:395)
        at org.rhq.modules.plugins.jbossas7.DatasourceComponent.getValues(DatasourceComponent.java:208)
        at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:616)
        at org.rhq.core.pc.inventory.ResourceContainer$ComponentInvocation.call(ResourceContainer.java:654)
        ...

Comment 1 Lukas Krejci 2014-04-22 12:22:04 UTC
Merged the PR into master

commit 4edaa3b2de2575f6fb875b428ec6db1b20afe0a9
Merge: 769bb64 e86ac44
Author: Lukas Krejci <metlos.github.com>
Date:   Tue Apr 22 14:17:41 2014 +0200

    Merge pull request #19 from lzoubek/bugs/1072543
    
    [BZ 1072543] Unable to collect metrics from EAP 6 datasources when a system properties is used in configuration

Comment 4 Simeon Pinder 2014-07-31 15:52:29 UTC
Moving to ON_QA as available to test with brew build of DR01: https://brewweb.devel.redhat.com//buildinfo?buildID=373993

Comment 5 Filip Brychta 2014-08-07 14:24:49 UTC
Original issue is fixed. But there is another severe problem related to min-pool-size or max-pool-size containing string value.
When you set min-pool-size or max-pool-size manually in standalone-*.xml to 
${rhq.server.datasource.nonxa.maxpool:5} the server is started correctly, but configuration of this resource in JON becomes invalid (string value for those properties is not expected) and any further attempt to update this configuration in JON fails with:
java.lang.NumberFormatException: For input string: "${rhq.server.datasource.nonxa.maxpool:5}"
	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
	at java.lang.Integer.parseInt(Integer.java:481)
	at java.lang.Integer.valueOf(Integer.java:582)
	at org.rhq.modules.plugins.jbossas7.ConfigurationWriteDelegate.getObjectWithType(ConfigurationWriteDelegate.java:704)
	at org.rhq.modules.plugins.jbossas7.ConfigurationWriteDelegate.preparePropertySimple(ConfigurationWriteDelegate.java:529)
	at org.rhq.modules.plugins.jbossas7.ConfigurationWriteDelegate.createWriteAttributePropertySimple(ConfigurationWriteDelegate.java:453)
	at org.rhq.modules.plugins.jbossas7.ConfigurationWriteDelegate.createWriteAttribute(ConfigurationWriteDelegate.java:332)
	at org.rhq.modules.plugins.jbossas7.ConfigurationWriteDelegate.updateProperty(ConfigurationWriteDelegate.java:324)
	at org.rhq.modules.plugins.jbossas7.ConfigurationWriteDelegate.updateGenerateOperationFromProperties(ConfigurationWriteDelegate.java:130)
	at org.rhq.modules.plugins.jbossas7.ConfigurationWriteDelegate.updateResourceConfiguration(ConfigurationWriteDelegate.java:103)
	at org.rhq.modules.plugins.jbossas7.DatasourceComponent.updateResourceConfiguration(DatasourceComponent.java:331)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)


Both properties must be set to integer and then the configuration update works.

Comment 6 Libor Zoubek 2014-08-07 14:35:20 UTC
This is different issue, please open another BZ

Comment 7 Filip Brychta 2014-08-07 15:09:28 UTC
Ok, I will open new one.
This is verified on
Version :	
3.3.0.DR01
Build Number :	
6468454:dda0a47