Bug 574045 - numeric overflow preventing data compression
Summary: numeric overflow preventing data compression
Status: CLOSED CURRENTRELEASE
Alias: None
Product: RHQ Project
Classification: Other
Component: Database   
(Show other bugs)
Version: 1.3.1
Hardware: All
OS: Linux
high
high vote
Target Milestone: ---
: ---
Assignee: Joseph Marques
QA Contact: Heiko W. Rupp
URL:
Whiteboard:
Keywords:
Depends On:
Blocks: 574133 jon-sprint11-bugs
TreeView+ depends on / blocked
 
Reported: 2010-03-16 13:28 UTC by Simeon Pinder
Modified: 2018-10-27 16:13 UTC (History)
1 user (show)

Fixed In Version: 2.4
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2010-08-12 16:55:00 UTC
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

Description Simeon Pinder 2010-03-16 13:28:27 UTC
Description of problem:
Problematic metric values prevent data compression.

Actual Value:
Diskc Queue: Min .049, max .0491, last = infinity

and using TO_CHAR and the default format yields '~' as the value.
> using TO_CHAR(value, '99999999999.....999') just yields ###.....##### as the value.


Resulting Exception:
2010-03-11 16:00:00,567 INFO [org.rhq.enterprise.server.measurement.Measurement
CompressionManagerBean] Begin compression from [RHQ_MEAS_DATA_NUM_R04] to [RHQ_M
EASUREMENT_DATA_NUM_1H]
2010-03-11 16:00:00,568 INFO [org.rhq.enterprise.server.measurement.Measurement
CompressionManagerBean] Begin compressing data from table [RHQ_MEAS_DATA_NUM_R04
] to table [RHQ_MEASUREMENT_DATA_NUM_1H] between [3/11/10 3:00:00 PM] and [3/11/
10 4:00:00 PM]
2010-03-11 16:00:04,275 ERROR [org.rhq.enterprise.server.measurement.Measurement
CompressionManagerBean] Unable to compress data from [RHQ_MEAS_DATA_NUM_R04] to
[RHQ_MEASUREMENT_DATA_NUM_1H] at 3/11/10 3:00:00 PM: java.sql.SQLException:ORA-0
1426: numeric overflow
[SQLException=ORA-01426: numeric overflow
]
2010-03-11 16:00:04,275 INFO [org.rhq.enterprise.server.measurement.Measurement
CompressionManagerBean] Finished compression from [RHQ_MEAS_DATA_NUM_R04] to [RH
Q_MEASUREMENT_DATA_NUM_1H], [0] compressed rows

Version-Release number of selected component (if applicable):
1.3.1

How reproducible:
Every time.

Steps to Reproduce:
1.
2.
3.
  
Actual results:
Exceptions.

Expected results:
Successful compression.

Additional info:

Comment 3 Charles Crouch 2010-04-26 23:09:05 UTC
So the workaround is to order the rows in the raw metric table by value descending then delete the bogus one.
The impact of this issue is that the compression for an entire hours worth of raw table data will fail and rollback given just one bad row. Bad rows spread across several hours can cause broader compression failure.

Comment 5 Joseph Marques 2010-06-21 17:57:30 UTC
commit 349ea8691296052a4012e5fd32979cfb04a07492
Author: Joseph Marques <joseph@redhat.com>
Date:   Mon Jun 21 13:53:15 2010 -0400

    BZ-574045: prevent "bad" numeric values making their way to the server
    
    * without this fix, bad values will bomb the metric compression routines
    * log a warning and identifying information when metrics must be dropped

Comment 6 Corey Welton 2010-06-29 17:44:56 UTC
Needs Repro Steps

Comment 7 Heiko W. Rupp 2010-07-16 15:38:22 UTC
Actually I think this issue is not yet really fixed, as Double.MAX_VALUE is giving 
errors like

16:32:29,896 WARN  [MeasurementDataManagerBean] Failure saving measurement numeric data:
java.sql.BatchUpdateException:Interner Fehler: Overflow Exception trying to bind 6.629794669917522E307[SQLException=Interner Fehler: Overflow Exception trying to bind 6.629794669917522E307 -> Interner Fehler: Overflow Exception trying to bind 6.629794669917522E307(error-code=17001,sql-state=99999)]

value column of the measurement data tables is a 
VALUE	FLOAT	15 
which has a range from -1.79e308 to 1.79e308 
Java DOuble type has -4.9E-324 to 1.7976931348623157E308

The interesting question here is Statement.setDouble() which may only accept
data in the range of NUMBER :
The NUMBER datatype stores zero as well as positive and negative fixed numbers with absolute values from 1.0 x 10^-130 to (but not including) 1.0 x 10^126.

 Overflow Exception trying to bind 1.1376750872916126E137


Having said this, filtering of NaN and Infinite is working and probably enough here.

Comment 8 Corey Welton 2010-08-12 16:55:00 UTC
Mass-closure of verified bugs against JON.


Note You need to log in before you can comment on or make changes to this bug.