Bug 1195395

Summary: ETL service aggregation to hourly tables is failing due to NullPointerException for specific timezones due to the way the ETL interprets the timezone
Product: Red Hat Enterprise Virtualization Manager Reporter: Bimal Chollera <bcholler>
Component: ovirt-engine-dwhAssignee: Shirly Radco <sradco>
Status: CLOSED ERRATA QA Contact: Lukas Svaty <lsvaty>
Severity: high Docs Contact:
Priority: high    
Version: 3.5.0CC: audgiri, bcholler, iheim, lsurette, mkalinin, ngupta, pstehlik, rbalakri, Rhev-m-bugs, sradco, stephane.lebreton, wili, yeylon, ykaul, ylavi
Target Milestone: ovirt-3.6.0-rcKeywords: Reopened, ZStream
Target Release: 3.6.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: ovirt-3.6.0-beta2 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1228974 (view as bug list) Environment:
Last Closed: 2016-03-09 21:22:55 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Infra RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1164125    
Bug Blocks: 1222447, 1228974    

Description Bimal Chollera 2015-02-23 18:15:40 UTC 
Description of problem:

After upgrading to RHEVM 3.5, a number of end-users have reported getting the following errors in the event log:

"ETL service aggregation to hourly tables has encountered an error. Please consult the service log for more details."

Version-Release number of selected component (if applicable):


How reproducible:

By upgrading to RHEVM 3.5


Steps to Reproduce:
1.
2.
3.

Actual results:

ETL service failing due to NPE

Expected results:

Should be able to aggregate

Additional info:

Errors in `/var/log/ovirt-engine/ovirt-engine-dwhd.log`

~~
2015-02-18 12:05:38|ETL Service Started
Exception in component tJDBCInput_1
Exception in component tJDBCInput_10
Exception in component tJDBCInput_11
Exception in component tJDBCInput_13
Exception in component tJDBCInput_2
Exception in component tJDBCInput_5
Exception in component tJDBCInput_3
Exception in component tJDBCInput_4
Exception in component tJDBCInput_9
java.lang.NullPointerException
        at ovirt_engine_dwh.aggregationtohourly_3_5.AggregationToHourly.tJDBCInput_1Process(AggregationToHourly.java:2377)
        at ovirt_engine_dwh.aggregationtohourly_3_5.AggregationToHourly$2.run(AggregationToHourly.java:25665)
java.lang.NullPointerException
        at ovirt_engine_dwh.aggregationtohourly_3_5.AggregationToHourly.tJDBCInput_11Process(AggregationToHourly.java:15308)
        at ovirt_engine_dwh.aggregationtohourly_3_5.AggregationToHourly$7.run(AggregationToHourly.java:25895)
java.lang.NullPointerException
        at ovirt_engine_dwh.aggregationtohourly_3_5.AggregationToHourly.tJDBCInput_9Process(AggregationToHourly.java:20681)
        at ovirt_engine_dwh.aggregationtohourly_3_5.AggregationToHourly$9.run(AggregationToHourly.java:25987)
java.lang.NullPointerException
        at ovirt_engine_dwh.aggregationtohourly_3_5.AggregationToHourly.tJDBCInput_13Process(AggregationToHourly.java:18186)
        at ovirt_engine_dwh.aggregationtohourly_3_5.AggregationToHourly$8.run(AggregationToHourly.java:25941)
java.lang.NullPointerException
        at ovirt_engine_dwh.aggregationtohourly_3_5.AggregationToHourly.tJDBCInput_5Process(AggregationToHourly.java:13356)
        at ovirt_engine_dwh.aggregationtohourly_3_5.AggregationToHourly$6.run(AggregationToHourly.java:25849)
java.lang.NullPointerException
        at ovirt_engine_dwh.aggregationtohourly_3_5.AggregationToHourly.tJDBCInput_4Process(AggregationToHourly.java:11256)
        at ovirt_engine_dwh.aggregationtohourly_3_5.AggregationToHourly$5.run(AggregationToHourly.java:25803)
java.lang.NullPointerException
        at ovirt_engine_dwh.aggregationtohourly_3_5.AggregationToHourly.tJDBCInput_3Process(AggregationToHourly.java:8231)
        at ovirt_engine_dwh.aggregationtohourly_3_5.AggregationToHourly$4.run(AggregationToHourly.java:25757)
java.lang.NullPointerException
        at ovirt_engine_dwh.aggregationtohourly_3_5.AggregationToHourly.tJDBCInput_2Process(AggregationToHourly.java:5968)
        at ovirt_engine_dwh.aggregationtohourly_3_5.AggregationToHourly$3.run(AggregationToHourly.java:25711)
java.lang.NullPointerException
        at ovirt_engine_dwh.aggregationtohourly_3_5.AggregationToHourly.tJDBCInput_10Process(AggregationToHourly.java:24091)
        at ovirt_engine_dwh.aggregationtohourly_3_5.AggregationToHourly$10.run(AggregationToHourly.java:26033)
2015-02-18 14:00:00|IWpxRl|R75JMm|ujbv9q|OVIRT_ENGINE_DWH|AggregationToHourly|Default|6|Java Exception|tJDBCInput_2|java.lang.NullPointerException:null|1
2015-02-18 14:00:00|IWpxRl|R75JMm|ujbv9q|OVIRT_ENGINE_DWH|AggregationToHourly|Default|6|Java Exception|tJDBCInput_9|java.lang.NullPointerException:null|1
2015-02-18 14:00:00|IWpxRl|R75JMm|ujbv9q|OVIRT_ENGINE_DWH|AggregationToHourly|Default|6|Java Exception|tJDBCInput_10|java.lang.NullPointerException:null|1
Exception in component tJDBCOutput_6
org.postgresql.util.PSQLException: ERROR: transacción abortada, las órdenes serán ignoradas hasta el fin de bloque de transacción
        at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2094)
        at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1827)
        at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)
        at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:508)
        at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:384)
        at org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:330)
        at ovirt_engine_dwh.aggregationtohourly_3_5.AggregationToHourly.tRowGenerator_1Process(AggregationToHourly.java:19905)
        at ovirt_engine_dwh.aggregationtohourly_3_5.AggregationToHourly.tPostjob_1Process(AggregationToHourly.java:19496)
        at ovirt_engine_dwh.aggregationtohourly_3_5.AggregationToHourly.runJobInTOS(AggregationToHourly.java:26081)
        at ovirt_engine_dwh.aggregationtohourly_3_5.AggregationToHourly.runJob(AggregationToHourly.java:25464)
        at ovirt_engine_dwh.hourlytimekeepingjob_3_5.HourlyTimeKeepingJob.tRunJob_1Process(HourlyTimeKeepingJob.java:2541)
        at ovirt_engine_dwh.hourlytimekeepingjob_3_5.HourlyTimeKeepingJob.tJDBCInput_1Process(HourlyTimeKeepingJob.java:2369)
        at ovirt_engine_dwh.hourlytimekeepingjob_3_5.HourlyTimeKeepingJob.tJDBCConnection_1Process(HourlyTimeKeepingJob.java:1822)
        at ovirt_engine_dwh.hourlytimekeepingjob_3_5.HourlyTimeKeepingJob.tJDBCConnection_2Process(HourlyTimeKeepingJob.java:1694)
        at ovirt_engine_dwh.hourlytimekeepingjob_3_5.HourlyTimeKeepingJob$2.run(HourlyTimeKeepingJob.java:6371)
2015-02-18 14:00:00|IWpxRl|R75JMm|ujbv9q|OVIRT_ENGINE_DWH|AggregationToHourly|Default|6|Java Exception|tJDBCOutput_6|org.postgresql.util.PSQLException:ERROR: transacción abortada, las órdenes serán ignoradas hasta el fin de bloque de transacción|1
Exception in component tRunJob_1
java.lang.RuntimeException: Child job running failed
~~~
Comment 9 Shirly Radco 2015-02-25 11:26:16 UTC 
Please also provide pg dumps of the engine and history databases:
su -
su - postgres
/usr/bin/pg_dump -U "postgres" -F custom -f "<file>" "engine"
/usr/bin/pg_dump -U "postgres" -F custom -f "<file>" "ovirt_engine_history"
Comment 22 Shirly Radco 2015-03-01 11:50:24 UTC 

*** This bug has been marked as a duplicate of bug 1164125 ***
Comment 28 Yaniv Lavi 2015-08-20 07:14:24 UTC 
*** Bug 1222447 has been marked as a duplicate of this bug. ***
Comment 30 errata-xmlrpc 2016-03-09 21:22:55 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-0404.html