Bug 1170501

Summary:	If connection to DB fails , the job that checks DisconnectDwh flag does not reconnect to engine db
Product:	[oVirt] ovirt-engine-dwh	Reporter:	Shirly Radco <sradco>
Component:	General	Assignee:	Shirly Radco <sradco>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Lukas Svaty <lsvaty>
Severity:	high	Docs Contact:
Priority:	high
Version:	---	CC:	bugs, ecohen, gklein, lsurette, lsvaty, rbalakri, sradco, yeylon, ylavi
Target Milestone:	ovirt-3.6.1	Flags:	rule-engine: ovirt-3.6.z+ ylavi: planning_ack+ ylavi: devel_ack+ pnovotny: testing_ack+
Target Release:	3.6.1
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:	dwh
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:
Clones:	1170631 1181616 (view as bug list)		Environment:
Last Closed:	2016-01-13 14:39:25 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	Metrics	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1170631, 1181616

Description Shirly Radco 2014-12-04 07:37:22 UTC

Description of problem:
If connection to DB fails , the job that checks "DisconnectDwh" flag does not reconnect to engine db.

Version-Release number of selected component (if applicable):
3.5

How reproducible:
100%

Steps to Reproduce:
1. Have engine, dwh running
2. Restart postgresql
3. Change value of "DisconnectDwh" to 1

Actual results:
The ETL does not stop

Expected results:
ETL should stop

Additional info:

Comment 1 Lukas Svaty 2015-12-10 13:28:04 UTC

package rhevm-dwh-3.6.1-1.el6ev.noarch

ovirt-engine-dwhd.log:
2015-12-10 14:24:10|ETL Service Started

# [root@test ~]# service postgresql restart
Stopping postgresql service:                               [  OK  ]
Starting postgresql service:                               [  OK  ]

in database:
engine=# update vdc_options set option_value = 1 where option_name = 'DisconnectDwh';
FATAL:  terminating connection due to administrator command
server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.
The connection to the server was lost. Attempting reset: Succeeded.


try again :)
engine=# update vdc_options set option_value = 1 where option_name = 'DisconnectDwh';
UPDATE 1

ovirt-engine-dwhd.log:
ETL Service will now stop because a flag was raised to disconnect DWH on Engine DB
Exception in component tJDBCRollback_1
org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command
	at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2094)
	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1827)
	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)
	at org.postgresql.jdbc2.AbstractJdbc2Connection.executeTransactionCommand(AbstractJdbc2Connection.java:669)
	at org.postgresql.jdbc2.AbstractJdbc2Connection.rollback(AbstractJdbc2Connection.java:706)
	at ovirt_engine_dwh.historyetl_3_6.HistoryETL.tJDBCRollback_1Process(HistoryETL.java:1820)
	at ovirt_engine_dwh.historyetl_3_6.HistoryETL.tPostjob_1Process(HistoryETL.java:1730)
	at ovirt_engine_dwh.historyetl_3_6.HistoryETL.runJobInTOS(HistoryETL.java:11073)
	at ovirt_engine_dwh.historyetl_3_6.HistoryETL.main(HistoryETL.java:10494)
2015-12-10 14:24:51|zifWGs|zifWGs|zifWGs|OVIRT_ENGINE_DWH|HistoryETL|Default|6|Java Exception|tJDBCRollback_1|org.postgresql.util.PSQLException:FATAL: terminating connection due to administrator command|1

Comment 2 Red Hat Bugzilla Rules Engine 2015-12-11 02:36:56 UTC

Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.

Comment 3 Shirly Radco 2015-12-13 11:12:01 UTC

According to the log it seems that ovirt-engine-dwhd is stopped.
What is the problem?

service ovirt-engine-dwhd status is runnig?

Comment 4 Lukas Svaty 2015-12-14 07:09:43 UTC

DWHd was stopped, however I would imagine that on successful workflow logs should not contain Tracebacks. Please catch the above exception.

Comment 5 Red Hat Bugzilla Rules Engine 2015-12-17 09:15:31 UTC

This bug is not marked for z-stream, yet the milestone is for a z-stream version, therefore the milestone has been reset.
Please set the correct milestone or add the z-stream flag.

Comment 6 Red Hat Bugzilla Rules Engine 2015-12-17 09:15:31 UTC

Fixed bug tickets must have target milestone set prior to fixing them. Please set the correct milestone and move the bugs back to the previous status after this is corrected.

Comment 7 Red Hat Bugzilla Rules Engine 2015-12-17 09:15:31 UTC

Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.

Comment 8 Red Hat Bugzilla Rules Engine 2015-12-17 09:19:04 UTC

Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.

Comment 9 Lukas Svaty 2016-01-07 16:13:18 UTC

verified in rhevm-reports-3.6.2.1-1.el6ev.noarch

Comment 10 Sandro Bonazzola 2016-01-13 14:39:25 UTC

oVirt 3.6.1 has been released, closing current release