Bug 1284438 - engine-setup fails complaining that DWH is running but it's not running
Summary: engine-setup fails complaining that DWH is running but it's not running
Keywords:
Status: CLOSED DUPLICATE of bug 1286441
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine-setup
Version: 3.6.0
Hardware: All
OS: Linux
unspecified
high
Target Milestone: ovirt-3.6.1
: ---
Assignee: Yedidyah Bar David
QA Contact: Pavel Stehlik
URL:
Whiteboard: integration
Depends On:
Blocks: RHEV_36_HTB
TreeView+ depends on / blocked
 
Reported: 2015-11-23 10:43 UTC by Julio Entrena Perez
Modified: 2016-05-23 11:03 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-12-01 06:33:48 UTC
oVirt Team: ---
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1286441 0 medium CLOSED If the dwhd lost the original connection to the db it will not update the audit_log and dwh_history_timekeeping when dwh... 2021-02-22 00:41:40 UTC

Internal Links: 1286441

Description Julio Entrena Perez 2015-11-23 10:43:07 UTC
Description of problem:
engine-setup fails complaining that DWH is running but DWH is not running:

          Please confirm installation settings (OK, Cancel) [OK]: 
[ INFO  ] Cleaning async tasks and compensations
[ INFO  ] Unlocking existing entities
[ INFO  ] Checking the Engine database consistency
[ INFO  ] Stage: Transaction setup
[ INFO  ] Stopping dwh service
[ INFO  ] Stopping reports service
[ INFO  ] Stopping engine service
[ INFO  ] Stopping ovirt-fence-kdump-listener service
[ INFO  ] Stopping websocket-proxy service
[ ERROR ] dwhd is currently running. Its hostname is rhevm3-375.usersys.redhat.com. Please stop it before running Setup.
[ ERROR ] Failed to execute stage 'Transaction setup': dwhd is currently running
[ INFO  ] Yum Performing yum transaction rollback
[ INFO  ] Stage: Clean up
          Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20151123103341-rnhsv2.log
[ INFO  ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20151123103440-setup.conf'
[ INFO  ] Stage: Pre-termination
[ INFO  ] Stage: Termination
[ ERROR ] Execution of setup failed
[root@rhevm3-375 ~]# hostname
rhevm3-375.usersys.redhat.com
[root@rhevm3-375 ~]# ps -ef | grep dwh
root     23198 21548  0 10:34 pts/0    00:00:00 grep dwh
[root@rhevm3-375 ~]# /etc/init.d/ovirt-engine-dwhd status
ovirt-engine-dwhd is stopped
[root@rhevm3-375 ~]# rpm -q rhevm
rhevm-3.5.5-0.1.el6ev.noarch
[root@rhevm3-375 ~]# rpm -q rhevm-setup
rhevm-setup-3.6.0.3-0.1.el6.noarch

Version-Release number of selected component (if applicable):
rhevm-setup-3.6.0.3-0.1.el6

How reproducible:
Always

Steps to Reproduce:
1. Take a running 3.5.5 RHEV-M and subscribe it to the 3.6 beta channel
2. yum update rhevm-setup
3.

Actual results:
[ ERROR ] dwhd is currently running. Its hostname is rhevm3-375.usersys.redhat.com. Please stop it before running Setup.
[ ERROR ] Failed to execute stage 'Transaction setup': dwhd is currently running

Expected results:
Installer detects that DWH is not running and upgrade proceeds.

Additional info:

Comment 1 Julio Entrena Perez 2015-11-23 10:55:44 UTC
In the setup log file:

2015-11-23 10:50:41 DEBUG otopi.plugins.otopi.services.rhel plugin.execute:936 execute-output: ('/sbin/service', 'ovirt-engine-dwhd', 'status') stdout:
ovirt-engine-dwhd is stopped

But later:

2015-11-23 10:50:42 DEBUG otopi.context context._executeMethod:142 Stage transaction-prepare METHOD otopi.plugins.ovirt_engine_setup.ovirt_engine_dwh.core.single_etl.Plugin._transactionBegin
2015-11-23 10:50:42 DEBUG otopi.ovirt_engine_setup.engine_common.database database.execute:171 Database: 'None', Statement: '
            select * from GetDwhHistoryTimekeepingByVarName(
                %(name)s
            )
        ', args: {'name': 'DwhCurrentlyRunning'}
2015-11-23 10:50:42 DEBUG otopi.ovirt_engine_setup.engine_common.database database.execute:176 Creating own connection
2015-11-23 10:50:42 DEBUG otopi.ovirt_engine_setup.engine_common.database database.execute:221 Result: [{'var_value': '1', 'var_datetime': None, 'var_name': 'DwhCurrentlyRunning'}]
2015-11-23 10:50:42 ERROR otopi.plugins.ovirt_engine_setup.ovirt_engine_dwh.core.single_etl single_etl._transactionBegin:136 dwhd is currently running.
Its hostname is rhevm3-375.usersys.**FILTERED**.com.
Please stop it before running Setup.
2015-11-23 10:50:42 DEBUG otopi.context context._executeMethod:156 method exception
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/otopi/context.py", line 146, in _executeMethod
    method['method']()
  File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine-dwh/core/single_etl.py", line 139, in _transactionBegin
    raise RuntimeError(_('dwhd is currently running'))
RuntimeError: dwhd is currently running

Comment 3 Julio Entrena Perez 2015-11-23 11:01:46 UTC
Seems a timing issue: starting engine, starting dwh, stopping dwh and giving it a minute before re-running the installer seems to allow the installer to proceed.

When the installer is run for the first time dwh is running and the installer stops it, but it seems that there is not enough time for the 'DwhCurrentlyRunning' database value to be updated.

Shouldn't checking that the service is stopped by enough?

Comment 4 Yedidyah Bar David 2015-11-23 13:58:31 UTC
(In reply to Julio Entrena Perez from comment #3)
> Seems a timing issue: starting engine, starting dwh, stopping dwh and giving
> it a minute before re-running the installer seems to allow the installer to
> proceed.
> 
> When the installer is run for the first time dwh is running and the
> installer stops it, but it seems that there is not enough time for the
> 'DwhCurrentlyRunning' database value to be updated.
> 
> Shouldn't checking that the service is stopped by enough?

IMO Yes. Shirly? Can we make sure that dwhd exits, that is - 'service ovirt-engine-dwhd stop' returns - only when all of its threads finished?

We have a similar bug 1075013 about the engine, probably didn't cause enough trouble yet.

Comment 5 Yedidyah Bar David 2015-11-23 14:01:13 UTC
BTW, I am pretty certain this affects 3.5 too, also not sure it's a 3.6/beta blocker.

Comment 6 Shirly Radco 2015-11-23 14:23:11 UTC
All threads and connections are closed when dwh exists.

Comment 7 Yedidyah Bar David 2015-11-29 15:19:04 UTC
Now got this error on a different flow and filed bug 1286441.

Comment 10 Yedidyah Bar David 2015-12-01 06:33:48 UTC
Following a private discussion, closing as duplicate of bug 1286441. Please reopen if relevant. Thanks.

*** This bug has been marked as a duplicate of bug 1286441 ***


Note You need to log in before you can comment on or make changes to this bug.