Bug 1284438 - engine-setup fails complaining that DWH is running but it's not running
engine-setup fails complaining that DWH is running but it's not running
Status: CLOSED DUPLICATE of bug 1286441
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine-setup (Show other bugs)
3.6.0
All Linux
unspecified Severity high
: ovirt-3.6.1
: ---
Assigned To: Yedidyah Bar David
Pavel Stehlik
integration
:
Depends On:
Blocks: RHEV_36_HTB
  Show dependency treegraph
 
Reported: 2015-11-23 05:43 EST by Julio Entrena Perez
Modified: 2016-05-23 07:03 EDT (History)
10 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-12-01 01:33:48 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Julio Entrena Perez 2015-11-23 05:43:07 EST
Description of problem:
engine-setup fails complaining that DWH is running but DWH is not running:

          Please confirm installation settings (OK, Cancel) [OK]: 
[ INFO  ] Cleaning async tasks and compensations
[ INFO  ] Unlocking existing entities
[ INFO  ] Checking the Engine database consistency
[ INFO  ] Stage: Transaction setup
[ INFO  ] Stopping dwh service
[ INFO  ] Stopping reports service
[ INFO  ] Stopping engine service
[ INFO  ] Stopping ovirt-fence-kdump-listener service
[ INFO  ] Stopping websocket-proxy service
[ ERROR ] dwhd is currently running. Its hostname is rhevm3-375.usersys.redhat.com. Please stop it before running Setup.
[ ERROR ] Failed to execute stage 'Transaction setup': dwhd is currently running
[ INFO  ] Yum Performing yum transaction rollback
[ INFO  ] Stage: Clean up
          Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20151123103341-rnhsv2.log
[ INFO  ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20151123103440-setup.conf'
[ INFO  ] Stage: Pre-termination
[ INFO  ] Stage: Termination
[ ERROR ] Execution of setup failed
[root@rhevm3-375 ~]# hostname
rhevm3-375.usersys.redhat.com
[root@rhevm3-375 ~]# ps -ef | grep dwh
root     23198 21548  0 10:34 pts/0    00:00:00 grep dwh
[root@rhevm3-375 ~]# /etc/init.d/ovirt-engine-dwhd status
ovirt-engine-dwhd is stopped
[root@rhevm3-375 ~]# rpm -q rhevm
rhevm-3.5.5-0.1.el6ev.noarch
[root@rhevm3-375 ~]# rpm -q rhevm-setup
rhevm-setup-3.6.0.3-0.1.el6.noarch

Version-Release number of selected component (if applicable):
rhevm-setup-3.6.0.3-0.1.el6

How reproducible:
Always

Steps to Reproduce:
1. Take a running 3.5.5 RHEV-M and subscribe it to the 3.6 beta channel
2. yum update rhevm-setup
3.

Actual results:
[ ERROR ] dwhd is currently running. Its hostname is rhevm3-375.usersys.redhat.com. Please stop it before running Setup.
[ ERROR ] Failed to execute stage 'Transaction setup': dwhd is currently running

Expected results:
Installer detects that DWH is not running and upgrade proceeds.

Additional info:
Comment 1 Julio Entrena Perez 2015-11-23 05:55:44 EST
In the setup log file:

2015-11-23 10:50:41 DEBUG otopi.plugins.otopi.services.rhel plugin.execute:936 execute-output: ('/sbin/service', 'ovirt-engine-dwhd', 'status') stdout:
ovirt-engine-dwhd is stopped

But later:

2015-11-23 10:50:42 DEBUG otopi.context context._executeMethod:142 Stage transaction-prepare METHOD otopi.plugins.ovirt_engine_setup.ovirt_engine_dwh.core.single_etl.Plugin._transactionBegin
2015-11-23 10:50:42 DEBUG otopi.ovirt_engine_setup.engine_common.database database.execute:171 Database: 'None', Statement: '
            select * from GetDwhHistoryTimekeepingByVarName(
                %(name)s
            )
        ', args: {'name': 'DwhCurrentlyRunning'}
2015-11-23 10:50:42 DEBUG otopi.ovirt_engine_setup.engine_common.database database.execute:176 Creating own connection
2015-11-23 10:50:42 DEBUG otopi.ovirt_engine_setup.engine_common.database database.execute:221 Result: [{'var_value': '1', 'var_datetime': None, 'var_name': 'DwhCurrentlyRunning'}]
2015-11-23 10:50:42 ERROR otopi.plugins.ovirt_engine_setup.ovirt_engine_dwh.core.single_etl single_etl._transactionBegin:136 dwhd is currently running.
Its hostname is rhevm3-375.usersys.**FILTERED**.com.
Please stop it before running Setup.
2015-11-23 10:50:42 DEBUG otopi.context context._executeMethod:156 method exception
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/otopi/context.py", line 146, in _executeMethod
    method['method']()
  File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine-dwh/core/single_etl.py", line 139, in _transactionBegin
    raise RuntimeError(_('dwhd is currently running'))
RuntimeError: dwhd is currently running
Comment 3 Julio Entrena Perez 2015-11-23 06:01:46 EST
Seems a timing issue: starting engine, starting dwh, stopping dwh and giving it a minute before re-running the installer seems to allow the installer to proceed.

When the installer is run for the first time dwh is running and the installer stops it, but it seems that there is not enough time for the 'DwhCurrentlyRunning' database value to be updated.

Shouldn't checking that the service is stopped by enough?
Comment 4 Yedidyah Bar David 2015-11-23 08:58:31 EST
(In reply to Julio Entrena Perez from comment #3)
> Seems a timing issue: starting engine, starting dwh, stopping dwh and giving
> it a minute before re-running the installer seems to allow the installer to
> proceed.
> 
> When the installer is run for the first time dwh is running and the
> installer stops it, but it seems that there is not enough time for the
> 'DwhCurrentlyRunning' database value to be updated.
> 
> Shouldn't checking that the service is stopped by enough?

IMO Yes. Shirly? Can we make sure that dwhd exits, that is - 'service ovirt-engine-dwhd stop' returns - only when all of its threads finished?

We have a similar bug 1075013 about the engine, probably didn't cause enough trouble yet.
Comment 5 Yedidyah Bar David 2015-11-23 09:01:13 EST
BTW, I am pretty certain this affects 3.5 too, also not sure it's a 3.6/beta blocker.
Comment 6 Shirly Radco 2015-11-23 09:23:11 EST
All threads and connections are closed when dwh exists.
Comment 7 Yedidyah Bar David 2015-11-29 10:19:04 EST
Now got this error on a different flow and filed bug 1286441.
Comment 10 Yedidyah Bar David 2015-12-01 01:33:48 EST
Following a private discussion, closing as duplicate of bug 1286441. Please reopen if relevant. Thanks.

*** This bug has been marked as a duplicate of bug 1286441 ***

Note You need to log in before you can comment on or make changes to this bug.