Bug 1293844 - [engine-setup] [ ERROR ] dwhd is currently running. Its hostname is example.com. Please stop it before running Setup.
Summary: [engine-setup] [ ERROR ] dwhd is currently running. Its hostname is example.c...
Keywords:
Status: CLOSED DUPLICATE of bug 1286441
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: Setup.Engine
Version: 3.6.1.3
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ovirt-3.6.3
: ---
Assignee: Yedidyah Bar David
QA Contact: Pavel Stehlik
URL:
Whiteboard: integration
Depends On:
Blocks: RHEV3.6Upgrade
TreeView+ depends on / blocked
 
Reported: 2015-12-23 09:24 UTC by Jiri Belka
Modified: 2016-01-12 06:24 UTC (History)
9 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2016-01-12 06:24:26 UTC
oVirt Team: Integration
Embargoed:
gklein: ovirt-3.6.z?
rule-engine: planning_ack?
rule-engine: devel_ack?
rule-engine: testing_ack?


Attachments (Terms of Use)

Description Jiri Belka 2015-12-23 09:24:04 UTC
Description of problem:

If engine-setup fails after it has stopped dwhd or reports and you run engine-setup again to continue upgrade, it fails because it tries to stop already non-running dwhd/reports. IMO the check should be corrected to not stop already stopped daemons and then proceed the setup. Current solution is to start again dhwd/reports so engine-setup can stop them couple of sec later - thus annoying.

The flow was:

~~~
         --== DATABASE CONFIGURATION ==--

[ ERROR ] Failed to execute stage 'Environment customization': Reports requires max_connections to be at least 150. Please fix max_connections before you continue.
[ INFO  ] Stage: Clean up
          Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20151222124149-r5l7xp.log
          [ INFO  ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20151222124229-setup.conf'
          [ INFO  ] Stage: Pre-termination
          [ INFO  ] Stage: Termination
~~~

Correction...

perl -i.orig -pe 's/^(max_connections).*/$1 = 150/;' /var/lib/pgsql/data/postgresql.conf
/etc/init.d/postgresql restart

~~~
engine-setup
...
[ INFO  ] Stopping dwh service
[ INFO  ] Stopping reports service
[ INFO  ] Stopping engine service
[ INFO  ] Stopping ovirt-fence-kdump-listener service
[ INFO  ] Stopping websocket-proxy service
[ ERROR ] dwhd is currently running. Its hostname is example.com. Please stop it before running Setup.
[ ERROR ] Failed to execute stage 'Transaction setup': dwhd is currently running
[ INFO  ] Yum Performing yum transaction rollback
[ INFO  ] Stage: Clean up
          Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20151222125113-x8id6n.log
          [ INFO  ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20151222125225-setup.conf'
          [ INFO  ] Stage: Pre-termination
          [ INFO  ] Stage: Termination
~~~

Version-Release number of selected component (if applicable):
rhevm-setup-3.6.1.3-0.1.el6.noarch

How reproducible:
100%

Steps to Reproduce:
1. install engine, dwh (i was doing upgrade from 3.5.x)
2. customize env so next engine-setup would fail ('max_connections' for postgresql seems like good candicate for now)
3. engine-setup

Actual results:
engine-setup fails because it tries to stop already non-running daemons (dhwd and probably reports too)

Expected results:
engine-setup should check if daemon is running, if not then continue; thus it should not stop what is not running and then fail

Additional info:

Comment 1 Yedidyah Bar David 2015-12-23 09:47:30 UTC
(In reply to Jiri Belka from comment #0)
> Description of problem:
> 
> If engine-setup fails after it has stopped dwhd or reports and you run
> engine-setup again to continue upgrade, it fails because it tries to stop
> already non-running dwhd/reports.

It tries to stop dwhd if the flag in the db says it's up. It does not check its status nor change this flag when stopping it. dwhd should reset this flag by itself when it stops.

Reports is not really related IIUC.

Are you sure you are not affected by bug 1286441 or something similar?

> IMO the check should be corrected to not
> stop already stopped daemons and then proceed the setup. Current solution is
> to start again dhwd/reports so engine-setup can stop them couple of sec
> later - thus annoying.
> 
> The flow was:
> 

Please retry this flow, but add as a first step:

restart dwhd

If it still reproduces, please attach more relevant logs, including dwhd and pg.

> ~~~
>          --== DATABASE CONFIGURATION ==--
> 
> [ ERROR ] Failed to execute stage 'Environment customization': Reports
> requires max_connections to be at least 150. Please fix max_connections
> before you continue.
> [ INFO  ] Stage: Clean up
>           Log file is located at
> /var/log/ovirt-engine/setup/ovirt-engine-setup-20151222124149-r5l7xp.log
>           [ INFO  ] Generating answer file
> '/var/lib/ovirt-engine/setup/answers/20151222124229-setup.conf'
>           [ INFO  ] Stage: Pre-termination
>           [ INFO  ] Stage: Termination
> ~~~
> 
> Correction...
> 
> perl -i.orig -pe 's/^(max_connections).*/$1 = 150/;'
> /var/lib/pgsql/data/postgresql.conf
> /etc/init.d/postgresql restart
> 
> ~~~
> engine-setup
> ...
> [ INFO  ] Stopping dwh service
> [ INFO  ] Stopping reports service
> [ INFO  ] Stopping engine service
> [ INFO  ] Stopping ovirt-fence-kdump-listener service
> [ INFO  ] Stopping websocket-proxy service
> [ ERROR ] dwhd is currently running. Its hostname is example.com. Please
> stop it before running Setup.
> [ ERROR ] Failed to execute stage 'Transaction setup': dwhd is currently
> running
> [ INFO  ] Yum Performing yum transaction rollback
> [ INFO  ] Stage: Clean up
>           Log file is located at
> /var/log/ovirt-engine/setup/ovirt-engine-setup-20151222125113-x8id6n.log
>           [ INFO  ] Generating answer file
> '/var/lib/ovirt-engine/setup/answers/20151222125225-setup.conf'
>           [ INFO  ] Stage: Pre-termination
>           [ INFO  ] Stage: Termination
> ~~~
> 
> Version-Release number of selected component (if applicable):
> rhevm-setup-3.6.1.3-0.1.el6.noarch
> 
> How reproducible:
> 100%
> 
> Steps to Reproduce:
> 1. install engine, dwh (i was doing upgrade from 3.5.x)
> 2. customize env so next engine-setup would fail ('max_connections' for
> postgresql seems like good candicate for now)
> 3. engine-setup
> 
> Actual results:
> engine-setup fails because it tries to stop already non-running daemons
> (dhwd and probably reports too)
> 
> Expected results:
> engine-setup should check if daemon is running, if not then continue; thus
> it should not stop what is not running and then fail

It can't check if dwhd is running if it is set up on a different machine.

Therefore it has to rely on this flag.

> 
> Additional info:

Comment 2 Jiri Belka 2015-12-23 15:17:06 UTC
(In reply to Yedidyah Bar David from comment #1)
> (In reply to Jiri Belka from comment #0)
> > Description of problem:
> > 
> > If engine-setup fails after it has stopped dwhd or reports and you run
> > engine-setup again to continue upgrade, it fails because it tries to stop
> > already non-running dwhd/reports.
> 
> It tries to stop dwhd if the flag in the db says it's up. It does not check
> its status nor change this flag when stopping it. dwhd should reset this
> flag by itself when it stops.
> 
> Reports is not really related IIUC.
> 
> Are you sure you are not affected by bug 1286441 or something similar?

The flow I did is described in original descr. I could re-try on another env but just new year as BRQ shutdown is starting today.

Comment 3 Yedidyah Bar David 2015-12-23 15:26:00 UTC
(In reply to Jiri Belka from comment #2)
> 
> The flow I did is described in original descr.

Obviously. I didn't say it isn't.

But if prepending to it a single command solves the current bug, it means there are two different flows to consider. And if the "other" flow is solved by another bug, this one can be closed as duplicate of it.

Specifically, the referenced bug will affect if, e.g.:
0. restart pg
1. Continue with original desc

But you didn't mention if you restarted pg (perhaps unknowingly).

If we have a different flow, that's a different bug.

> I could re-try on another env
> but just new year as BRQ shutdown is starting today.

OK, please do. Thanks.

Comment 4 Jiri Belka 2016-01-07 09:49:12 UTC
> But you didn't mention if you restarted pg (perhaps unknowingly).

I did as i was changing max_connections.

I'm going to re-test it now.

Comment 5 Jiri Belka 2016-01-11 15:33:04 UTC
I can confirm that after engine-setup fails complaining about 'max_connections' to low and after DB settings is adjusted, DB restarted and DWH restarted as well, next run of engine-setup works OK.

But DWH restart is not really obvious.

Comment 6 Yedidyah Bar David 2016-01-12 06:24:26 UTC
(In reply to Jiri Belka from comment #5)
> I can confirm that after engine-setup fails complaining about
> 'max_connections' to low and after DB settings is adjusted, DB restarted and
> DWH restarted as well, next run of engine-setup works OK.

OK.

> 
> But DWH restart is not really obvious.

Indeed, and with bug 1286441 fixed that's not needed - just waiting for a minute. You might claim that's not obvious either, please discuss this there if you want. Not sure what can be done about that.

Closing as duplicate.

*** This bug has been marked as a duplicate of bug 1286441 ***


Note You need to log in before you can comment on or make changes to this bug.