Bug 782176 - [engine-cleanup] engine-cleanup should work even when there are active connections to DB
Summary: [engine-cleanup] engine-cleanup should work even when there are active connec...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: oVirt
Classification: Retired
Component: ovirt-engine-installer
Version: unspecified
Hardware: x86_64
OS: Linux
high
medium
Target Milestone: ---
: 3.2
Assignee: Alex Lourie
QA Contact:
URL:
Whiteboard: integration
: 773723 886019 (view as bug list)
Depends On:
Blocks: 760640 888027
TreeView+ depends on / blocked
 
Reported: 2012-01-16 18:35 UTC by zvi
Modified: 2013-07-25 20:46 UTC (History)
10 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2013-07-25 20:46:21 UTC
oVirt Team: ---
Embargoed:


Attachments (Terms of Use)

Description zvi 2012-01-16 18:35:39 UTC
Description of problem:
An engine-cleanup fails when there are active connections to DB (pgADMIN)  


How reproducible:
Always

Steps to Reproduce:
1.Clean engine DB via "engine-cleanup" when there is at least one connection to the DB (pgADMIN)

2.
3.
  
Actual results:
engine-cleanup fails:

2012-01-16 16:46:47::DEBUG::common_utils::205::root:: stderr = dropdb: database removal failed: ERROR:  database "engine" is being accessed by other users

 DETAIL:  There are 1 other session(s) using the database.

 2012-01-16 16:46:47::DEBUG::common_utils::206::root:: retcode = 1

 2012-01-16 16:46:47::ERROR::engine-cleanup::347::root:: Traceback (most recent call last):

  File "/usr/bin/engine-cleanup", line 341, in runFunc

   funcName()

   File "/usr/bin/engine-cleanup", line 282, in drop

    output, rc = utils.execCmd(cmd, None, True, MSG_ERROR_DROP_DB, [])

   File "/usr/share/ovirt-engine/scripts/common_utils.py", line 209, in execCmd

     raise Exception(msg)

 Exception: Error: Database drop failed


Expected results:
engine-cleanup should success  

Additional info:
A -F flag can be added to engine-cleanup - to force disconnection of all active db connections

Comment 1 Alex Lourie 2012-02-07 10:11:12 UTC
*** Bug 773723 has been marked as a duplicate of this bug. ***

Comment 3 Ofer Schreiber 2012-05-08 09:37:33 UTC
This one was partially solved:
1. If the DB is local - services should be restarted
2. If the DB is remote - there's nothing we can do.

Comment 4 Itamar Heim 2012-08-09 08:05:19 UTC
closing ON_QA bugs as oVirt 3.1 was released:
http://www.ovirt.org/get-ovirt/

Comment 5 Stephen Gordon 2012-12-19 17:13:15 UTC
I'm re-opening this as I did an installation using the 3.2 alpha bits and selected local DB but when I tried to clean-up (my install failed on iptables setup after creating the database because well iptables doesn't exist in F18) I got this output:

# engine-cleanup 
WARNING: Executing oVirt Engine cleanup utility.
This utility will wipe all existing data including configuration settings, certificates and database.
In addition, all existing DB connections will be closed.
Would you like to proceed? (yes|no): yes

Stopping ovirt-engine service...                         [ DONE ]
Removing Database...                                    [ ERROR ]
Removing CA...                                           [ DONE ]
Stopping engine-notifierd service...                     [ DONE ]

Cleanup finished with errors, please see log file
Error: failed to clear active DB connections
Cleanup log available at /var/log/ovirt-engine/engine-cleanup_2012_12_19_12_05_10.log

The cleanup log shows the cause of failure:

# tail /var/log/ovirt-engine/engine-cleanup_2012_12_19_12_05_10.log
2012-12-19 12:05:12::DEBUG::engine-cleanup::261::root:: CA Remove completed successfully
2012-12-19 12:05:12::DEBUG::engine-cleanup::389::root:: stoping engine-notifierd service.
2012-12-19 12:05:12::DEBUG::common_utils::1163::root:: stopping engine-notifierd
2012-12-19 12:05:12::DEBUG::common_utils::1200::root:: executing action engine-notifierd on service stop
2012-12-19 12:05:12::DEBUG::common_utils::390::root:: Executing command --> '/sbin/service engine-notifierd stop'
2012-12-19 12:05:12::DEBUG::common_utils::428::root:: output = Stopping engine-notifierd (via systemctl):  [  OK  ]

2012-12-19 12:05:12::DEBUG::common_utils::429::root:: stderr = 
2012-12-19 12:05:12::DEBUG::common_utils::430::root:: retcode = 0
2012-12-19 12:05:12::INFO::engine-cleanup::414::root:: Error: failed to clear active DB connections

Attempting to re-run engine-setup at this point results in:

Creating Database...                                  [ ERROR ]
Database creation failed
Please check log file /var/log/ovirt-engine/engine-setup_2012_12_19_12_08_02.log for more information
[root@sgordon-ovirt-engineusersysredhatcom ~]# tail /var/log/ovirt-engine/engine-setup_2012_12_19_12_08_02.log
  File "/usr/share/ovirt-engine/scripts/setup_sequences.py", line 154, in run
    step.run()
  File "/usr/share/ovirt-engine/scripts/setup_sequences.py", line 60, in run
    function()
  File "/usr/bin/engine-setup", line 969, in _createDB
    output, rc = utils.execCmd(cmdList=cmd, failOnError=True, msg=output_messages.ERR_DB_CREATE_FAILED, maskList=masked_value_set)
  File "/usr/share/ovirt-engine/scripts/common_utils.py", line 433, in execCmd
    raise Exception(msg)
Exception: Database creation failed

So even though this is a local database, the active connections weren't cleared and as a result the database wasn't removed.

Comment 6 Stephen Gordon 2012-12-19 17:15:51 UTC
Oh in addition at this point re-running engine-cleanup results in this:

# engine-cleanup 
WARNING: Executing oVirt Engine cleanup utility.
This utility will wipe all existing data including configuration settings, certificates and database.
In addition, all existing DB connections will be closed.
Would you like to proceed? (yes|no): yes

Stopping ovirt-engine service...                         [ DONE ]

Error: Couldn't connect to the database server.Check that connection is working and rerun the cleanup utility
Error: Cleanup failed.
please check log at /var/log/ovirt-engine/engine-cleanup_2012_12_19_12_13_48.log

# tail /var/log/ovirt-engine/engine-cleanup_2012_12_19_12_15_09.log
2012-12-19 12:15:23::DEBUG::common_utils::430::root:: retcode = 2
2012-12-19 12:15:23::ERROR::engine-cleanup::476::root:: Traceback (most recent call last):
  File "/usr/bin/engine-cleanup", line 471, in <module>
    main(options)
  File "/usr/bin/engine-cleanup", line 433, in main
    if db.exists() and options.drop_db:
  File "/usr/bin/engine-cleanup", line 366, in exists
    raise Exception(MSG_ERROR_CONNECT_DB)
Exception: Error: Couldn't connect to the database server.Check that connection is working and rerun the cleanup utility

Comment 7 Alex Lourie 2012-12-20 09:19:40 UTC
Stephen

Could you please attach the complete log file? It would help me to understand the problem.

Thanks.

Comment 8 Stephen Gordon 2012-12-20 13:12:20 UTC
(In reply to comment #7)
> Stephen
> 
> Could you please attach the complete log file? It would help me to
> understand the problem.
> 
> Thanks.

I don't have the log from that exact run anymore but I just tried it on another fresh install of the 3.2 alpha bits and got the same result. Will attach.

Comment 10 Alex Lourie 2012-12-23 13:20:15 UTC
Thank you Stephen, now I see what the problem is.

Comment 11 Alex Lourie 2012-12-26 15:16:53 UTC
The fix is posted for review: http://gerrit.ovirt.org/#/c/10389/

Comment 12 Alex Lourie 2012-12-27 12:02:22 UTC
*** Bug 886019 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.