Bug 863244

Summary: rhevm-upgrade does not stop dwh service and as a result fails when renaming the database
Product: Red Hat Enterprise Virtualization Manager Reporter: Stephen Gordon <sgordon>
Component: ovirt-engine-setupAssignee: Alex Lourie <alourie>
Status: CLOSED CURRENTRELEASE QA Contact: David Botzer <dbotzer>
Severity: high Docs Contact:
Priority: unspecified    
Version: 3.1.0CC: bazulay, dyasny, iheim, mgoldboi, pstehlik, Rhev-m-bugs, sgrinber, ykaul
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: integration
Fixed In Version: si21 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-12-04 20:03:16 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Stephen Gordon 2012-10-04 19:47:23 UTC
Description of problem:

When running rhevm-upgrade to move from si19.1 to si20 I received an error indicating that "Database rename failed. Check that there are no active connections to the DB and try again.":

# rhevm-upgrade 
Loaded plugins: product-id, rhnplugin
Checking for updates... (This may take several minutes)
14 Updates available:
 * rhevm-3.1.0-20.el6ev.noarch
 * rhevm-backend-3.1.0-20.el6ev.noarch
 * rhevm-config-3.1.0-20.el6ev.noarch
 * rhevm-dbscripts-3.1.0-20.el6ev.noarch
 * rhevm-genericapi-3.1.0-20.el6ev.noarch
 * rhevm-image-uploader-3.1.0-4.el6ev.noarch
 * rhevm-iso-uploader-3.1.0-5.el6ev.noarch
 * rhevm-log-collector-3.1.0-7.el6ev.noarch
 * rhevm-notification-service-3.1.0-20.el6ev.noarch
 * rhevm-restapi-3.1.0-20.el6ev.noarch
 * rhevm-tools-common-3.1.0-20.el6ev.noarch
 * rhevm-userportal-3.1.0-20.el6ev.noarch
 * rhevm-webadmin-portal-3.1.0-20.el6ev.noarch
 * vdsm-bootstrap-4.9.6-37.0.el6_3.noarch
Stopping ovirt-engine Service...                         [ DONE ]
Pre-upgrade validations...                               [ DONE ]
Backing Up Database...                                   [ DONE ]
Rename Database...                                       [ ERROR ]
Error: Database rename failed. Check that there are no active connections to the DB and try again.

Investigating the log I found further messages:

2012-10-04 19:39:08::DEBUG::common_utils::361::root:: running sql query 'ALTER DATABASE engine RENAME TO engine_2012_10_04_19_37_43' on db server: 'localhost'.
2012-10-04 19:39:08::DEBUG::common_utils::302::root:: Executing command --> '/usr/bin/psql -h localhost -p 5432 -U engine -d template1 -c ALTER DATABASE engine RENAME TO engine_2012_10_04_19_37_43'
2012-10-04 19:39:14::DEBUG::common_utils::340::root:: output = 
2012-10-04 19:39:14::DEBUG::common_utils::341::root:: stderr = ERROR:  database "engine" is being accessed by other users
DETAIL:  There are 2 other session(s) using the database.

2012-10-04 19:39:14::DEBUG::common_utils::342::root:: retcode = 1
2012-10-04 19:39:14::ERROR::rhevm-upgrade::1036::root:: Traceback (most recent call last):
  File "/usr/bin/rhevm-upgrade", line 1029, in <module>
    main(options)
  File "/usr/bin/rhevm-upgrade", line 940, in main
    runFunc([[db.rename, DB_NAME_TEMP]], MSG_INFO_RENAME_DB)
  File "/usr/bin/rhevm-upgrade", line 618, in runFunc
    func[0](*func[1:])
  File "/usr/bin/rhevm-upgrade", line 574, in rename
    utils.execRemoteSqlCommand(SERVER_ADMIN, SERVER_NAME, SERVER_PORT, basedefs.DB_TEMPLATE, query, True, MSG_ERROR_RENAME_DB)
  File "/usr/share/ovirt-engine/scripts/common_utils.py", line 370, in execRemoteSqlCommand
    return execCmd(cmdList=cmd, failOnError=failOnError, msg=errMsg, envDict=getPgPassEnv())
  File "/usr/share/ovirt-engine/scripts/common_utils.py", line 345, in execCmd
    raise Exception(msg)
Exception: Error: Database rename failed. Check that there are no active connections to the DB and try again.

After checking that ovirt-engine had indeed been stopped correctly (by rhevm-upgrade) I proceeded to look for another culprit that might be accessing the database. I found that the 'ovirt-engine-dwhd' service was still running, the rhevm-upgrade script does not appear to attempt to stop it.

Version-Release number of selected component (if applicable):

si20

rhevm-setup.noarch 0:3.1.0-20.el6ev

Actual results:

ovirt-engine-dwhd service was not stopped, upgrade failed.

Expected results:

All ovirt/rhevm services detected should be stopped, allowing for successful upgrades.

Comment 1 Stephen Gordon 2012-10-04 19:48:10 UTC
Stopped ovirt-engine-dwhd service manually, re-attempted rhevm-upgrade run and upgrade succeeded.

Comment 3 David Botzer 2012-10-24 11:45:30 UTC
Fixed, 3.1/si22.1
Upgraded si21 to si22 & used si22.1 for reports
(Remote Setup)
Fixed, 3.1/si22.1

Would you like to proceed? (yes|no): y
Stopping ovirt-engine Service...                         [ DONE ]
Stopping DB related services...                          [ DONE ]
Pre-upgrade validations...                               [ DONE ]
Backing Up Database...                                   [ DONE ]
Rename Database...                                       [ DONE ]
Updating rpms...                                         [ DONE ]
Updating Database...                                     [ DONE ]
Restore Database name...                                 [ DONE ]
Running post install configuration...                    [ DONE ]
Starting ovirt-engine...                                 [ DONE ]

RHEV Manager upgrade completed successfully!