Created attachment 707864 [details]
## Log

Description of problem:  Failed upgrade RHEVM from 3.1.3 to 3.2


Version-Release number of selected component (if applicable):
RHEVM 3.1.3 - SI27.3 environment: 

RHEVM: rhevm-3.1.0-50.el6ev.noarch 
VDSM: vdsm-4.10.2-1.6.el6.x86_64 
LIBVIRT: libvirt-0.10.2-18.el6.x86_64 
QEMU & KVM: qemu-kvm-rhev-0.12.1.2-2.355.el6_4.1.x86_64 
SANLOCK: sanlock-2.6-2.el6.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Create RHEVM 3.1.3 SI27.3 with local DB
2. Edit REPO file with relevant links to RHEVM 3.2 SF9
3. rhevm-check-update
4. yum update rhevm-setup
5. rhevm-upgrade
  
Actual results:
Failed updating database

Expected results:
Successfully upgrade RHEVM from 3.1.3 to 3.2

Additional info:

/var/log/ovirt-engine/ovirt-engine-upgrade_2013_03_10_14_04_14.log

2013-03-10 14:05:50::DEBUG::common_utils::332::root:: YUM: VERB: Transaction processed
2013-03-10 14:05:50::DEBUG::rhevm-upgrade::402::root:: DB Update started
2013-03-10 14:05:50::DEBUG::common_utils::435::root:: Executing command --> '/usr/share/ovirt-engine/dbscripts/upgrade.sh -s localhost -p 5432 -u engine -d engine_2013_03_10_14_04_15' in working directory '/usr/share/ovirt-engine/dbscrip
ts'
2013-03-10 14:05:58::DEBUG::common_utils::473::root:: output = /usr/share/ovirt-engine/dbscripts /usr/share/ovirt-engine/dbscripts
upgrade script detected a change in Config, View or Stored Procedure...

2013-03-10 14:05:58::DEBUG::common_utils::474::root:: stderr = psql:drop_old_uuid_functions.sql:25: ERROR:  must be owner of function uuid_nil
CONTEXT:  SQL statement "drop function if exists uuid_nil()"
PL/pgSQL function "__temp_drop_old_uuid_functions" line 9 at SQL statement
psql:create_functions.sql:670: ERROR:  must be owner of function uuid_generate_v1

2013-03-10 14:05:58::DEBUG::common_utils::475::root:: retcode = 3
2013-03-10 14:05:58::ERROR::rhevm-upgrade::1181::root:: Traceback (most recent call last):
  File "/usr/bin/rhevm-upgrade", line 1168, in main
    runFunc([db.update], MSG_INFO_DB_UPDATE)
  File "/usr/bin/rhevm-upgrade", line 600, in runFunc
    func()
  File "/usr/bin/rhevm-upgrade", line 419, in update
    output, rc = utils.execCmd(cmdList=cmd, failOnError=True, msg=MSG_ERROR_UPDATE_DB)
  File "/usr/share/ovirt-engine/scripts/common_utils.py", line 478, in execCmd
    raise Exception(msg)
Exception: Error: Database update failed

2013-03-10 14:05:58::ERROR::rhevm-upgrade::1182::root:: Rolling back update
2013-03-10 14:05:58::ERROR::rhevm-upgrade::553::root:: PKI: cannot remove '/etc/httpd/conf.d/ssl.conf.tmp'
2013-03-10 14:05:58::ERROR::rhevm-upgrade::553::root:: PKI: cannot remove '/etc/pki/ovirt-engine/keys/engine.p12'
2013-03-10 14:05:58::ERROR::rhevm-upgrade::553::root:: PKI: cannot remove '/etc/pki/ovirt-engine/keys/apache.p12'
2013-03-10 14:05:58::ERROR::rhevm-upgrade::553::root:: PKI: cannot remove '/etc/pki/ovirt-engine/apache-ca.pem'
2013-03-10 14:05:58::ERROR::rhevm-upgrade::553::root:: PKI: cannot remove '/etc/pki/ovirt-engine/certs/apache.cer'
2013-03-10 14:05:58::ERROR::rhevm-upgrade::553::root:: PKI: cannot remove '/etc/pki/ovirt-engine/keys/apache.key.nopass'

[root@puma31 yum.repos.d]# rhevm-upgrade

Checking for updates... (This may take several minutes)...[ DONE ]
14 Updates available:
 * rhevm-3.2.0-10.10.beta1.el6ev.noarch
 * otopi-1.0.0-1.el6ev.noarch
 * otopi-java-1.0.0-1.el6ev.noarch
 * ovirt-host-deploy-1.0.0-2.el6ev.noarch
 * ovirt-host-deploy-java-1.0.0-2.el6ev.noarch
 * rhevm-backend-3.2.0-10.10.beta1.el6ev.noarch
 * rhevm-config-3.2.0-10.10.beta1.el6ev.noarch
 * rhevm-dbscripts-3.2.0-10.10.beta1.el6ev.noarch
 * rhevm-genericapi-3.2.0-10.10.beta1.el6ev.noarch
 * rhevm-notification-service-3.2.0-10.10.beta1.el6ev.noarch
 * rhevm-restapi-3.2.0-10.10.beta1.el6ev.noarch
 * rhevm-tools-common-3.2.0-10.10.beta1.el6ev.noarch
 * rhevm-userportal-3.2.0-10.10.beta1.el6ev.noarch
 * rhevm-webadmin-portal-3.2.0-10.10.beta1.el6ev.noarch

During the upgrade process, RHEV Manager  will not be accessible.
All existing running virtual machines will continue but you will not be able to
start or stop any new virtual machines during the process.

Would you like to proceed? (yes|no): yes 
Stopping ovirt-engine service...                     [ DONE ]
Stopping DB related services...                      [ DONE ]
Cleaning async tasks...                              [ DONE ]
Pre-upgrade validations...                           [ DONE ]
Backing Up Database...                               [ DONE ]
Rename Database...                                   [ DONE ]
Updating rpms...                                     [ DONE ]
Updating Database...                                 [ ERROR ]

 **Error: Upgrade failed, rolling back**
 **Reason: Error: Database update failed**

Restoring CA...                                      [ DONE ]
Restoring Database...                                [ ERROR ]
Starting ovirt-engine service...                     [ DONE ]
Rolling back rpms...
Error: Upgrade failed.
please check log at /var/log/ovirt-engine/ovirt-engine-upgrade_2013_03_10_14_04_14.log
* DB Backup available at  /var/lib/ovirt-engine/backups/ovirt-engine_db_backup_2013_03_10_14_04_15.sql