+++ This bug is an upstream to downstream clone. The original bug is: +++ +++ bug 1416459 +++ ====================================================================== Created attachment 1244274 [details] restore log Description of problem: Restore HE backup will fail if the HE SD has disks of non-HE VM's Version-Release number of selected component (if applicable): rhevm-4.1.0.2-0.2.el7.noarch How reproducible: Always Steps to Reproduce: 1. Deploy HE environment 2. Add the storage domain to the engine(to start auto-import process) 3. Wait until the engine will have HE VM 4. Add non-HE VM with the disk on the HE SD 5. Set global maintenance 6. Backup the engine: # engine-backup --mode=backup --file=engine.backup --log=engine-backup.log 7. Copy the backup file from the HE VM to the host 8. Clean host from HE deploy(reprovisioning) 9. Run the HE deployment again 10. Answer No on the question "Automatically execute engine-setup on the engine appliance on first boot (Yes, No)[Yes]? " 11. Enter to the HE VM and copy the backup file from the host to the HE VM 12. Run restore command: # engine-backup --mode=restore --scope=all --file=engine.backup --log=engine-restore.log --he-remove-storage-vm --he-remove-hosts --restore-permissions --provision-all-databases Actual results: Restore operation failed with the error: SELECT DeleteHostedEngineStorageVM(); ERROR: The hosted-engine storage domain contains more than one vm. FATAL: Cannot execute sql command: --command=SELECT DeleteHostedEngineStorageVM(); 2017-01-25 09:25:05 9460: FATAL: Failed cleaning hosted-engine Expected results: Restore operation succeeds. I do not sure what the correct way to fix this problem(maybe we just need to prevent adding non-HE VM disks on the HE SD) Additional info: (Originally by Artyom Lukianov)
Let's remove the other disks from the SD during restore and prompt that we have done so for this use case. This because its not the same physical device. (Originally by Yaniv Dary)
Verified on: rhevm-4.0.7.4-0.1.el7ev.noarch # rpm -qa | grep hosted ovirt-hosted-engine-setup-2.0.4.3-3.el7ev.noarch ovirt-hosted-engine-ha-2.0.7-2.el7ev.noarch 1. Deploy HE environment 2. Add the storage domain to the engine(to start auto-import process) 3. Wait until the engine will have HE VM 4. Create the additional VM with the disk on the HE SD 5. Backup the engine: # engine-backup --mode=backup --file=engine.backup --log=engine-backup.log 6. Copy the backup file from the HE VM to the host 7. Clean host from HE deploy(reprovisioning) 8. Run the HE deployment again 9. Answer No on the question "Automatically execute engine-setup on the engine appliance on first boot (Yes, No)[Yes]? " 10. Enter to the HE VM and copy the backup file from the host to the HE VM 11. Run restore command: # engine-backup --mode=restore --scope=all --file=engine.backup --log=engine-restore.log --he-remove-storage-vm --he-remove-hosts --restore-permissions --provision-dwh-db --provision-db 12. Run engine setup: # engine-setup --offline 13. Finish HE deployment process Engine UP and have HE SD and HE VM in the active state
Also worked for me: 1.Deployed clean HE over two 3.6RHEL7.3 hosts, over NFS and added 2 data storage domains over NFS, got auto-imported HE-VM and HE storage domain. Components on engine: rhevm-setup-plugin-websocket-proxy-3.6.10.2-0.2.el6.noarch rhevm-doc-3.6.10-1.el6ev.noarch rhevm-webadmin-portal-3.6.10.2-0.2.el6.noarch rhevm-spice-client-x64-cab-3.6-7.el6.noarch rhevm-setup-plugins-3.6.5-1.el6ev.noarch rhevm-setup-base-3.6.10.2-0.2.el6.noarch rhevm-setup-plugin-ovirt-engine-common-3.6.10.2-0.2.el6.noarch rhevm-websocket-proxy-3.6.10.2-0.2.el6.noarch rhevm-dbscripts-3.6.10.2-0.2.el6.noarch rhevm-branding-rhev-3.6.0-10.el6ev.noarch rhevm-sdk-python-3.6.9.1-1.el6ev.noarch rhevm-dependencies-3.6.1-1.el6ev.noarch rhevm-tools-backup-3.6.10.2-0.2.el6.noarch rhevm-restapi-3.6.10.2-0.2.el6.noarch rhevm-spice-client-x86-cab-3.6-7.el6.noarch rhevm-guest-agent-common-1.0.11-6.el6ev.noarch rhevm-setup-3.6.10.2-0.2.el6.noarch rhevm-image-uploader-3.6.1-2.el6ev.noarch rhevm-setup-plugin-ovirt-engine-3.6.10.2-0.2.el6.noarch rhevm-tools-3.6.10.2-0.2.el6.noarch rhevm-log-collector-3.6.1-1.el6ev.noarch rhevm-backend-3.6.10.2-0.2.el6.noarch rhevm-spice-client-x86-msi-3.6-7.el6.noarch rhevm-lib-3.6.10.2-0.2.el6.noarch rhevm-setup-plugin-vmconsole-proxy-helper-3.6.10.2-0.2.el6.noarch rhevm-vmconsole-proxy-helper-3.6.10.2-0.2.el6.noarch rhevm-extensions-api-impl-3.6.10.2-0.2.el6.noarch rhevm-3.6.10.2-0.2.el6.noarch rhev-guest-tools-iso-3.6-6.el6ev.noarch rhevm-cli-3.6.9.0-1.el6ev.noarch rhevm-userportal-3.6.10.2-0.2.el6.noarch rhevm-spice-client-x64-msi-3.6-7.el6.noarch rhevm-iso-uploader-3.6.0-1.el6ev.noarch Linux version 2.6.32-642.el6.x86_64 (mockbuild.eng.bos.redhat.com) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-17) (GCC) ) #1 SMP Wed Apr 13 00:51:26 EDT 2016 Linux 2.6.32-642.el6.x86_64 #1 SMP Wed Apr 13 00:51:26 EDT 2016 x86_64 x86_64 x86_64 GNU/Linux Red Hat Enterprise Linux Server release 6.8 (Santiago) Components on hosts: ovirt-hosted-engine-ha-1.3.5.9-2.el7ev.noarch libvirt-client-2.0.0-10.el7_3.4.x86_64 mom-0.5.6-1.el7ev.noarch vdsm-4.17.37-1.el7ev.noarch ovirt-hosted-engine-setup-1.3.7.3-1.el7ev.noarch sanlock-3.4.0-1.el7.x86_64 ovirt-setup-lib-1.0.1-1.el7ev.noarch ovirt-vmconsole-host-1.0.4-1.el7ev.noarch qemu-kvm-rhev-2.6.0-28.el7_3.6.x86_64 ovirt-host-deploy-1.4.1-1.el7ev.noarch rhevm-sdk-python-3.6.9.1-1.el7ev.noarch ovirt-vmconsole-1.0.4-1.el7ev.noarch Linux version 3.10.0-514.6.1.el7.x86_64 (mockbuild.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC) ) #1 SMP Sat Dec 10 11:15:38 EST 2016 Linux 3.10.0-514.6.1.el7.x86_64 #1 SMP Sat Dec 10 11:15:38 EST 2016 x86_64 x86_64 x86_64 GNU/Linux Red Hat Enterprise Linux Server release 7.3 (Maipo) 2)Created guest-VM with bootable disk. 3)Set puma18 to maintenance from WEBUI and updated it to 4.1 repos, then updated it to latest 4.1 components e.g. "yum update ovirt-hosted-engine-setup -y", then "yum update -y" and then rebooted the host. 6)Host got updated to these components: mom-0.5.9-1.el7ev.noarch ovirt-engine-sdk-python-3.6.9.1-1.el7ev.noarch sanlock-3.4.0-1.el7.x86_64 ovirt-vmconsole-host-1.0.4-1.el7ev.noarch vdsm-4.19.6-1.el7ev.x86_64 qemu-kvm-rhev-2.6.0-28.el7_3.6.x86_64 ovirt-host-deploy-1.6.0-1.el7ev.noarch ovirt-vmconsole-1.0.4-1.el7ev.noarch ovirt-imageio-common-1.0.0-0.el7ev.noarch ovirt-imageio-daemon-1.0.0-0.el7ev.noarch ovirt-setup-lib-1.1.0-1.el7ev.noarch libvirt-client-2.0.0-10.el7_3.5.x86_64 ovirt-hosted-engine-ha-2.1.0.3-1.el7ev.noarch ovirt-hosted-engine-setup-2.1.0.3-1.el7ev.noarch Linux version 3.10.0-514.10.2.el7.x86_64 (mockbuild.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC) ) #1 SMP Mon Feb 20 02:37:52 EST 2017 Linux 3.10.0-514.10.2.el7.x86_64 #1 SMP Mon Feb 20 02:37:52 EST 2017 x86_64 x86_64 x86_64 GNU/Linux Red Hat Enterprise Linux Server release 7.3 (Maipo) 4)Activated puma18 via WEBUI and waited until it's score became positive (3400). 5)Selected puma18 as SPM. 6)Migrated HE-VM from puma19 (3.6 host) to puma18 (4.1 host). 7)Installed on puma18 rhevm-appliance-4.0.20170227.0-1.el7ev.noarch.rpm. 8)Created backup on engine "engine-backup --mode=backup --file=nsednev_from_puma19_rhevm_3_6 --log=Log_nsednev_from_puma19_rhevm_3_6", while environment is not in global maintenance, to verify bug 1403903. 9)Copied backup files to puma18 (4.1 host on which HE-VM was running) to "/backup" directory. 10)Set puma18 in global maintenance, so step 11 could be accomplished. 11)Ran hosted-engine --upgrade-appliance on puma18. 12)Successfully upgraded the engine from 3.6 to 4.0: puma18 ~]# hosted-engine --upgrade-appliance [ INFO ] Stage: Initializing [ INFO ] Stage: Environment setup During customization use CTRL-D to abort. ================================================================================== Welcome to the oVirt Self Hosted Engine setup/Upgrade tool. Please refer to the oVirt install guide: https://www.ovirt.org/documentation/how-to/hosted-engine/#fresh-install Please refer to the oVirt upgrade guide: https://www.ovirt.org/documentation/how-to/hosted-engine/#upgrade-hosted-engine ================================================================================== Continuing will upgrade the engine VM running on this hosts deploying and configuring a new appliance. If your engine VM is already based on el7 you can also simply upgrade the engine there. This procedure will create a new disk on the hosted-engine storage domain and it will backup there the content of your current engine VM disk. The new el7 based appliance will be deployed over the existing disk destroying its content; at any time you will be able to rollback using the content of the backup disk. You will be asked to take a backup of the running engine and copy it to this host. The engine backup will be automatically injected and recovered on the new appliance. Are you sure you want to continue? (Yes, No)[Yes]: Configuration files: [] Log file: /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20170305144438-4cclp3.log Version: otopi-1.6.0 (otopi-1.6.0-1.el7ev) [ INFO ] Detecting available oVirt engine appliances [ INFO ] Stage: Environment packages setup [ INFO ] Stage: Programs detection [ INFO ] Stage: Environment setup [ INFO ] Checking maintenance mode [ INFO ] The engine VM is running on this host [ INFO ] Stage: Environment customization [ INFO ] Answer file successfully loaded Enter engine admin password: Confirm engine admin password: [ INFO ] Acquiring internal CA cert from the engine [ INFO ] The following CA certificate is going to be used, please immediately interrupt if not correct: [ INFO ] Issuer: C=US, O=scl.lab.tlv.redhat.com, CN=nsednev-he-4.scl.lab.tlv.redhat.com.89576, Subject: C=US, O=scl.lab.tlv.redhat.com, CN=nsednev-he-4.scl.lab.tlv.redhat.com.89576, Fingerprint (SHA-1): D7E41C9B457311C6635CB27EA6F1B1DF4B7ED0D6 [ INFO ] Bridge ovirtmgmt already created [WARNING] Unable to uniquely detect the interface where Bridge ovirtmgmt has been created on, [u'vnet0', u'enp4s0f0'] appear to be valid alternatives [ INFO ] Checking SPM status on this host [ INFO ] Connecting to the Engine [ INFO ] This upgrade tool is running on the SPM host The following appliance have been found on your system: [1] - The RHEV-M Appliance image (OVA) - 4.0.20170227.0-1.el7ev [2] - Directly select an OVA file Please select an appliance (1, 2) [1]: [ INFO ] Verifying its sha1sum [ INFO ] Checking OVF archive content (could take a few minutes depending on archive size) [ INFO ] Checking OVF XML content (could take a few minutes depending on archive size) Please specify the size of the VM disk in GB: [50]: [ INFO ] Connecting to the Engine [ INFO ] The hosted-engine storage domain has enough free space to contain a new backup disk. [ INFO ] Checking version requirements [ INFO ] Connecting to the Engine [ INFO ] All the datacenters and clusters are at a compatible level [WARNING] Unsupported upgrade path This procedure has been designed and tested only for upgrading the engine VM from ['3.6'] to ['4.0']. Any other usage is highly experimental and potentially dangerous: Current engine: 3.6 Selected appliance: 4.0.20170227.0-1.el7ev Do you want to abort the upgrade process? (Yes, No) [Yes]: no [WARNING] Proceeding on an unsupported and highly experimental path [ INFO ] Detecting host timezone. Enter root password that will be used for the engine appliance (leave it empty to skip): Confirm appliance root password: Enter ssh public key for the root user that will be used for the engine appliance (leave it empty to skip): [WARNING] Skipping appliance root ssh public key Do you want to enable ssh access for the root user (yes, no, without-password) [yes]: Please take a backup of the current engine running this command on the engine VM: engine-backup --mode=backup --archive-compressor=gzip --file=engine_backup.tar.gz --log=engine_backup.log Then copy the backup archive to this host and input here its path when ready. Please specify path to engine backup archive you would like to restore on the new appliance: /backup/nsednev_from_puma19_rhevm_3_6 [ INFO ] Validating backup file '/backup/nsednev_from_puma19_rhevm_3_6' [ INFO ] Validating authentication plugins [ INFO ] Connecting to the Engine [ INFO ] '/backup/nsednev_from_puma19_rhevm_3_6' is a sane backup file [WARNING] Please take care that this will simply add an entry for this host under /etc/hosts on the engine VM. If in the past you added other entries there, recovering them is up to you. [ INFO ] Stage: Setup validation [ INFO ] Stage: Transaction setup [ INFO ] Stage: Misc configuration [ INFO ] Stage: Package installation [ INFO ] Stage: Misc configuration [ INFO ] Connecting to the Engine [ INFO ] Waiting for the engine to complete disk creation. This may take several minutes... [ INFO ] Still waiting for engine VM backup disk to be created. This may take several minutes... [ INFO ] Still waiting for engine VM backup disk to be created. This may take several minutes... [ INFO ] Still waiting for engine VM backup disk to be created. This may take several minutes... [ INFO ] Still waiting for engine VM backup disk to be created. This may take several minutes... [ INFO ] Still waiting for engine VM backup disk to be created. This may take several minutes... [ INFO ] Still waiting for engine VM backup disk to be created. This may take several minutes... [ INFO ] Still waiting for engine VM backup disk to be created. This may take several minutes... [ INFO ] Still waiting for engine VM backup disk to be created. This may take several minutes... [ INFO ] Still waiting for engine VM backup disk to be created. This may take several minutes... [ INFO ] Still waiting for engine VM backup disk to be created. This may take several minutes... [ INFO ] Still waiting for engine VM backup disk to be created. This may take several minutes... [ INFO ] Still waiting for engine VM backup disk to be created. This may take several minutes... [ INFO ] Still waiting for engine VM backup disk to be created. This may take several minutes... [ INFO ] Still waiting for engine VM backup disk to be created. This may take several minutes... [ INFO ] Still waiting for engine VM backup disk to be created. This may take several minutes... [ INFO ] The engine VM backup disk is now ready [ INFO ] Shutting down the current engine VM [ INFO ] Creating a backup of the engine VM disk (could take a few minutes depending on archive size) [ INFO ] Successfully created [ INFO ] Extracting disk image from OVF archive (could take a few minutes depending on archive size) [ INFO ] Validating pre-allocated volume size [ INFO ] Uploading volume to data domain (could take a few minutes depending on archive size) [ INFO ] Injecting engine backup [ INFO ] Backup successfully injected [ INFO ] Image successfully imported from OVF [ INFO ] Stage: Transaction commit [ INFO ] Stage: Closing up [ INFO ] Reloading vm.conf from the shared storage domain [ INFO ] Trying to get a fresher copy of vm configuration from the OVF_STORE [ INFO ] Found an OVF for HE VM, trying to convert [ INFO ] Got vm.conf from OVF_STORE [ INFO ] Running engine-setup on the appliance |- Preparing to restore: |- - Unpacking file '/root/nsednev_from_puma19_rhevm_3_6' |- Restoring: |- - Files |- ------------------------------------------------------------------------------ |- Please note: |- Operating system is different from the one used during backup. |- Current operating system: redhat7 |- Operating system at backup: redhat6 |- Apache httpd configuration will not be restored. |- You will be asked about it on the next engine-setup run. |- ------------------------------------------------------------------------------ |- Provisioning PostgreSQL users/databases: |- - user 'engine', database 'engine' |- Restoring: |- - Engine database 'engine' |- - Cleaning up temporary tables in engine database 'engine' |- - Inserting DbJustRestored VdcOption in engine database |- ------------------------------------------------------------------------------ |- Please note: |- The engine database was backed up at 2017-03-05 14:25:50.000000000 +0200 . |- Objects that were added, removed or changed after this date, such as virtual |- machines, disks, etc., are missing in the engine, and will probably require |- recovery or recreation. |- ------------------------------------------------------------------------------ |- You should now run engine-setup. |- Done. |- HE_APPLIANCE_ENGINE_RESTORE_SUCCESS [ INFO ] Engine backup successfully restored |- [ INFO ] Stage: Initializing |- [ INFO ] Stage: Environment setup |- Configuration files: ['/etc/ovirt-engine-setup.conf.d/10-packaging-wsp.conf', '/etc/ovirt-engine-setup.conf.d/10-packaging.conf', '/etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf', '/root/ovirt-engine-answers', '/root/heanswers.conf'] |- Log file: /var/log/ovirt-engine/setup/ovirt-engine-setup-20170305150314-tznjmq.log |- Version: otopi-1.5.2 (otopi-1.5.2-1.el7ev) |- [ INFO ] The engine DB has been restored from a backup |- [ INFO ] Stage: Environment packages setup |- [ INFO ] Stage: Programs detection |- [ INFO ] Stage: Environment setup |- [ INFO ] Stage: Environment customization |- |- Welcome to the RHEV 4.0 setup/upgrade. |- Please read the RHEV 4.0 install guide |- https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Virtualization/4.0/html/Installation_Guide/index.html. |- Please refer to the RHEV Upgrade Helper application |- https://access.redhat.com/labs/rhevupgradehelper/ which will guide you in the upgrading process. |- Would you like to proceed? (Yes, No) [Yes]: |- |- --== PRODUCT OPTIONS ==-- |- |- Configure Image I/O Proxy on this host? (Yes, No) [Yes]: |- Please note: Data Warehouse is required for the engine. If you choose to not configure it on this host, you have to configure it on a remote host, and then configure the engine on this host so that it can access the database of the remote Data Warehouse host. |- Configure Data Warehouse on this host (Yes, No) [Yes]: |- |- --== PACKAGES ==-- |- |- |- --== NETWORK CONFIGURATION ==-- |- |- [ INFO ] firewalld will be configured as firewall manager. |- |- --== DATABASE CONFIGURATION ==-- |- |- Where is the DWH database located? (Local, Remote) [Local]: |- Setup can configure the local postgresql server automatically for the DWH to run. This may conflict with existing applications. |- Would you like Setup to automatically configure postgresql and create DWH database, or prefer to perform that manually? (Automatic, Manual) [Automatic]: |- |- --== OVIRT ENGINE CONFIGURATION ==-- |- |- |- --== STORAGE CONFIGURATION ==-- |- |- |- --== PKI CONFIGURATION ==-- |- |- |- --== APACHE CONFIGURATION ==-- |- |- |- --== SYSTEM CONFIGURATION ==-- |- |- |- --== MISC CONFIGURATION ==-- |- |- Please choose Data Warehouse sampling scale: |- (1) Basic |- (2) Full |- (1, 2)[1]: |- |- --== END OF CONFIGURATION ==-- |- |- [ INFO ] Stage: Setup validation |- [ INFO ] Cleaning stale zombie tasks and commands |- |- --== CONFIGURATION PREVIEW ==-- |- |- Application mode : both |- Default SAN wipe after delete : False |- Firewall manager : firewalld |- Update Firewall : True |- Host FQDN : nsednev-he-4.scl.lab.tlv.redhat.com |- Engine database secured connection : False |- Engine database user name : engine |- Engine database name : engine |- Engine database host : localhost |- Engine database port : 5432 |- Engine database host name validation : False |- DWH database secured connection : False |- DWH database host : localhost |- DWH database user name : ovirt_engine_history |- DWH database name : ovirt_engine_history |- DWH database port : 5432 |- DWH database host name validation : False |- Engine installation : True |- PKI organization : scl.lab.tlv.redhat.com |- Configure local Engine database : True |- Set application as default page : True |- Configure Apache SSL : True |- DWH installation : True |- Configure local DWH database : True |- Engine Host FQDN : nsednev-he-4.scl.lab.tlv.redhat.com |- Configure Image I/O Proxy : True |- Configure VMConsole Proxy : True |- Configure WebSocket Proxy : True |- [ INFO ] Cleaning async tasks and compensations |- [ INFO ] Unlocking existing entities |- [ INFO ] Checking the Engine database consistency |- [ INFO ] Stage: Transaction setup |- [ INFO ] Stopping engine service |- [ INFO ] Stopping ovirt-fence-kdump-listener service |- [ INFO ] Stopping dwh service |- [ INFO ] Stopping Image I/O Proxy service |- [ INFO ] Stopping websocket-proxy service |- [ INFO ] Stage: Misc configuration |- [ INFO ] Stage: Package installation |- [ INFO ] Stage: Misc configuration |- [ INFO ] Upgrading CA |- [ INFO ] Creating PostgreSQL 'ovirt_engine_history' database |- [ INFO ] Configuring PostgreSQL |- [ INFO ] Backing up database localhost:engine to '/var/lib/ovirt-engine/backups/engine-20170305150324.CXePP1.dump'. |- [ INFO ] Creating/refreshing Engine database schema |- [ INFO ] Creating/refreshing DWH database schema |- [ INFO ] Configuring Image I/O Proxy |- [ INFO ] Configuring WebSocket Proxy |- [ INFO ] Creating/refreshing Engine 'internal' domain database schema |- [ INFO ] Generating post install configuration file '/etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf' |- [ INFO ] Stage: Transaction commit |- [ INFO ] Stage: Closing up |- [ INFO ] Starting engine service |- [ INFO ] Starting dwh service |- [ INFO ] Restarting ovirt-vmconsole proxy service |- |- --== SUMMARY ==-- |- |- [ INFO ] Restarting httpd |- Web access is enabled at: |- http://nsednev-he-4.scl.lab.tlv.redhat.com:80/ovirt-engine |- https://nsednev-he-4.scl.lab.tlv.redhat.com:443/ovirt-engine |- Internal CA D7:E4:1C:9B:45:73:11:C6:63:5C:B2:7E:A6:F1:B1:DF:4B:7E:D0:D6 |- SSH fingerprint: 3f:ae:99:ac:54:9c:f2:4f:23:ec:bc:be:aa:5a:bf:77 |- |- --== END OF SUMMARY ==-- |- |- [ INFO ] Stage: Clean up |- Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20170305150314-tznjmq.log |- [ INFO ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20170305150417-setup.conf' |- [ INFO ] Stage: Pre-termination |- [ INFO ] Stage: Termination |- [ INFO ] Execution of setup completed successfully |- HE_APPLIANCE_ENGINE_SETUP_SUCCESS [ INFO ] Engine-setup successfully completed [ INFO ] Engine is still unreachable [ INFO ] Engine is still not reachable, waiting... [ INFO ] Engine replied: DB Up!Welcome to Health Status! [ INFO ] Connecting to the Engine [ INFO ] Connecting to the Engine [ INFO ] Registering the hosted-engine backup disk in the DB [ INFO ] Waiting for the engine to complete disk registration. This may take several minutes... [ INFO ] The engine VM backup disk is now ready [ INFO ] Shutting down the engine VM [ INFO ] Stage: Clean up [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination [ INFO ] Hosted Engine successfully upgraded [ INFO ] Please exit global maintenance mode to restart the engine VM. [ INFO ] Please note that the engine VM ssh keys have changed. Please remove the engine VM entry in ssh known_hosts on your clients. You have new mail in /var/spool/mail/root 13)Removed environment from global maintenance e.g. "hosted-engine --set-maintenance --mode=none". Components on upgraded 4.0 engine: rhevm-spice-client-x64-msi-4.0-3.el7ev.noarch rhevm-spice-client-x86-msi-4.0-3.el7ev.noarch rhev-guest-tools-iso-4.0-7.el7ev.noarch rhevm-setup-plugins-4.0.0.3-1.el7ev.noarch rhevm-doc-4.0.7-1.el7ev.noarch rhevm-dependencies-4.0.0-1.el7ev.noarch rhevm-guest-agent-common-1.0.12-4.el7ev.noarch rhevm-branding-rhev-4.0.0-7.el7ev.noarch rhevm-4.0.7.3-0.1.el7ev.noarch Linux version 3.10.0-514.6.2.el7.x86_64 (mockbuild.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC) ) #1 SMP Fri Feb 17 19:21:31 EST 2017 Linux 3.10.0-514.6.2.el7.x86_64 #1 SMP Fri Feb 17 19:21:31 EST 2017 x86_64 x86_64 x86_64 GNU/Linux Red Hat Enterprise Linux Server release 7.3 (Maipo) 14)Set puma19 to maintenance from WEBUI and updated it to 4.1 repos, then updated it to latest 4.1 components e.g. "yum update ovirt-hosted-engine-setup -y", then "yum update -y" and then rebooted the host. 15)Host got updated to these components: mom-0.5.9-1.el7ev.noarch ovirt-engine-sdk-python-3.6.9.1-1.el7ev.noarch sanlock-3.4.0-1.el7.x86_64 ovirt-vmconsole-host-1.0.4-1.el7ev.noarch vdsm-4.19.6-1.el7ev.x86_64 qemu-kvm-rhev-2.6.0-28.el7_3.6.x86_64 ovirt-host-deploy-1.6.0-1.el7ev.noarch ovirt-vmconsole-1.0.4-1.el7ev.noarch ovirt-imageio-common-1.0.0-0.el7ev.noarch ovirt-imageio-daemon-1.0.0-0.el7ev.noarch ovirt-setup-lib-1.1.0-1.el7ev.noarch libvirt-client-2.0.0-10.el7_3.5.x86_64 ovirt-hosted-engine-ha-2.1.0.3-1.el7ev.noarch ovirt-hosted-engine-setup-2.1.0.3-1.el7ev.noarch Linux version 3.10.0-514.10.2.el7.x86_64 (mockbuild.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC) ) #1 SMP Mon Feb 20 02:37:52 EST 2017 Linux 3.10.0-514.10.2.el7.x86_64 #1 SMP Mon Feb 20 02:37:52 EST 2017 x86_64 x86_64 x86_64 GNU/Linux Red Hat Enterprise Linux Server release 7.3 (Maipo) 16)Activated puma19 via WEBUI and waited until it's score became positive (3400). 17)Bumped up host cluster compatibility mode from 3.6 to 4.0. 18)Bumped up data center compatibility mode from 3.6 to 4.0. 19)Upgraded engine only from 4.0 to latest 4.1, using global maintenance on hosts.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2017-0542.html