Bug 1397810 - The rollback will be uncorrected prevented if the 4.0 appliance disk is smaller than the on of 3.6 custom VM
Summary: The rollback will be uncorrected prevented if the 4.0 appliance disk is small...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-hosted-engine-setup
Classification: oVirt
Component: Plugins.General
Version: 2.0.3.1
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ovirt-4.0.6
: 2.0.4.1
Assignee: Simone Tiraboschi
QA Contact: Nikolai Sednev
URL:
Whiteboard: integration
Depends On: 1403903
Blocks: 1396051
TreeView+ depends on / blocked
 
Reported: 2016-11-23 11:55 UTC by Simone Tiraboschi
Modified: 2017-05-11 09:26 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-01-18 07:28:50 UTC
oVirt Team: Integration
Embargoed:
rule-engine: ovirt-4.0.z+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 67220 0 master MERGED rollback: correctly evaluate the space on the VM disk 2016-11-28 14:24:05 UTC
oVirt gerrit 67437 0 ovirt-hosted-engine-setup-2.0 MERGED rollback: correctly evaluate the space on the VM disk 2016-11-28 15:50:03 UTC

Description Simone Tiraboschi 2016-11-23 11:55:22 UTC
Description of problem:
The disk of the 3.6 appliance is 100 GB since the user deployed it with a custom VM; the he decided to upgrade using a 50 GB appliance and he also choose to avoid using the whole 100 GB but just 50 GB.

After the upgrade:

the backup disk has:
u'description': u'hosted-engine-backup-20161123122527'
u'truesize': u'23660318720'
u'capacity': u'107374182400',
u'apparentsize': u'107374182400'

The VM disk (now with the appliance):
u'description': u'Hosted Engine Image'
u'truesize': u'3455299584'
u'capacity': u'107374182400'
u'apparentsize': u'53687091200'

If the user tries to rollback, the rollback procedure will be prevented by:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/otopi/context.py", line 132, in _executeMethod
    method['method']()
  File "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/gr-he-rollbackupgrade/core/misc.py", line 308, in _validate_disks
    destination=destination_size,
RuntimeError: Error on volume size: the selected backup (size 107374182400) doesn't fit the engine VM disk (size 53687091200)
2016-11-23 13:01:21 ERROR otopi.context context._executeMethod:151 Failed to execute stage 'Setup validation': Error on volume size: the selected backup (size 107374182400) doesn't fit the engine VM disk (size 53687091200)

Since the code is comparing against apparentsize instead of capacity

Version-Release number of selected component (if applicable):


How reproducible:
100%

Steps to Reproduce:
1. deploy hosted-engine 3.6 with a 100 GB VM
2. upgrade the engine VM to 4.0 using a 50 GB appliance
3. Try to rollback

Actual results:
2016-11-23 13:01:21 ERROR otopi.context context._executeMethod:151 Failed to execute stage 'Setup validation': Error on volume size: the selected backup (size 107374182400) doesn't fit the engine VM disk (size 53687091200)

Expected results:
It successfully rollbacks.

Additional info:
getVolumeSize doesn't report the capacity, only getVolumeINFO does

Comment 2 Nikolai Sednev 2017-01-05 01:05:59 UTC
Works for me:

1-Deployed HE on pair of 3.6's RHEVH7.2, over NFS and added two data storage domains.
2-Got hosted-storage auto-imported.

These are host's components while still in 3.6:
ovirt-vmconsole-host-1.0.4-1.el7ev.noarch
libvirt-client-1.2.17-13.el7_2.5.x86_64
ovirt-node-lib-config-3.6.1-15.0.el7ev.noarch
ovirt-node-selinux-3.6.1-15.0.el7ev.noarch
ovirt-node-plugin-snmp-logic-3.6.1-15.0.el7ev.noarch
qemu-kvm-rhev-2.3.0-31.el7_2.21.x86_64
vdsm-4.17.35-1.el7ev.noarch
ovirt-node-plugin-rhn-3.6.1-15.0.el7ev.noarch
ovirt-host-deploy-1.4.1-1.el7ev.noarch
sanlock-3.2.4-1.el7.x86_64
ovirt-setup-lib-1.0.1-1.el7ev.noarch
ovirt-vmconsole-1.0.4-1.el7ev.noarch
ovirt-node-lib-legacy-3.6.1-15.0.el7ev.noarch
ovirt-node-lib-3.6.1-15.0.el7ev.noarch
ovirt-node-3.6.1-15.0.el7ev.noarch
ovirt-node-plugin-cim-logic-3.6.1-15.0.el7ev.noarch
ovirt-hosted-engine-ha-1.3.5.8-1.el7ev.noarch
ovirt-node-plugin-hosted-engine-0.3.0-8.el7ev.noarch
ovirt-host-deploy-offline-1.4.0-1.el7ev.x86_64
ovirt-node-plugin-vdsm-0.6.1-11.el7ev.noarch
ovirt-node-plugin-snmp-3.6.1-15.0.el7ev.noarch
mom-0.5.6-1.el7ev.noarch
ovirt-hosted-engine-setup-1.3.7.3-1.el7ev.noarch
ovirt-node-plugin-cim-3.6.1-15.0.el7ev.noarch
rhevm-sdk-python-3.6.9.1-1.el7ev.noarch
ovirt-node-branding-rhev-3.6.1-15.0.el7ev.noarch
Linux version 3.10.0-327.36.1.el7.x86_64 (mockbuild.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-4) (GCC) ) #1 SMP Wed Aug 17 03:02:37 EDT 2016
Linux 3.10.0-327.36.1.el7.x86_64 #1 SMP Wed Aug 17 03:02:37 EDT 2016 x86_64 x86_64 x86_64 GNU/Linux
Red Hat Enterprise Virtualization Hypervisor release 7.2 (20160920.1.el7ev)

Engine:
rhv-release-3.6.10-3-001.noarch
rhevm-setup-base-3.6.10.2-0.2.el6.noarch
rhevm-extensions-api-impl-3.6.10.2-0.2.el6.noarch
rhevm-backend-3.6.10.2-0.2.el6.noarch
ovirt-vmconsole-1.0.4-1.el6ev.noarch
rhevm-doc-3.6.10-1.el6ev.noarch
rhev-guest-tools-iso-3.6-6.el6ev.noarch
rhevm-spice-client-x86-cab-3.6-7.el6.noarch
rhevm-setup-plugin-ovirt-engine-3.6.10.2-0.2.el6.noarch
rhevm-dbscripts-3.6.10.2-0.2.el6.noarch
rhevm-reports-setup-3.6.5.1-1.el6ev.noarch
rhevm-image-uploader-3.6.1-2.el6ev.noarch
rhevm-spice-client-x86-msi-3.6-7.el6.noarch
rhevm-setup-plugins-3.6.5-1.el6ev.noarch
rhevm-setup-plugin-vmconsole-proxy-helper-3.6.10.2-0.2.el6.noarch
rhevm-setup-3.6.10.2-0.2.el6.noarch
rhevm-tools-3.6.10.2-0.2.el6.noarch
rhevm-restapi-3.6.10.2-0.2.el6.noarch
rhevm-reports-3.6.5.1-1.el6ev.noarch
ovirt-engine-extension-aaa-jdbc-1.0.7-2.el6ev.noarch
rhevm-iso-uploader-3.6.0-1.el6ev.noarch
rhevm-dependencies-3.6.1-1.el6ev.noarch
rhevm-guest-agent-common-1.0.11-6.el6ev.noarch
ovirt-vmconsole-proxy-1.0.4-1.el6ev.noarch
rhevm-branding-rhev-3.6.0-10.el6ev.noarch
rhevm-spice-client-x64-cab-3.6-7.el6.noarch
rhevm-setup-plugin-ovirt-engine-common-3.6.10.2-0.2.el6.noarch
rhevm-websocket-proxy-3.6.10.2-0.2.el6.noarch
rhevm-tools-backup-3.6.10.2-0.2.el6.noarch
rhevm-userportal-3.6.10.2-0.2.el6.noarch
rhevm-dwh-3.6.8-1.el6ev.noarch
rhevm-sdk-python-3.6.9.1-1.el6ev.noarch
rhevm-log-collector-3.6.1-1.el6ev.noarch
ovirt-host-deploy-1.4.1-1.el6ev.noarch
rhevm-spice-client-x64-msi-3.6-7.el6.noarch
rhevm-setup-plugin-websocket-proxy-3.6.10.2-0.2.el6.noarch
rhevm-webadmin-portal-3.6.10.2-0.2.el6.noarch
ovirt-setup-lib-1.0.1-1.el6ev.noarch
rhevm-dwh-setup-3.6.8-1.el6ev.noarch
rhevm-cli-3.6.9.0-1.el6ev.noarch
ovirt-host-deploy-java-1.4.1-1.el6ev.noarch
rhevm-lib-3.6.10.2-0.2.el6.noarch
rhevm-vmconsole-proxy-helper-3.6.10.2-0.2.el6.noarch
rhevm-3.6.10.2-0.2.el6.noarch
Linux version 2.6.32-642.11.1.el6.x86_64 (mockbuild.eng.bos.redhat.com) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-17) (GCC) ) #1 SMP Wed Oct 26 10:25:23 EDT 2016
Linux 2.6.32-642.11.1.el6.x86_64 #1 SMP Wed Oct 26 10:25:23 EDT 2016 x86_64 x86_64 x86_64 GNU/Linux
Red Hat Enterprise Linux Server release 6.8 (Santiago)

3-Installed on the engine the latest rhev-hypervisor7.noarch 0:7.3-20161028.1.el6ev
4-Updated both hosts from RHEV Hypervisor - 7.2 - 20160920.1.el7ev 
OS Version:
RHEV Hypervisor - 7.2 - 20160920.1.el7ev
Kernel Version:
3.10.0 - 327.36.1.el7.x86_64
KVM Version:
2.3.0 - 31.el7_2.21
LIBVIRT Version:
libvirt-1.2.17-13.el7_2.5
VDSM Version:
vdsm-4.17.35-1.el7ev
SPICE Version:
0.12.4 - 15.el7_2.2
GlusterFS Version:
[N/A]
CEPH Version:
librbd1-0.94.5-15.el7cp

to 
OS Version:
RHEV Hypervisor - 7.3 - 20161028.1.el7ev
Kernel Version:
3.10.0 - 514.el7.x86_64
KVM Version:
2.6.0 - 27.el7
LIBVIRT Version:
libvirt-2.0.0-10.el7
VDSM Version:
vdsm-4.17.35-1.el7ev
SPICE Version:
0.12.4 - 19.el7
GlusterFS Version:
[N/A]
CEPH Version:
librbd1-0.94.5-1.el7

ovirt-node-lib-3.6.1-34.0.el7ev.noarch
ovirt-node-plugin-cim-logic-3.6.1-34.0.el7ev.noarch
qemu-kvm-rhev-2.6.0-27.el7.x86_64
ovirt-node-plugin-hosted-engine-0.3.0-8.el7ev.noarch
ovirt-node-plugin-vdsm-0.6.1-11.el7ev.noarch
ovirt-node-plugin-cim-3.6.1-34.0.el7ev.noarch
ovirt-node-branding-rhev-3.6.1-34.0.el7ev.noarch
rhevm-sdk-python-3.6.9.1-1.el7ev.noarch
ovirt-host-deploy-1.4.1-1.el7ev.noarch
ovirt-vmconsole-1.0.4-1.el7ev.noarch
libvirt-client-2.0.0-10.el7.x86_64
ovirt-node-lib-legacy-3.6.1-34.0.el7ev.noarch
ovirt-node-selinux-3.6.1-34.0.el7ev.noarch
ovirt-node-plugin-snmp-logic-3.6.1-34.0.el7ev.noarch
mom-0.5.6-1.el7ev.noarch
vdsm-4.17.35-1.el7ev.noarch
ovirt-hosted-engine-setup-1.3.7.3-1.el7ev.noarch
ovirt-node-plugin-rhn-3.6.1-34.0.el7ev.noarch
ovirt-node-plugin-snmp-3.6.1-34.0.el7ev.noarch
sanlock-3.2.4-1.el7.x86_64
ovirt-vmconsole-host-1.0.4-1.el7ev.noarch
ovirt-node-lib-config-3.6.1-34.0.el7ev.noarch
ovirt-node-3.6.1-34.0.el7ev.noarch
ovirt-hosted-engine-ha-1.3.5.8-1.el7ev.noarch
ovirt-host-deploy-offline-1.4.0-1.el7ev.x86_64
ovirt-setup-lib-1.0.1-1.el7ev.noarch
Linux version 3.10.0-514.el7.x86_64 (mockbuild.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC) ) #1 SMP Wed Oct 19 11:24:13 EDT 2016
Linux 3.10.0-514.el7.x86_64 #1 SMP Wed Oct 19 11:24:13 EDT 2016 x86_64 x86_64 x86_64 GNU/Linux
Red Hat Enterprise Virtualization Hypervisor release 7.3 (20161028.1.el7ev)

4-Redeployed second host as 4.0.6 RHEL7.3, to get mixed environment, components on RHEL host are:
rhevm-appliance-20161130.0-1.el7ev.noarch
mom-0.5.8-1.el7ev.noarch
ovirt-hosted-engine-setup-2.0.4.1-2.el7ev.noarch
ovirt-host-deploy-1.5.3-1.el7ev.noarch
ovirt-engine-sdk-python-3.6.9.1-1.el7ev.noarch
ovirt-imageio-daemon-0.4.0-0.el7ev.noarch
libvirt-client-2.0.0-10.el7_3.2.x86_64
rhev-release-4.0.6-8-001.noarch
sanlock-3.4.0-1.el7.x86_64
qemu-kvm-rhev-2.6.0-28.el7_3.2.x86_64
ovirt-hosted-engine-ha-2.0.6-1.el7ev.noarch
rhevm-appliance-20161130.0-1.el7ev.noarch
ovirt-setup-lib-1.0.2-1.el7ev.noarch
ovirt-imageio-common-0.3.0-0.el7ev.noarch
ovirt-vmconsole-1.0.4-1.el7ev.noarch
vdsm-4.18.21-1.el7ev.x86_64
ovirt-vmconsole-host-1.0.4-1.el7ev.noarch
Linux version 3.10.0-514.2.2.el7.x86_64 (mockbuild.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC) ) #1 SMP Wed Nov 16 13:15:13 EST 2016
Linux 3.10.0-514.2.2.el7.x86_64 #1 SMP Wed Nov 16 13:15:13 EST 2016 x86_64 x86_64 x86_64 GNU/Linux
Red Hat Enterprise Linux Server release 7.3 (Maipo)

OS Version:
RHEL - 7.3 - 7.el7
Kernel Version:
3.10.0 - 514.2.2.el7.x86_64
KVM Version:
2.6.0 - 28.el7_3.2
LIBVIRT Version:
libvirt-2.0.0-10.el7_3.2
VDSM Version:
vdsm-4.18.21-1.el7ev
SPICE Version:
0.12.4 - 19.el7
GlusterFS Version:
[N/A]
CEPH Version:
librbd1-0.94.5-1.el7

5-Followed the https://access.redhat.com/documentation/en/red-hat-virtualization/4.0/single/self-hosted-engine-guide/#Upgrading_the_Self-Hosted_Engine to upgrade my HE mixed (RHEVH&RHEL) environment environment.
Engine was running on 4.0.6 RHEL7.3 host, while it was SPM.
Engine had disk at the size of 200Gig, while was still in 3.6.10
# lsblk
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
vda    252:0    0   200G  0 disk 
├─vda1 252:1    0   200M  0 part /boot
├─vda2 252:2    0     2G  0 part [SWAP]
└─vda3 252:3    0 197.8G  0 part /
sr0     11:0    1  1024M  0 rom  
6-Upgraded my environment from 3.6.10 to 4.0.6.2-0.1.el7ev and then successfully rolled back to 3.6.10.

alma04 /]# hosted-engine --upgrade-appliance
[ INFO  ] Stage: Initializing
[ INFO  ] Stage: Environment setup
          During customization use CTRL-D to abort.
          Continuing will upgrade the engine VM running on this hosts deploying and configuring a new appliance.
          If your engine VM is already based on el7 you can also simply upgrade the engine there.
          This procedure will create a new disk on the hosted-engine storage domain and it will backup there the content of your current engine VM disk.
          The new el7 based appliance will be deployed over the existing disk destroying its content; at any time you will be able to rollback using the content of the backup disk.
          You will be asked to take a backup of the running engine and copy it to this host.
          The engine backup will be automatically injected and recovered on the new appliance.
          Are you sure you want to continue? (Yes, No)[Yes]: 
          Configuration files: []
          Log file: /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20170104234955-al49pm.log
          Version: otopi-1.5.2 (otopi-1.5.2-1.el7ev)
[ INFO  ] Stage: Environment packages setup
[ INFO  ] Stage: Programs detection
[ INFO  ] Stage: Environment setup
[ INFO  ] Checking maintenance mode
[ INFO  ] The engine VM is running on this host
[ INFO  ] Stage: Environment customization
[ INFO  ] Answer file successfully loaded
          Enter engine admin password: 
          Confirm engine admin password: 
[ INFO  ] Acquiring internal CA cert from the engine
[ INFO  ] The following CA certificate is going to be used, please immediately interrupt if not correct:
[ INFO  ] Issuer: C=US, O=qa.lab.tlv.redhat.com, CN=nsednev-he-1.qa.lab.tlv.redhat.com.28618, Subject: C=US, O=qa.lab.tlv.redhat.com, CN=nsednev-he-1.qa.lab.tlv.redhat.com.28618, Fingerprint (SHA-1): 8CACF3CA06BECBC9F4DCB2F4E74BF2EDED3EBEC5
[ INFO  ] Bridge ovirtmgmt already created
[ INFO  ] Checking SPM status on this host
[ INFO  ] Connecting to the Engine
[ INFO  ] This upgrade tool is running on the SPM host
[ INFO  ] Detecting available oVirt engine appliances
          The following appliance have been found on your system:
                [1] - The RHEV-M Appliance image (OVA) - 20161130.0-1.el7ev
                [2] - Directly select an OVA file
          Please select an appliance (1, 2) [1]: 
[ INFO  ] Verifying its sha1sum
[ INFO  ] Checking OVF archive content (could take a few minutes depending on archive size)
[ INFO  ] Checking OVF XML content (could take a few minutes depending on archive size)
[WARNING] OVF does not contain a valid image description, using default.
[ INFO  ] Connecting to the Engine
[ INFO  ] The hosted-engine storage domain has enough free space to contain a new backup disk.
          Please take a backup of the current engine running this command on the engine VM:
           engine-backup --mode=backup --archive-compressor=gzip --file=engine_backup.tar.gz --log=engine_backup.log
          Then copy the backup archive to this host and input here its path when ready.
          Please specify path to engine backup archive you would like to restore on the new appliance: /backup/backup
[ INFO  ] Validating backup file '/backup/backup'
[ INFO  ] The provided file contains also a DWH DB backup: it will be restored as well
[ INFO  ] The provided file contains also a Reports DB backup: it will be restored as well
[ INFO  ] Validating authentication plugins
[ INFO  ] Connecting to the Engine
[ INFO  ] '/backup/backup' is a sane backup file
[ INFO  ] Checking version requirements
[ INFO  ] Connecting to the Engine
[ INFO  ] All the datacenters and clusters are at a compatible level
          Please provide the FQDN you would like to use for the engine appliance.
          Note: This will be the FQDN of the engine VM you are now going to launch,
          it should not point to the base host or to any other existing machine.
          Engine VM FQDN: (leave it empty to skip):  []: nsednev-he-1.qa.lab.tlv.redhat.com
          Please provide the domain name you would like to use for the engine appliance.
          Engine VM domain: [qa.lab.tlv.redhat.com]
          Enter root password that will be used for the engine appliance (leave it empty to skip): 
          Confirm appliance root password: 
          How should the engine VM network be configured (DHCP, Static)[DHCP]? 
          Add lines for the appliance itself and for this host to /etc/hosts on the engine VM?
          Note: ensuring that this host could resolve the engine VM hostname is still up to you
          (Yes, No)[No] yes
[WARNING] Please take care that this will simply add an entry for this host under /etc/hosts on the engine VM. If in the past you added other entries there, recovering them is up to you.
[ INFO  ] Stage: Setup validation
[ INFO  ] Stage: Transaction setup
[ INFO  ] Stage: Misc configuration
[ INFO  ] Stage: Package installation
[ INFO  ] Stage: Misc configuration
[ INFO  ] Connecting to the Engine
[ INFO  ] Waiting for the engine to complete disk creation. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] Still waiting for engine VM backup disk to be created. This may take several minutes...
[ INFO  ] The engine VM backup disk is now ready
[ INFO  ] Shutting down the current engine VM
[ INFO  ] Creating a backup of the engine VM disk (could take a few minutes depending on archive size)

[ INFO  ] Successfully created
[ INFO  ] Extracting disk image from OVF archive (could take a few minutes depending on archive size)
[ INFO  ] Validating pre-allocated volume size
[ INFO  ] Uploading volume to data domain (could take a few minutes depending on archive size)
[ INFO  ] Injecting engine backup
[ INFO  ] Backup successfully injected
[ INFO  ] Image successfully imported from OVF
[ INFO  ] Stage: Transaction commit
[ INFO  ] Stage: Closing up
[ INFO  ] Trying to get a fresher copy of vm configuration from the OVF_STORE
[ INFO  ] Found an OVF for HE VM, trying to convert
[ INFO  ] Got vm.conf from OVF_STORE
[ INFO  ] Running engine-setup on the appliance
          |- Preparing to restore:
          |- - Unpacking file '/root/backup'
          |- ------------------------------------------------------------------------------
          |- Please note:
          |- A Reports dump was found inside the backup.
          |- This version does not support Reports, and the dump and configuration files
          |- will not be restored.
          |- ------------------------------------------------------------------------------
          |- Restoring:
          |- - Files
          |- ------------------------------------------------------------------------------
          |- Please note:
          |- Operating system is different from the one used during backup.
          |- Current operating system: redhat7
          |- Operating system at backup: redhat6
          |- Apache httpd configuration will not be restored.
          |- You will be asked about it on the next engine-setup run.
          |- ------------------------------------------------------------------------------
          |- Provisioning PostgreSQL users/databases:
          |- - user 'engine', database 'engine'
          |- - user 'ovirt_engine_history', database 'ovirt_engine_history'
          |- Restoring:
          |- - Engine database 'engine'
          |-   - Cleaning up temporary tables in engine database 'engine'
          |-   - Resetting DwhCurrentlyRunning in dwh_history_timekeeping in engine database
          |- ------------------------------------------------------------------------------
          |- Please note:
          |- The engine database was backed up at 2017-01-04 16:56:08.000000000 -0500 .
          |- Objects that were added, removed or changed after this date, such as virtual
          |- machines, disks, etc., are missing in the engine, and will probably require
          |- recovery or recreation.
          |- ------------------------------------------------------------------------------
          |- - DWH database 'ovirt_engine_history'
          |- You should now run engine-setup.
          |- Done.
          |- HE_APPLIANCE_ENGINE_RESTORE_SUCCESS
[ INFO  ] Engine backup successfully restored
          |- [ INFO  ] Stage: Initializing
          |- [ INFO  ] Stage: Environment setup
          |-           Configuration files: ['/etc/ovirt-engine-setup.conf.d/10-packaging-wsp.conf', '/etc/ovirt-engine-setup.conf.d/10-packaging.conf', '/etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf', '/root/ovirt-engine-answers', '/root/heanswers.conf']
          |-           Log file: /var/log/ovirt-engine/setup/ovirt-engine-setup-20170104183554-n3qpwx.log
          |-           Version: otopi-1.5.2 (otopi-1.5.2-1.el7ev)
          |- [ INFO  ] Stage: Environment packages setup
          |- [ INFO  ] Stage: Programs detection
          |- [ INFO  ] Stage: Environment setup
          |- [ INFO  ] Stage: Environment customization
          |-          
          |-           Welcome to the RHEV 4.0 setup/upgrade.
          |-           Please read the RHEV 4.0 install guide
          |-           https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Virtualization/4.0/html/Installation_Guide/index.html.
          |-           Please refer to the RHEV Upgrade Helper application
          |-           https://access.redhat.com/labs/rhevupgradehelper/ which will guide you in the upgrading process.
          |-           Would you like to proceed? (Yes, No) [Yes]: 
          |-          
          |-           --== PRODUCT OPTIONS ==--
          |-          
          |-           Configure Image I/O Proxy on this host? (Yes, No) [Yes]: 
          |-          
          |-           --== PACKAGES ==--
          |-          
          |-          
          |-           --== NETWORK CONFIGURATION ==--
          |-          
          |- [ INFO  ] firewalld will be configured as firewall manager.
          |-          
          |-           --== DATABASE CONFIGURATION ==--
          |-          
          |-           The detected DWH database size is 25 MB.
          |-           Setup can backup the existing database. The time and space required for the database backup depend on its size. This process takes time, and in some cases (for instance, when the size is few GBs) may take several hours to complete.
          |-           If you choose to not back up the database, and Setup later fails for some reason, it will not be able to restore the database and all DWH data will be lost.
          |-           Would you like to backup the existing database before upgrading it? (Yes, No) [Yes]: 
          |-          
          |-           --== OVIRT ENGINE CONFIGURATION ==--
          |-          
          |-          
          |-           --== STORAGE CONFIGURATION ==--
          |-          
          |-          
          |-           --== PKI CONFIGURATION ==--
          |-          
          |-          
          |-           --== APACHE CONFIGURATION ==--
          |-          
          |-          
          |-           --== SYSTEM CONFIGURATION ==--
          |-          
          |-          
          |-           --== MISC CONFIGURATION ==--
          |-          
          |-          
          |-           --== END OF CONFIGURATION ==--
          |-          
          |- [ INFO  ] Stage: Setup validation
          |- [ INFO  ] Hosted Engine HA is in Global Maintenance mode.
          |- [ INFO  ] Cleaning stale zombie tasks and commands
          |-          
          |-           --== CONFIGURATION PREVIEW ==--
          |-          
          |-           Application mode                        : both
          |-           Default SAN wipe after delete           : False
          |-           Firewall manager                        : firewalld
          |-           Update Firewall                         : True
          |-           Host FQDN                               : nsednev-he-1.qa.lab.tlv.redhat.com
          |-           Engine database secured connection      : False
          |-           Engine database host                    : localhost
          |-           Engine database user name               : engine
          |-           Engine database name                    : engine
          |-           Engine database port                    : 5432
          |-           Engine database host name validation    : False
          |-           DWH database secured connection         : False
          |-           DWH database host                       : localhost
          |-           DWH database user name                  : ovirt_engine_history
          |-           DWH database name                       : ovirt_engine_history
          |-           DWH database port                       : 5432
          |-           DWH database host name validation       : False
          |-           Engine installation                     : True
          |-           PKI organization                        : qa.lab.tlv.redhat.com
          |-           Configure local Engine database         : True
          |-           Set application as default page         : True
          |-           Configure Apache SSL                    : True
          |-           DWH installation                        : True
          |-           Backup DWH database                     : True
          |-           Engine Host FQDN                        : nsednev-he-1.qa.lab.tlv.redhat.com
          |-           Configure Image I/O Proxy               : True
          |-           Configure VMConsole Proxy               : True
          |-           Configure WebSocket Proxy               : True
          |- [ INFO  ] Cleaning async tasks and compensations
          |- [ INFO  ] Unlocking existing entities
          |- [ INFO  ] Checking the Engine database consistency
          |- [ INFO  ] Stage: Transaction setup
          |- [ INFO  ] Stopping engine service
          |- [ INFO  ] Stopping ovirt-fence-kdump-listener service
          |- [ INFO  ] Stopping dwh service
          |- [ INFO  ] Stopping Image I/O Proxy service
          |- [ INFO  ] Stopping websocket-proxy service
          |- [ INFO  ] Stage: Misc configuration
          |- [ INFO  ] Stage: Package installation
          |- [ INFO  ] Stage: Misc configuration
          |- [ INFO  ] Upgrading CA
          |- [ INFO  ] Backing up database localhost:engine to '/var/lib/ovirt-engine/backups/engine-20170104183603.NDQBSP.dump'.
          |- [ INFO  ] Creating/refreshing Engine database schema
          |- [ INFO  ] Backing up database localhost:ovirt_engine_history to '/var/lib/ovirt-engine-dwh/backups/dwh-20170104183628.0MAEbi.dump'.
          |- [ INFO  ] Creating/refreshing DWH database schema
          |- [ INFO  ] Configuring Image I/O Proxy
          |- [ INFO  ] Configuring WebSocket Proxy
          |- [ INFO  ] Creating/refreshing Engine 'internal' domain database schema
          |- [ INFO  ] Generating post install configuration file '/etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf'
          |- [ INFO  ] Stage: Transaction commit
          |- [ INFO  ] Stage: Closing up
          |- [ INFO  ] Starting engine service
          |- [ INFO  ] Starting dwh service
          |- [ INFO  ] Restarting ovirt-vmconsole proxy service
          |-          
          |-           --== SUMMARY ==--
          |-          
          |- [ INFO  ] Restarting httpd
          |-           Web access is enabled at:
          |-               http://nsednev-he-1.qa.lab.tlv.redhat.com:80/ovirt-engine
          |-               https://nsednev-he-1.qa.lab.tlv.redhat.com:443/ovirt-engine
          |-           Internal CA 8C:AC:F3:CA:06:BE:CB:C9:F4:DC:B2:F4:E7:4B:F2:ED:ED:3E:BE:C5
          |-           SSH fingerprint: bb:d1:59:ad:85:44:1d:f1:d4:71:8f:b8:33:59:11:db
          |-          
          |-           --== END OF SUMMARY ==--
          |-          
          |- [ INFO  ] Stage: Clean up
          |-           Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20170104183554-n3qpwx.log
          |- [ INFO  ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20170104183759-setup.conf'
          |- [ INFO  ] Stage: Pre-termination
          |- [ INFO  ] Stage: Termination
          |- [ INFO  ] Execution of setup completed successfully
          |- HE_APPLIANCE_ENGINE_SETUP_SUCCESS
[ INFO  ] Engine-setup successfully completed 
[ INFO  ] Engine is still unreachable
[ INFO  ] Engine is still not reachable, waiting...
[ INFO  ] Engine is still unreachable
[ INFO  ] Engine is still not reachable, waiting...
[ INFO  ] Engine replied: DB Up!Welcome to Health Status!
[ INFO  ] Connecting to the Engine
[ INFO  ] Connecting to the Engine
[ INFO  ] Registering the hosted-engine backup disk in the DB
[ INFO  ] Waiting for the engine to complete disk registration. This may take several minutes...
[ INFO  ] The engine VM backup disk is now ready
[ INFO  ] Shutting down the engine VM
[ INFO  ] Stage: Clean up
[ INFO  ] Stage: Pre-termination
[ INFO  ] Stage: Termination
[ INFO  ] Hosted Engine successfully upgraded
[ INFO  ] Please exit global maintenance mode to restart the engine VM.
[ INFO  ] Please note that the engine VM ssh keys have changed. Please remove the engine VM entry in ssh known_hosts on your clients.

7-Got the engine with these components:
rhevm-dependencies-4.0.0-1.el7ev.noarch
rhevm-spice-client-x86-msi-4.0-3.el7ev.noarch
rhevm-4.0.6.2-0.1.el7ev.noarch
rhevm-spice-client-x64-msi-4.0-3.el7ev.noarch
rhevm-doc-4.0.6-1.el7ev.noarch
rhev-guest-tools-iso-4.0-6.el7ev.noarch
rhevm-branding-rhev-4.0.0-6.el7ev.noarch
rhevm-setup-plugins-4.0.0.3-1.el7ev.noarch
rhevm-guest-agent-common-1.0.12-3.el7ev.noarch
Linux version 3.10.0-514.el7.x86_64 (mockbuild.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC) ) #1 SMP Wed Oct 19 11:24:13 EDT 2016
Linux 3.10.0-514.el7.x86_64 #1 SMP Wed Oct 19 11:24:13 EDT 2016 x86_64 x86_64 x86_64 GNU/Linux
Red Hat Enterprise Linux Server release 7.3 (Maipo)

8-Set hosts to global maintenance.

9-Stopped engine's VM and verified that it is not running on any HE-host.

10-Successfully executed rollback-upgrade back to 3.5.6 old engine:
alma04 /]# hosted-engine --rollback-upgrade
[ INFO  ] Stage: Initializing
[ INFO  ] Stage: Environment setup
          During customization use CTRL-D to abort.
          Continuing will rollback the engine VM from a previous upgrade attempt.
          This procedure will restore an engine VM image from a backup taken during an upgrade attempt.
          The result of any action occurred after the backup creation instant could be definitively lost.
          Are you sure you want to continue? (Yes, No)[Yes]: 
          Configuration files: []
          Log file: /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20170105020238-fgyioe.log
          Version: otopi-1.5.2 (otopi-1.5.2-1.el7ev)
[ INFO  ] Stage: Environment packages setup
[ INFO  ] Stage: Programs detection
[ INFO  ] Stage: Environment setup
[ INFO  ] Checking maintenance mode
[ INFO  ] The engine VM is down.
[ INFO  ] Stage: Environment customization
[ INFO  ] Answer file successfully loaded
          The following backup disk have been found on your system:
                [1] - hosted-engine-backup-20170105000015
                [2] - hosted-engine-backup-20170104203956
          Please select one of them  (1, 2) [1]: 
[ INFO  ] Bridge ovirtmgmt already created
[ INFO  ] Stage: Setup validation
[ INFO  ] Stage: Transaction setup
[ INFO  ] Stage: Misc configuration
[ INFO  ] Stage: Package installation
[ INFO  ] Stage: Misc configuration
[ INFO  ] Restoring a backup of the engine VM disk (could take a few minutes depending on archive size)
[ INFO  ] Successfully restored
[ INFO  ] Stage: Transaction commit
[ INFO  ] Stage: Closing up
[ INFO  ] Stage: Clean up
[ INFO  ] Stage: Pre-termination
[ INFO  ] Stage: Termination
[ INFO  ] Hosted Engine successfully rolled back
[ INFO  ] Please exit global maintenance mode to restart the engine VM.
[ INFO  ] Please note that the engine VM ssh keys have changed. Please remove the engine VM entry in ssh known_hosts on your clients.

11-Returned hosts from global maintenance to none.
12-Logged in successfully to WEBADMIN of  Red Hat Enterprise Virtualization Manager Version: 3.5.8-0.1.el6ev.

Comment 3 Nikolai Sednev 2017-01-05 01:09:20 UTC
Mistyped in previous comment, fixing here.
12- Logged in successfully to WEBADMIN of Red Hat Enterprise Virtualization Manager Version: 3.6.10.2-0.2.el6


Note You need to log in before you can comment on or make changes to this bug.