Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1562349

Summary: Upgrade appliance failed with 'Environment customization': 'OVEHOSTED_STORAGE/spUUID'
Product: [oVirt] ovirt-hosted-engine-setup Reporter: Yihui Zhao <yzhao>
Component: GeneralAssignee: Ido Rosenzwig <irosenzw>
Status: CLOSED CURRENTRELEASE QA Contact: meital avital <mavital>
Severity: high Docs Contact:
Priority: unspecified    
Version: 2.2.14CC: bugs, cshao, huzhao, jiaczhan, phbailey, qiyuan, rbarry, sbonazzo, stirabos, weiwang, yaniwang, ycui, yturgema
Target Milestone: ovirt-4.2.2Flags: rule-engine: ovirt-4.2+
Target Release: 2.2.16   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: ovirt-hosted-engine-setup-2.2.16-1.el7ev Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-04-18 12:25:29 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Integration RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1458709    
Attachments:
Description Flags
answers
none
setup.log none

Description Yihui Zhao 2018-03-30 09:45:10 UTC
Description of problem: 
Upgrade appliance failed with 'Environment customization': 'OVEHOSTED_STORAGE/spUUID'

From the CLI console:
[root@ibm-x3650m5-06 opt]# hosted-engine --upgrade-appliance
[ INFO  ] Stage: Initializing
[ INFO  ] Stage: Environment setup
          During customization use CTRL-D to abort.
         
          ==================================================================================
          Welcome to the Self Hosted Engine Setup/Upgrade Tool.
         
          On new deployment, please refer to the Red Hat Virtualization Installation Guide:
          https://access.redhat.com/documentation/en/red-hat-virtualization/4.1/paged/self-hosted-engine-guide/chapter-2-deploying-self-hosted-engine
         
          On upgrades, please refer to the Red Hat Virtualization Upgrade Helper:
          https://access.redhat.com/labs/rhevupgradehelper/
          And specifically for migration between of the engine VM between 3.6/el6 and 4.0/el7:
          https://access.redhat.com/documentation/en/red-hat-virtualization/4.0/single/self-hosted-engine-guide/#Upgrading_the_Self-Hosted_Engine
          ==================================================================================
          Continuing will upgrade the engine VM running on this hosts deploying and configuring a new appliance.
          If your engine VM is already based on el7 you can also simply upgrade the engine there.
          This procedure will create a new disk on the hosted-engine storage domain and it will backup there the content of your current engine VM disk.
          The new el7 based appliance will be deployed over the existing disk destroying its content; at any time you will be able to rollback using the content of the backup disk.
          You will be asked to take a backup of the running engine and copy it to this host.
          The engine backup will be automatically injected and recovered on the new appliance.
          Are you sure you want to continue? (Yes, No)[Yes]: 
          Configuration files: []
          Log file: /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20180330173438-av2erz.log
          Version: otopi-1.7.7 (otopi-1.7.7-1.el7ev)
[ INFO  ] Detecting available oVirt engine appliances
[ INFO  ] Stage: Environment packages setup
[ INFO  ] Stage: Programs detection
[ INFO  ] Stage: Environment setup
[ INFO  ] Checking maintenance mode
[ INFO  ] The engine VM is running on this host
[ INFO  ] Stage: Environment customization
         
          --== STORAGE CONFIGURATION ==--
         
[ ERROR ] Failed to execute stage 'Environment customization': 'OVEHOSTED_STORAGE/spUUID'
[ INFO  ] Stage: Clean up
[ INFO  ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answers/answers-20180330173441.conf'
[ INFO  ] Stage: Pre-termination
[ INFO  ] Stage: Termination
[ ERROR ] Hosted Engine upgrade failed
          Log file is located at /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20180330173438-av2erz.log


Version-Release number of selected component (if applicable): 
rhvh-4.2.2.0-0.20180322.0+1
cockpit-ovirt-dashboard-0.11.19-1.el7ev.noarch
ovirt-hosted-engine-ha-2.2.9-1.el7ev.noarch
ovirt-hosted-engine-setup-2.2.15-1.el7ev.noarch
rhvm-appliance-4.2-20180202.0.el7.4.2.rpm



How reproducible: 
100%

Steps to Reproduce: 
1. Deploy HE with nfs storage via cockpit
2. Upgrade rhvm-appliance pkg from rhvm-appliance-4.2-20180202.0.el7.4.2.rpm to rhvm-appliance-4.2-20180322.0.el7.noarch
#rpm -Uvh  rhvm-appliance-4.2-20180322.0.el7.4.2.rpm
3. Set global maintenance
4. "hosted-engine --upgrade-appliance"
  
Actual results:  
The same as the description. 

Expected results: 
Upgrade the appliance successfully.

Additional info:

Comment 1 Yihui Zhao 2018-03-30 09:47:33 UTC
Created attachment 1415056 [details]
answers

Comment 2 Yihui Zhao 2018-03-30 09:48:01 UTC
Created attachment 1415057 [details]
setup.log

Comment 3 Simone Tiraboschi 2018-03-30 15:17:14 UTC
(In reply to Yihui Zhao from comment #0)
>           And specifically for migration between of the engine VM between
> 3.6/el6 and 4.0/el7:

This is a really a corner case since the upgrade was was just intended for 3.6/el6 -> 4.0/el7 upgrades of the engine VM and here the system has been deployed with 4.2 from cockpit with ansible.

Still better to make it working or clearly prevent it.

Comment 4 Yihui Zhao 2018-04-10 10:50:50 UTC
Tested with these versions:
ovirt-hosted-engine-setup-2.2.16

The upgrade flow just support 3.6/el6 to 4.x/el7.

From the CLI:
[root@ibm-x3650m5-05 opt]# hosted-engine --upgrade-appliance
[ INFO  ] Stage: Initializing
[ INFO  ] Stage: Environment setup
          During customization use CTRL-D to abort.
         
          ==================================================================================
          Welcome to the Self Hosted Engine Setup/Upgrade Tool.
         
          On new deployment, please refer to the Red Hat Virtualization Installation Guide:
          https://access.redhat.com/documentation/en/red-hat-virtualization/4.1/paged/self-hosted-engine-guide/chapter-2-deploying-self-hosted-engine
         
          On upgrades, please refer to the Red Hat Virtualization Upgrade Helper:
          https://access.redhat.com/labs/rhevupgradehelper/
          And specifically for migration between of the engine VM between 3.6/el6 and 4.0/el7:
          https://access.redhat.com/documentation/en/red-hat-virtualization/4.0/single/self-hosted-engine-guide/#Upgrading_the_Self-Hosted_Engine
          ==================================================================================
          Continuing will upgrade the engine VM running on this hosts deploying and configuring a new appliance.
          If your engine VM is already based on el7 you can also simply upgrade the engine there.
          This procedure will create a new disk on the hosted-engine storage domain and it will backup there the content of your current engine VM disk.
          The new el7 based appliance will be deployed over the existing disk destroying its content; at any time you will be able to rollback using the content of the backup disk.
          You will be asked to take a backup of the running engine and copy it to this host.
          The engine backup will be automatically injected and recovered on the new appliance.
          Are you sure you want to continue? (Yes, No)[Yes]: 
          Configuration files: []
          Log file: /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20180410184254-50gwkw.log
          Version: otopi-1.7.7 (otopi-1.7.7-1.el7ev)
[ INFO  ] Detecting available oVirt engine appliances
[ INFO  ] Stage: Environment packages setup
[ INFO  ] Stage: Programs detection
[ INFO  ] Stage: Environment setup
[ INFO  ] Checking maintenance mode
[ INFO  ] The engine VM is running on this host
[ INFO  ] Stage: Environment customization
         
          --== STORAGE CONFIGURATION ==--
         
[ INFO  ] Answer file successfully loaded
[ INFO  ] Acquiring internal CA cert from the engine
[ INFO  ] The following CA certificate is going to be used, please immediately interrupt if not correct:
[ INFO  ] Issuer: C=US, O=lab.eng.pek2.redhat.com, CN=rhevh-hostedengine-vm-05.lab.eng.pek2.redhat.com.79677, Subject: C=US, O=lab.eng.pek2.redhat.com, CN=rhevh-hostedengine-vm-05.lab.eng.pek2.redhat.com.79677, Fingerprint (SHA-1): 9708B0E3A2FB29A9724C9176623DF6C9D13CFAD9
         
          --== HOST NETWORK CONFIGURATION ==--
         
[ INFO  ] Checking SPM status on this host
[ INFO  ] Connecting to Engine
          Enter engine admin username [admin@internal]: 
          Enter engine admin password: 
[ INFO  ] Connecting to Engine
[ INFO  ] This upgrade tool is running on the SPM host
[ INFO  ] Bridge ovirtmgmt already created
[WARNING] Unable to uniquely detect the interface where Bridge ovirtmgmt has been created on, [u'eno1', u'vnet0'] appear to be valid alternatives
         
          --== VM CONFIGURATION ==--
         
          The following appliance have been found on your system:
          	[1] - The RHV-M Appliance image (OVA) - 4.2-20180404.0.el7
          	[2] - Directly select an OVA file
          Please select an appliance (1, 2) [1]: 
"""

So, change this bug' status to verified.

Comment 5 Sandro Bonazzola 2018-04-18 12:25:29 UTC
This bugzilla is included in oVirt 4.2.2 release, published on March 28th 2018.

Since the problem described in this bug report should be
resolved in oVirt 4.2.2 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.