+++ This bug was initially created as a clone of Bug #1163385 +++

Description of problem:

I have upgraded my oVirt 3.4 hosted engine cluster to oVirt 3.5 using the upgrade instructions on the Wiki.  Everything appears to be working fine after the upgrade.

However, I am now trying to add a new host to the hosted engine configuration but the hosted-engine --deploy fails after sshing the answers file from the upgraded primary configuration.  The following errors can be found in the setup log:


Answer file lacks lockspace UUIDs, please use an answer file generated from the same version you are using on this additional host

I confirmed that the answers file on the upgraded host does not have any lockspace UUIDs:

OVEHOSTED_STORAGE/storageDatacenterName=str:hosted_datacenter
OVEHOSTED_STORAGE/storageDomainName=str:hosted_storage
OVEHOSTED_STORAGE/storageType=none:None
OVEHOSTED_STORAGE/volUUID=str:da160775-07fe-4569-b45f-03be0c5896a5
OVEHOSTED_STORAGE/domainType=str:nfs3
OVEHOSTED_STORAGE/imgSizeGB=str:25
OVEHOSTED_STORAGE/storageDomainConnection=str:192.168.8.12:/mnt/data2/vm/engine
OVEHOSTED_STORAGE/connectionUUID=str:880093ea-b0c1-448d-ac55-cde99feebc23
OVEHOSTED_STORAGE/spUUID=str:5e7ff7c2-6e75-4ba8-a5cc-e8dc5d37e478
OVEHOSTED_STORAGE/imgUUID=str:c9466bb6-a78c-4caa-bce3-22c87a5f3f1a
OVEHOSTED_STORAGE/sdUUID=str:b12fd59c-380a-40b3-b7f2-02d455de1d3b

Is there something I can do to update the answers file on the updated 3.5 working host so this will work?

As an aside the new host is EL7 whereas the existing hosts are EL6.

Version-Release number of selected component (if applicable):
ovirt-hosted-engine-setup-1.2.1-1.el7.noarch

How reproducible:


Steps to Reproduce:
1. Install an oVirt 3.4 hosted-engine environment.
2. Upgrade environment to 3.5 using information in Wiki.
3. Attempt to add new hosted-engine host using hosted-engine --deploy

Actual results:
Adding new host fails with error:

Answer file lacks lockspace UUIDs, please use an answer file generated from the same version you are using on this additional host

Expected results:
Able to add new host to hosted-engine cluster.

Additional info:

Here is the relevant section of the hosted-engine setup log file:

2014-11-11 22:57:04 DEBUG otopi.context context._executeMethod:138 Stage validation METHOD otopi.plugins.ovirt_hosted_engine_setup.sanlock.lockspace.Plugin._validation
2014-11-11 22:57:04 DEBUG otopi.context context._executeMethod:152 method exception
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/otopi/context.py", line 142, in _executeMethod
    method['method']()
  File "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/ovirt-hosted-engine-setup/sanlock/lockspace.py", line 102, in _validation
    'Answer file lacks lockspace UUIDs, please use an '
RuntimeError: Answer file lacks lockspace UUIDs, please use an answer file generated from the same version you are using on this additional host
2014-11-11 22:57:04 ERROR otopi.context context._executeMethod:161 Failed to execute stage 'Setup validation': Answer file lacks lockspace UUIDs, please use an answer file generated from the same version you are using on this additional host
2014-11-11 22:57:04 DEBUG otopi.context context.dumpEnvironment:490 ENVIRONMENT DUMP - BEGIN
2014-11-11 22:57:04 DEBUG otopi.context context.dumpEnvironment:500 ENV BASE/error=bool:'True'
2014-11-11 22:57:04 DEBUG otopi.context context.dumpEnvironment:500 ENV BASE/exceptionInfo=list:'[(<type 'exceptions.RuntimeError'>, RuntimeError('Answer file lacks lockspace UUIDs, please use an answer file generated from the same version you are using on this additional host',), <traceback object at 0x34c85a8>)]'
2014-11-11 22:57:04 DEBUG otopi.context context.dumpEnvironment:504 ENVIRONMENT DUMP - END
2014-11-11 22:57:04 INFO otopi.context context.runSequence:417 Stage: Clean up
2014-11-11 22:57:04 DEBUG otopi.context context.runSequence:421 STAGE cleanup
2014-11-11 22:57:04 DEBUG otopi.context context._executeMethod:138 Stage cleanup METHOD otopi.plugins.ovirt_hosted_engine_setup.core.remote_answerfile.Plugin._cleanup
2014-11-11 22:57:04 DEBUG otopi.context context._executeMethod:138 Stage cleanup METHOD otopi.plugins.ovirt_hosted_engine_setup.engine.add_host.Plugin._cleanup
2014-11-11 22:57:04 DEBUG otopi.context context._executeMethod:138 Stage cleanup METHOD otopi.plugins.ovirt_hosted_engine_setup.pki.vdsmpki.Plugin._cleanup
2014-11-11 22:57:04 DEBUG otopi.context context._executeMethod:138 Stage cleanup METHOD otopi.plugins.ovirt_hosted_engine_setup.storage.storage.Plugin._cleanup
2014-11-11 22:57:04 DEBUG otopi.plugins.ovirt_hosted_engine_setup.storage.storage storage._spmStop:609 spmStop
2014-11-11 22:57:04 DEBUG otopi.plugins.ovirt_hosted_engine_setup.storage.storage storage._cleanup:970 Not SPM?
Traceback (most recent call last):
  File "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/ovirt-hosted-engine-setup/storage/storage.py", line 968, in _cleanup
    self._spmStop()
  File "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/ovirt-hosted-engine-setup/storage/storage.py", line 617, in _spmStop
    raise RuntimeError(status_uuid)
RuntimeError: Not SPM
2014-11-11 22:57:04 DEBUG otopi.plugins.ovirt_hosted_engine_setup.storage.storage storage._storagePoolConnection:580 disconnectStoragePool
2014-11-11 22:57:08 DEBUG otopi.context context._executeMethod:138 Stage cleanup METHOD otopi.plugins.ovirt_hosted_engine_setup.vm.boot_disk.Plugin._cleanup
2014-11-11 22:57:08 DEBUG otopi.context context._executeMethod:138 Stage cleanup METHOD otopi.plugins.ovirt_hosted_engine_setup.core.answerfile.Plugin._save_answers_at_cleanup
2014-11-11 22:57:08 INFO otopi.plugins.ovirt_hosted_engine_setup.core.answerfile answerfile._save_answers:48 Generating answer file '/etc/ovirt-hosted-engine/answers.conf'
2014-11-11 22:57:09 INFO otopi.plugins.ovirt_hosted_engine_setup.core.answerfile answerfile._save_answers_at_cleanup:104 Answer file '/etc/ovirt-hosted-engine/answers.conf' has been updated

--- Additional comment from Jiri Moskovcak on 2014-11-13 03:21:52 EST ---

In case of updating from 3.4 we need to use FileSystemBackend to handle the storage init while adding new host. 

idea:
if the answer file lacks lockspace uuid and the storage tyupe is nfs we can assume it's 3.4 answer file, so we should use FileSystemBackend