Bug 1291234 - with 3.6.1, on additional hosts, the deploy fails with 'RuntimeError: Dirty Storage Domain: Cannot find master domain...'
with 3.6.1, on additional hosts, the deploy fails with 'RuntimeError: Dirty ...
Status: CLOSED CURRENTRELEASE
Product: ovirt-hosted-engine-setup
Classification: oVirt
Component: Plugins.General (Show other bugs)
1.3.1
Unspecified Unspecified
urgent Severity urgent (vote)
: ovirt-3.6.1
: 1.3.1.3
Assigned To: Simone Tiraboschi
Nikolai Sednev
integration
: Triaged
Depends On: 1291736
Blocks: 1284979 RHEV3.6Upgrade
  Show dependency treegraph
 
Reported: 2015-12-14 06:35 EST by Simone Tiraboschi
Modified: 2016-03-07 15:22 EST (History)
13 users (show)

See Also:
Fixed In Version: ovirt-hosted-engine-setup-1.3.1.3-1
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-12-16 07:20:17 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
rule-engine: ovirt‑3.6.z+
rule-engine: blocker+
ylavi: planning_ack+
sbonazzo: devel_ack+
mavital: testing_ack+


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 50415 master MERGED packaging: setup: fixing _storagePoolConnection on additional hosts Never
oVirt gerrit 50417 ovirt-hosted-engine-setup-1.3 MERGED packaging: setup: fixing _storagePoolConnection on additional hosts Never
oVirt gerrit 50509 ovirt-hosted-engine-setup-1.3.1.z MERGED packaging: setup: fixing _storagePoolConnection on additional hosts Never

  None (edit)
Description Simone Tiraboschi 2015-12-14 06:35:51 EST
Description of problem:

Before 3.6.1 _storagePoolConnection was not required on additional hosts since the hosted-engine storage domain wasn't connected to any storage pool.

With 3.6.1 the engine is able to import the hosted-engine storage domain and attach it to its datacenter storagePool.
So _storagePoolConnection is needed again on additional hosts and we have to fix it since the master storage domain is no the bootstrap fake storage domain but the hosted-engine-storage domain.

Version-Release number of selected component (if applicable):
ovirt-hosted-engine-setup.noarch      1.3.1.2-1.el7.centos        @ovirt-3.6-pre

How reproducible:
100%

Steps to Reproduce:
1. deploy hosted-engine on first host, use at least 3.6.1 RC4 
2. ensure that the hosted-engine storage domain got imported and activated within the engine 
3. try to deploy an additional host

Actual results:
It's not possible to deploy additional hosts, the setup fails with:
2015-12-14 11:40:02 INFO otopi.plugins.ovirt_hosted_engine_setup.storage.storage storage._handleHostId:249 Installing on additional host
2015-12-14 11:40:02 DEBUG otopi.plugins.otopi.dialog.human human.queryString:156 query OVEHOSTED_HOST_ID
2015-12-14 11:40:02 DEBUG otopi.plugins.otopi.dialog.human dialog.__logString:219 DIALOG:SEND                 Please specify the Host ID [Must be integer, default: 2]: 
2015-12-14 11:40:05 DEBUG otopi.plugins.otopi.dialog.human dialog.__logString:219 DIALOG:RECEIVE    3
2015-12-14 11:40:05 DEBUG otopi.plugins.ovirt_hosted_engine_setup.storage.storage storage._storagePoolConnection:922 connectStoragePool
2015-12-14 11:40:05 DEBUG otopi.context context._executeMethod:156 method exception
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/otopi/context.py", line 146, in _executeMethod
    method['method']()
  File "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/ovirt-hosted-engine-setup/storage/storage.py", line 1265, in _late_customization
    self._getExistingDomain()
  File "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/ovirt-hosted-engine-setup/storage/storage.py", line 559, in _getExistingDomain
    self._storagePoolConnection()
  File "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/ovirt-hosted-engine-setup/storage/storage.py", line 930, in _storagePoolConnection
    message=status['status']['message'],
RuntimeError: Dirty Storage Domain: Cannot find master domain: 'spUUID=00000001-0001-0001-0001-00000000009e, msdUUID=154ef8b4-fad7-44cb-b7e7-25af78a51f98'
Please clean the storage device and try again
2015-12-14 11:40:05 ERROR otopi.context context._executeMethod:165 Failed to execute stage 'Environment customization': Dirty Storage Domain: Cannot find master domain: 'spUUID=00000001-0001-0001-0001-00000000009e, msdUUID=154ef8b4-fad7-44cb-b7e7-25af78a51f98'
Please clean the storage device and try again


Expected results:
it correctly deploy the additional host.

Additional info:
The hosted-engine storage domain must be visible and active in the engine to trigger it.
Comment 1 Nikolai Sednev 2015-12-15 08:34:49 EST
Successfully added third host to the HE with 2 hosts running, met no errors as described within the bug.
Works for me on these components:
Host:
qemu-kvm-rhev-2.3.0-31.el7_2.5.x86_64
ovirt-vmconsole-host-1.0.0-1.el7ev.noarch
ovirt-host-deploy-1.4.1-1.el7ev.noarch
libvirt-client-1.2.17-13.el7_2.2.x86_64
vdsm-4.17.13-1.el7ev.noarch
sanlock-3.2.4-2.el7_2.x86_64
ovirt-setup-lib-1.0.0-1.el7ev.noarch
ovirt-vmconsole-1.0.0-1.el7ev.noarch
ovirt-hosted-engine-ha-1.3.3.3-1.el7ev.noarch
mom-0.5.1-1.el7ev.noarch
ovirt-hosted-engine-setup-1.3.1.3-1.el7ev.noarch
Linux version 3.10.0-327.el7.x86_64 (mockbuild@x86-034.build.eng.bos.redhat.com) (gcc version 4.8.3 20140911 (Red Hat 4.8.3-9) (GCC) ) #1 SMP Thu Oct 29 17:29:29 EDT 2015

Engine:
ovirt-host-deploy-java-1.4.1-1.el6ev.noarch
ovirt-vmconsole-1.0.0-1.el6ev.noarch
ovirt-host-deploy-1.4.1-1.el6ev.noarch
ovirt-vmconsole-proxy-1.0.0-1.el6ev.noarch
ovirt-engine-extension-aaa-jdbc-1.0.4-1.el6ev.noarch
rhevm-3.6.1.3-0.1.el6.noarch
Linux version 2.6.32-573.8.1.el6.x86_64 (mockbuild@x86-033.build.eng.bos.redhat.com) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-16) (GCC) ) #1 SMP Fri Sep 25 19:24:22 EDT 2015
Comment 2 Sandro Bonazzola 2015-12-16 07:20:17 EST
According to verification status and target milestone this issue should be fixed in oVirt 3.6.1. Closing current release.
Comment 3 Douglas Duckworth 2016-03-07 15:22:55 EST
Workaround:

In RHEV-M 3.5.7-0.1.el6ev on additional RHEV Hypervisor - 7.2 - 20160219.0.el7ev I was able to add additional Hosted-Engine Host by doing:

hosted-engine --set-maintenance --mode=global
hosted-engine  --vm-stop
hosted-engine  --deploy

Dirty Storage Domain message gone.

Note You need to log in before you can comment on or make changes to this bug.