Description of problem: If during engine-setup we create an iso domain, and later fail and rollback, we remove the files we created there, but do not remove directories. So these are left behind. A next attempt to run engine-setup with the answer file created by the first one will try to create an iso domain in the same location. It checks if it exists, and if so, uses the directory's name found there as the uuid of the domain. Later, if such a uuid is defined, we do not populate it with actual files (including metadata), thus leaving it empty. engine-setup will finish successfully, but when actually trying to attach it, this will fail, with: engine.log: 2016-01-28 15:27:34,235 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (org.ovirt.thread.pool-7-thread-26) [fd1aad0] Command 'AttachStorageDomainVDSCommand( AttachStorageDomainVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-00000000023b', ignoreFailoverLimit='false', storageDomainId='3b1a2f5b-f807-4e09-86fa-a31c9e59305c'})' execution failed: IRSGenericException: IRSErrorException: Failed to AttachStorageDomainVDS, error = Error in storage domain action: (u'sdUUID=3b1a2f5b-f807-4e09-86fa-a31c9e59305c, spUUID=00000001-0001-0001-0001-00000000023b',), code = 350 2016-01-28 15:27:34,235 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (org.ovirt.thread.pool-7-thread-26) [fd1aad0] FINISH, AttachStorageDomainVDSCommand, log id: 2d2f5ac6 2016-01-28 15:27:34,235 ERROR [org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand] (org.ovirt.thread.pool-7-thread-26) [fd1aad0] Command 'org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand' failed: EngineException: org.ovirt.engine.core.vdsbroker.irsbroker.IrsOperationFailedNoFailoverException: IRSGenericException: IRSErrorException: Failed to AttachStorageDomainVDS, error = Error in storage domain action: (u'sdUUID=3b1a2f5b-f807-4e09-86fa-a31c9e59305c, spUUID=00000001-0001-0001-0001-00000000023b',), code = 350 (Failed with error StorageDomainActionError and code 350) vdsm.log: jsonrpc.Executor/7::DEBUG::2016-01-28 15:27:33,168::fileSD::159::Storage.StorageDomainManifest::(__init__) Reading domain in path /rhev/data-center/mnt/didi-f19-engine.eng.lab.tlv.redhat.com:_var_lib_exports_iso/3b1a2f5b-f807-4e09-86fa-a31c9e59305c jsonrpc.Executor/7::DEBUG::2016-01-28 15:27:33,169::__init__::318::IOProcessClient::(_run) Starting IOProcess... jsonrpc.Executor/7::DEBUG::2016-01-28 15:27:33,198::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend jsonrpc.Executor/7::DEBUG::2016-01-28 15:27:33,202::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=[] jsonrpc.Executor/7::DEBUG::2016-01-28 15:27:33,203::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata jsonrpc.Executor/7::ERROR::2016-01-28 15:27:33,203::task::867::Storage.TaskManager.Task::(_setError) Task=`130a4b02-138c-401b-b618-31f50760c333`::Unexpected error Traceback (most recent call last): File "/usr/share/vdsm/storage/task.py", line 874, in _run return fn(*args, **kargs) File "/usr/share/vdsm/logUtils.py", line 49, in wrapper res = f(*args, **kwargs) File "/usr/share/vdsm/storage/hsm.py", line 1210, in attachStorageDomain pool.attachSD(sdUUID) File "/usr/share/vdsm/storage/securable.py", line 77, in wrapper return method(self, *args, **kwargs) File "/usr/share/vdsm/storage/sp.py", line 884, in attachSD dom = sdCache.produce(sdUUID) File "/usr/share/vdsm/storage/sdc.py", line 100, in produce domain.getRealDomain() File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain return self._cache._realProduce(self._sdUUID) File "/usr/share/vdsm/storage/sdc.py", line 124, in _realProduce domain = self._findDomain(sdUUID) File "/usr/share/vdsm/storage/sdc.py", line 143, in _findDomain dom = findMethod(sdUUID) File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) File "/usr/share/vdsm/storage/fileSD.py", line 330, in __init__ manifest = self.manifestClass(domainPath) File "/usr/share/vdsm/storage/fileSD.py", line 168, in __init__ sd.StorageDomainManifest.__init__(self, sdUUID, domaindir, metadata) File "/usr/share/vdsm/storage/sd.py", line 310, in __init__ self._domainLock = self._makeDomainLock() File "/usr/share/vdsm/storage/sd.py", line 439, in _makeDomainLock domVersion = self.getVersion() File "/usr/share/vdsm/storage/sd.py", line 359, in getVersion return self.getMetaParam(DMDK_VERSION) File "/usr/share/vdsm/storage/sd.py", line 356, in getMetaParam return self._metadata[key] File "/usr/share/vdsm/storage/persistentDict.py", line 89, in __getitem__ return dec(self._dict[key]) File "/usr/share/vdsm/storage/persistentDict.py", line 201, in __getitem__ raise KeyError(key) KeyError: 'VERSION' The web interface will show: VDSM command failed: Error in storage domain action: (u'sdUUID=3b1a2f5b-f807-4e09-86fa-a31c9e59305c, spUUID=00000001-0001-0001-0001-00000000023b',) Failed to attach Storage Domain ISO_DOMAIN to Data Center Default. (User: admin@internal) Version-Release number of selected component (if applicable): Current master. engine-setup behavior is way older, not sure about ui/engine/vdsm. How reproducible: Always I think Steps to Reproduce: 1. engine-setup, choose to create an nfs iso domain 2. fail engine-setup after it prepared the iso domain 3. run engine-setup again with the answerfile generated by 1. Actual results: engine-setup finishes successfully, leaving an empty domain, attaching it fails Expected results: I guess engine-setup should try to also remove the empty directories it created during rollback. It's probably not very easy to do properly with current transaction rollback and current filetransaction. Additional info: Workarounds: If between (2.) and (3.) above, remove the iso domain If after (3.), destroy the iso domain from the web interface and create a new one.
Moving from 4.0 alpha to 4.0 beta since 4.0 alpha has been already released and bug is not ON_QA.
oVirt 4.0 beta has been released, moving to RC milestone.
We will not longer allow users to define a ISO domain in the setup, therefore closing this bug.
This change is from 4.0 and on.