Attachment 641527 Details for Bug 874120 – Full engine.log

Full engine.log

engine.log (text/x-log), 866.76 KB, created by Valentin Bajrami on 2012-11-09 13:15:40 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Valentin Bajrami

Created: 2012-11-09 13:15:40 UTC

Size: 866.76 KB

patch

obsolete

>2012-11-07 08:30:00,088 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-48) [13a7e7e6] Lock freed to object EngineLock [exclusiveLocks= key: b1658676-2dc3-4357-b112-3852a056c663 value: STORAGE
>, sharedLocks= ]
>2012-11-07 08:30:00,089 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-48) [13a7e7e6] ActivateStorage Domain. Before Connect all hosts to pool. Time:11/7/12 8:30 AM
>2012-11-07 08:30:00,122 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-3-thread-47) [7317b880] START, ValidateStorageServerConnectionVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 039341b1-a8c0-4003-845a-4ef53f74c7c8, connection: localhost:/mnt/data };]), log id: 72c4f1e9
>2012-11-07 08:30:00,146 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-3-thread-47) [7317b880] FINISH, ValidateStorageServerConnectionVDSCommand, return: {039341b1-a8c0-4003-845a-4ef53f74c7c8=0}, log id: 72c4f1e9
>2012-11-07 08:30:00,146 INFO  [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-3-thread-47) [7317b880] Running command: ConnectStorageToVdsCommand internal: true. Entities affected :  ID: aaa00000-0000-0000-0000-123456789aaa Type: System
>2012-11-07 08:30:00,150 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-3-thread-47) [7317b880] START, ConnectStorageServerVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 039341b1-a8c0-4003-845a-4ef53f74c7c8, connection: localhost:/mnt/data };]), log id: 4c8e902d
>2012-11-07 08:30:17,408 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-76) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:30:17,410 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-76) Vds: Ovirt
>2012-11-07 08:30:17,410 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-76) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:30:17,411 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-76) FINISH, ConnectStoragePoolVDSCommand, log id: 51bc48
>2012-11-07 08:30:17,411 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-76) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:30:17,412 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-76) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:30:17,421 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-76) [445a8dd8] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:30:17,451 WARN  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-76) [445a8dd8] CanDoAction of action ReconstructMasterDomain failed. Reasons:VAR__ACTION__RECONSTRUCT_MASTER,VAR__TYPE__STORAGE__DOMAIN,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL2,$status Locked
>2012-11-07 08:30:27,495 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-82) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:30:27,508 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-82) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 6f004977
>2012-11-07 08:31:30,350 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-82) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:31:30,352 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-82) Vds: Ovirt
>2012-11-07 08:31:30,353 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-82) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:31:30,354 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-82) FINISH, ConnectStoragePoolVDSCommand, log id: 6f004977
>2012-11-07 08:31:30,354 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-82) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:31:30,355 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-82) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:31:30,363 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-82) [790472d] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:31:30,415 WARN  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-82) [790472d] CanDoAction of action ReconstructMasterDomain failed. Reasons:VAR__ACTION__RECONSTRUCT_MASTER,VAR__TYPE__STORAGE__DOMAIN,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL2,$status Locked
>2012-11-07 08:31:40,459 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-69) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:31:40,497 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-69) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 7ebb49f6
>2012-11-07 08:32:43,326 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-69) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:32:43,328 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-69) Vds: Ovirt
>2012-11-07 08:32:43,328 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-69) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:32:43,329 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-69) FINISH, ConnectStoragePoolVDSCommand, log id: 7ebb49f6
>2012-11-07 08:32:43,330 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-69) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:32:43,330 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-69) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:32:43,338 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-69) [44b56c5c] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:32:43,390 WARN  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-69) [44b56c5c] CanDoAction of action ReconstructMasterDomain failed. Reasons:VAR__ACTION__RECONSTRUCT_MASTER,VAR__TYPE__STORAGE__DOMAIN,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL2,$status Locked
>2012-11-07 08:32:53,438 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-68) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:32:53,474 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-68) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 434cdf44
>2012-11-07 08:33:00,152 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (pool-3-thread-47) [7317b880] XML RPC error in command ConnectStorageServerVDS ( Vds: Ovirt ), the error was: java.util.concurrent.TimeoutException, TimeoutException:  
>2012-11-07 08:33:00,153 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-3-thread-47) [7317b880] FINISH, ConnectStorageServerVDSCommand, log id: 4c8e902d
>2012-11-07 08:33:00,154 ERROR [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-3-thread-47) [7317b880] Command org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSNetworkException: 
>2012-11-07 08:33:00,158 ERROR [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-3-thread-47) [7317b880] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand.
>2012-11-07 08:33:00,159 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-48) [13a7e7e6] ActivateStorage Domain. After Connect all hosts to pool. Time:11/7/12 8:33 AM
>2012-11-07 08:33:00,160 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (QuartzScheduler_Worker-48) [13a7e7e6] START, ActivateStorageDomainVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, storageDomainId = b1658676-2dc3-4357-b112-3852a056c663), log id: 4858f894
>2012-11-07 08:33:31,145 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-32) starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-07 08:33:31,150 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-32) domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-07 08:33:31,155 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-32) [65e1d394] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:33:31,241 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-32) [65e1d394] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-07 08:33:31,263 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-32) [65e1d394] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-07 08:33:31,266 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-32) [65e1d394] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-07 08:33:31,267 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-32) [65e1d394] No string for UNASSIGNED type. Use default Log
>2012-11-07 08:33:32,953 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-36) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-07 08:33:56,321 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-68) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:33:56,322 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-68) Vds: Ovirt
>2012-11-07 08:33:56,323 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-68) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:33:56,324 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-68) FINISH, ConnectStoragePoolVDSCommand, log id: 434cdf44
>2012-11-07 08:33:56,324 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-68) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:33:56,325 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-68) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:33:56,333 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-68) [95ea5e3] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:33:56,366 WARN  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-68) [95ea5e3] CanDoAction of action ReconstructMasterDomain failed. Reasons:VAR__ACTION__RECONSTRUCT_MASTER,VAR__TYPE__STORAGE__DOMAIN,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL2,$status Locked
>2012-11-07 08:33:56,408 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-48) [13a7e7e6] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:33:56,422 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-48) [13a7e7e6] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 6a9a525a
>2012-11-07 08:34:59,288 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-48) [13a7e7e6] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:34:59,290 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-48) [13a7e7e6] Vds: Ovirt
>2012-11-07 08:34:59,291 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-48) [13a7e7e6] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:34:59,292 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-48) [13a7e7e6] FINISH, ConnectStoragePoolVDSCommand, log id: 6a9a525a
>2012-11-07 08:34:59,292 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-48) [13a7e7e6] IrsBroker::Failed::ActivateStorageDomainVDS
>2012-11-07 08:34:59,293 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-48) [13a7e7e6] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:34:59,301 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-48) [684fbf57] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:34:59,349 WARN  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-48) [684fbf57] CanDoAction of action ReconstructMasterDomain failed. Reasons:VAR__ACTION__RECONSTRUCT_MASTER,VAR__TYPE__STORAGE__DOMAIN,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL2,$status Locked
>2012-11-07 08:34:59,368 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (QuartzScheduler_Worker-48) [684fbf57] FINISH, ActivateStorageDomainVDSCommand, log id: 4858f894
>2012-11-07 08:34:59,368 ERROR [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-48) [684fbf57] Command org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.irsbroker.IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:34:59,373 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-48) [684fbf57] Command [id=35701d8d-b2d9-4d31-b9a5-8f0990ec4421]: Compensating CHANGED_STATUS_ONLY of org.ovirt.engine.core.common.businessentities.storage_pool_iso_map; snapshot: EntityStatusSnapshot [id=storagePoolId = c02503a4-274e-11e2-b33c-002590730631, storageId = b1658676-2dc3-4357-b112-3852a056c663, status=InActive].
>2012-11-07 08:34:59,388 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-48) [684fbf57] Checking autorecoverable storage domains done
>2012-11-07 08:34:59,391 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-94) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:34:59,409 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-94) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: e442cf4
>2012-11-07 08:35:00,001 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-75) Checking autorecoverable hosts
>2012-11-07 08:35:00,005 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-75) Autorecovering 0 hosts
>2012-11-07 08:35:00,006 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-75) Checking autorecoverable hosts done
>2012-11-07 08:35:00,006 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-75) Checking autorecoverable storage domains
>2012-11-07 08:35:00,023 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-75) Autorecovering 1 storage domains
>2012-11-07 08:35:00,024 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-75) Autorecovering storage domains: b1658676-2dc3-4357-b112-3852a056c663
>2012-11-07 08:35:00,025 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-75) [452c8361] Lock Acquired to object EngineLock [exclusiveLocks= key: b1658676-2dc3-4357-b112-3852a056c663 value: STORAGE
>, sharedLocks= ]
>2012-11-07 08:35:00,044 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-75) [452c8361] Running command: ActivateStorageDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-07 08:35:00,065 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-75) [452c8361] Lock freed to object EngineLock [exclusiveLocks= key: b1658676-2dc3-4357-b112-3852a056c663 value: STORAGE
>, sharedLocks= ]
>2012-11-07 08:35:00,065 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-75) [452c8361] ActivateStorage Domain. Before Connect all hosts to pool. Time:11/7/12 8:35 AM
>2012-11-07 08:35:00,077 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-3-thread-50) [230b1fa2] START, ValidateStorageServerConnectionVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 039341b1-a8c0-4003-845a-4ef53f74c7c8, connection: localhost:/mnt/data };]), log id: 30bf69fc
>2012-11-07 08:35:00,091 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-3-thread-50) [230b1fa2] FINISH, ValidateStorageServerConnectionVDSCommand, return: {039341b1-a8c0-4003-845a-4ef53f74c7c8=0}, log id: 30bf69fc
>2012-11-07 08:35:00,092 INFO  [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-3-thread-50) [230b1fa2] Running command: ConnectStorageToVdsCommand internal: true. Entities affected :  ID: aaa00000-0000-0000-0000-123456789aaa Type: System
>2012-11-07 08:35:00,094 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-3-thread-50) [230b1fa2] START, ConnectStorageServerVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 039341b1-a8c0-4003-845a-4ef53f74c7c8, connection: localhost:/mnt/data };]), log id: 3028d97f
>2012-11-07 08:36:02,320 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-94) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:36:02,321 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-94) Vds: Ovirt
>2012-11-07 08:36:02,322 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-94) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:36:02,323 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-94) FINISH, ConnectStoragePoolVDSCommand, log id: e442cf4
>2012-11-07 08:36:02,323 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-94) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:36:02,324 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-94) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:36:02,331 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-94) [29489679] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:36:02,385 WARN  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-94) [29489679] CanDoAction of action ReconstructMasterDomain failed. Reasons:VAR__ACTION__RECONSTRUCT_MASTER,VAR__TYPE__STORAGE__DOMAIN,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL2,$status Locked
>2012-11-07 08:36:12,429 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-63) [3e434ec7] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:36:12,467 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-63) [3e434ec7] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 7d556ea
>2012-11-07 08:37:15,342 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-63) [3e434ec7] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:37:15,343 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-63) [3e434ec7] Vds: Ovirt
>2012-11-07 08:37:15,344 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-63) [3e434ec7] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:37:15,345 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-63) [3e434ec7] FINISH, ConnectStoragePoolVDSCommand, log id: 7d556ea
>2012-11-07 08:37:15,345 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-63) [3e434ec7] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:37:15,345 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-63) [3e434ec7] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:37:15,351 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-63) [6958084a] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:37:15,401 WARN  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-63) [6958084a] CanDoAction of action ReconstructMasterDomain failed. Reasons:VAR__ACTION__RECONSTRUCT_MASTER,VAR__TYPE__STORAGE__DOMAIN,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL2,$status Locked
>2012-11-07 08:37:25,448 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-79) [7a7a0d2b] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:37:25,489 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-79) [7a7a0d2b] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 3ecbca0b
>2012-11-07 08:38:00,097 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (pool-3-thread-50) [230b1fa2] XML RPC error in command ConnectStorageServerVDS ( Vds: Ovirt ), the error was: java.util.concurrent.TimeoutException, TimeoutException:  
>2012-11-07 08:38:00,099 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-3-thread-50) [230b1fa2] FINISH, ConnectStorageServerVDSCommand, log id: 3028d97f
>2012-11-07 08:38:00,099 ERROR [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-3-thread-50) [230b1fa2] Command org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSNetworkException: 
>2012-11-07 08:38:00,103 ERROR [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-3-thread-50) [230b1fa2] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand.
>2012-11-07 08:38:00,104 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-75) [452c8361] ActivateStorage Domain. After Connect all hosts to pool. Time:11/7/12 8:38 AM
>2012-11-07 08:38:00,105 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (QuartzScheduler_Worker-75) [452c8361] START, ActivateStorageDomainVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, storageDomainId = b1658676-2dc3-4357-b112-3852a056c663), log id: 29d48783
>2012-11-07 08:38:28,307 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-79) [7a7a0d2b] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:38:28,309 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-79) [7a7a0d2b] Vds: Ovirt
>2012-11-07 08:38:28,309 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-79) [7a7a0d2b] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:38:28,311 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-79) [7a7a0d2b] FINISH, ConnectStoragePoolVDSCommand, log id: 3ecbca0b
>2012-11-07 08:38:28,311 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-79) [7a7a0d2b] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:38:28,312 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-79) [7a7a0d2b] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:38:28,320 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-79) [7d6885ac] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:38:28,374 WARN  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-79) [7d6885ac] CanDoAction of action ReconstructMasterDomain failed. Reasons:VAR__ACTION__RECONSTRUCT_MASTER,VAR__TYPE__STORAGE__DOMAIN,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL2,$status Locked
>2012-11-07 08:38:28,418 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-75) [452c8361] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:38:28,439 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-75) [452c8361] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 745553e
>2012-11-07 08:38:32,954 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-80) starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-07 08:38:32,958 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-80) domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-07 08:38:32,963 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-80) [1a785bce] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:38:33,047 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-80) [1a785bce] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-07 08:38:33,069 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-80) [1a785bce] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-07 08:38:33,072 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-80) [1a785bce] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-07 08:38:33,073 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-80) [1a785bce] No string for UNASSIGNED type. Use default Log
>2012-11-07 08:38:35,017 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-65) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-07 08:39:31,272 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-75) [452c8361] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:39:31,273 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-75) [452c8361] Vds: Ovirt
>2012-11-07 08:39:31,273 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-75) [452c8361] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:39:31,273 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-75) [452c8361] FINISH, ConnectStoragePoolVDSCommand, log id: 745553e
>2012-11-07 08:39:31,274 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-75) [452c8361] IrsBroker::Failed::ActivateStorageDomainVDS
>2012-11-07 08:39:31,274 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-75) [452c8361] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:39:31,280 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-75) [4a92da64] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:39:31,314 WARN  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-75) [4a92da64] CanDoAction of action ReconstructMasterDomain failed. Reasons:VAR__ACTION__RECONSTRUCT_MASTER,VAR__TYPE__STORAGE__DOMAIN,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL2,$status Locked
>2012-11-07 08:39:31,331 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (QuartzScheduler_Worker-75) [4a92da64] FINISH, ActivateStorageDomainVDSCommand, log id: 29d48783
>2012-11-07 08:39:31,332 ERROR [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-75) [4a92da64] Command org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.irsbroker.IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:39:31,340 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-75) [4a92da64] Command [id=fd4a8c68-0da6-498b-becb-dec6e9f323be]: Compensating CHANGED_STATUS_ONLY of org.ovirt.engine.core.common.businessentities.storage_pool_iso_map; snapshot: EntityStatusSnapshot [id=storagePoolId = c02503a4-274e-11e2-b33c-002590730631, storageId = b1658676-2dc3-4357-b112-3852a056c663, status=InActive].
>2012-11-07 08:39:31,352 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-75) [4a92da64] Checking autorecoverable storage domains done
>2012-11-07 08:39:31,359 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-15) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:39:31,374 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-15) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 626b1ff4
>2012-11-07 08:39:52,309 WARN  [org.ovirt.engine.core.dal.dbbroker.DbFacadeLocator] (MSC service thread 1-5) /etc/engine/engine.conf file is not found. Please check your engine installation. Default values will be used
>2012-11-07 08:39:53,698 INFO  [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-5) Start time: 11/7/12 8:39 AM
>2012-11-07 08:39:53,745 WARN  [org.ovirt.engine.core.utils.ConfigUtilsBase] (MSC service thread 1-5) Could not find enum value for option: CbcCheckOnVdsChange
>2012-11-07 08:39:53,747 WARN  [org.ovirt.engine.core.utils.ConfigUtilsBase] (MSC service thread 1-5) Could not find enum value for option: CAEngineKey
>2012-11-07 08:39:53,750 WARN  [org.ovirt.engine.core.utils.ConfigUtilsBase] (MSC service thread 1-5) Could not find enum value for option: MinimalETLVersion
>2012-11-07 08:39:53,753 WARN  [org.ovirt.engine.core.utils.ConfigUtilsBase] (MSC service thread 1-5) Could not find enum value for option: SQLServerI18NPrefix
>2012-11-07 08:39:53,778 WARN  [org.ovirt.engine.core.utils.ConfigUtilsBase] (MSC service thread 1-5) Could not find enum value for option: ScriptsPath
>2012-11-07 08:39:53,797 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (MSC service thread 1-5) Failed to decryptData must start with zero
>2012-11-07 08:39:53,797 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-5) Failed to decrypt value for property TruststorePass will be used encrypted value
>2012-11-07 08:39:53,798 WARN  [org.ovirt.engine.core.utils.ConfigUtilsBase] (MSC service thread 1-5) Could not find enum value for option: ENGINEEARLib
>2012-11-07 08:39:53,830 INFO  [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-5) VDSBrokerFrontend: 11/7/12 8:39 AM
>2012-11-07 08:39:53,833 INFO  [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-5) CpuFlagsManager: 11/7/12 8:39 AM
>2012-11-07 08:39:53,834 INFO  [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-5) Setting audit clean up manager to run at: 35 35 3 * * ?
>2012-11-07 08:39:53,843 INFO  [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-5) AuditLogCleanupManager: 11/7/12 8:39 AM
>2012-11-07 08:39:53,844 INFO  [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-5) TagsDirector initialization
>2012-11-07 08:39:53,851 INFO  [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-5) Tag root added to tree
>2012-11-07 08:39:53,857 INFO  [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-5) TagsDirector: 11/7/12 8:39 AM
>2012-11-07 08:39:53,861 INFO  [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-5) IsoDomainListSyncronizer: 11/7/12 8:39 AM
>2012-11-07 08:39:53,866 INFO  [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-5) InitHandlers: 11/7/12 8:39 AM
>2012-11-07 08:39:53,871 WARN  [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-5) Code VDS_FENCING_OPERATION_FAILED appears more then once in string table.
>2012-11-07 08:39:53,872 INFO  [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-5) ErrorTranslator: 11/7/12 8:39 AM
>2012-11-07 08:39:53,873 INFO  [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-5) VdsErrorTranslator: 11/7/12 8:39 AM
>2012-11-07 08:39:53,873 INFO  [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-5) Mark uncompleted jobs as UNKNOWN: 11/7/12 8:39 AM
>2012-11-07 08:39:53,904 INFO  [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-5) JobRepositoryCleanupManager: 11/7/12 8:39 AM
>2012-11-07 08:39:53,905 INFO  [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-5) AutoRecoveryManager: 11/7/12 8:39 AM
>2012-11-07 08:39:53,907 INFO  [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-5) ExecutionMessageDirector: 11/7/12 8:39 AM
>2012-11-07 08:39:53,912 INFO  [org.ovirt.engine.core.bll.adbroker.UsersDomainsCacheManagerService] (MSC service thread 1-4) UsersDomainsCacheManager: 11/7/12 8:39 AM
>2012-11-07 08:39:53,912 INFO  [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-5) InitResourceManager: 11/7/12 8:39 AM
>2012-11-07 08:39:53,914 INFO  [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-5) ResourceManager::ResourceManager::Entered
>2012-11-07 08:39:53,971 INFO  [org.ovirt.engine.core.bll.adbroker.UsersDomainsCacheManagerService] (MSC service thread 1-4) DbUserCacheManager: 11/7/12 8:39 AM
>2012-11-07 08:39:53,987 INFO  [org.ovirt.engine.core.bll.DbUserCacheManager] (QuartzScheduler_Worker-1) DbUserCacheManager::refreshAllUserData() - entered
>2012-11-07 08:40:00,003 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-2) Checking autorecoverable hosts
>2012-11-07 08:40:00,022 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-2) Autorecovering 0 hosts
>2012-11-07 08:40:00,022 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-2) Checking autorecoverable hosts done
>2012-11-07 08:40:00,024 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-2) Checking autorecoverable storage domains
>2012-11-07 08:40:00,074 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-2) Autorecovering 1 storage domains
>2012-11-07 08:40:00,074 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-2) Autorecovering storage domains: b1658676-2dc3-4357-b112-3852a056c663
>2012-11-07 08:40:00,089 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-2) [21ebdfba] Lock Acquired to object EngineLock [exclusiveLocks= key: b1658676-2dc3-4357-b112-3852a056c663 value: STORAGE
>, sharedLocks= ]
>2012-11-07 08:40:00,189 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-2) [21ebdfba] Running command: ActivateStorageDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-07 08:40:00,250 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-2) [21ebdfba] Lock freed to object EngineLock [exclusiveLocks= key: b1658676-2dc3-4357-b112-3852a056c663 value: STORAGE
>, sharedLocks= ]
>2012-11-07 08:40:00,251 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-2) [21ebdfba] ActivateStorage Domain. Before Connect all hosts to pool. Time:11/7/12 8:40 AM
>2012-11-07 08:40:00,287 WARN  [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (QuartzScheduler_Worker-2) [21ebdfba] Adding new value to configuration cache.
>2012-11-07 08:40:00,287 WARN  [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (QuartzScheduler_Worker-2) [21ebdfba] Didnt find the value of DBI18NPrefix in DB for version general - using default: 
>2012-11-07 08:40:00,288 WARN  [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (QuartzScheduler_Worker-2) [21ebdfba] Adding new value to configuration cache.
>2012-11-07 08:40:00,288 WARN  [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (QuartzScheduler_Worker-2) [21ebdfba] Didnt find the value of DBPagingType in DB for version general - using default: Offset
>2012-11-07 08:40:00,289 WARN  [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (QuartzScheduler_Worker-2) [21ebdfba] Adding new value to configuration cache.
>2012-11-07 08:40:00,290 WARN  [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (QuartzScheduler_Worker-2) [21ebdfba] Didnt find the value of DBPagingSyntax in DB for version general - using default: OFFSET (%1$s -1) LIMIT %2$s
>2012-11-07 08:40:00,290 WARN  [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (QuartzScheduler_Worker-2) [21ebdfba] Adding new value to configuration cache.
>2012-11-07 08:40:00,290 WARN  [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (QuartzScheduler_Worker-2) [21ebdfba] Didnt find the value of DBSearchTemplate in DB for version general - using default: SELECT * FROM (%2$s) %1$s) as T1 %3$s
>2012-11-07 08:40:00,349 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (pool-3-thread-1) [4ec645b7] Cannot get vdsManager for vdsid=093d532e-2750-11e2-a9ed-002590730631
>2012-11-07 08:40:00,354 ERROR [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-3-thread-1) [4ec645b7] Error during CanDoActionFailure.: java.lang.RuntimeException: VdcBLLException: Vds with id: 093d532e-2750-11e2-a9ed-002590730631 was not found
>	at org.ovirt.engine.core.vdsbroker.ResourceManager.CreateCommand(ResourceManager.java:372) [engine-vdsbroker.jar:]
>	at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:410) [engine-vdsbroker.jar:]
>	at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:35) [engine-bll.jar:]
>	at org.ovirt.engine.core.bll.storage.BaseFsStorageHelper.ValidateStoragePoolConnectionsInHost(BaseFsStorageHelper.java:46) [engine-bll.jar:]
>	at org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand.canDoAction(ConnectStorageToVdsCommand.java:25) [engine-bll.jar:]
>	at org.ovirt.engine.core.bll.CommandBase.InternalCanDoAction(CommandBase.java:486) [engine-bll.jar:]
>	at org.ovirt.engine.core.bll.CommandBase.ExecuteAction(CommandBase.java:261) [engine-bll.jar:]
>	at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:336) [engine-bll.jar:]
>	at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:289) [engine-bll.jar:]
>	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_09-icedtea]
>	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_09-icedtea]
>	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_09-icedtea]
>	at java.lang.reflect.Method.invoke(Method.java:601) [rt.jar:1.7.0_09-icedtea]
>	at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final]
>	at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.1.Final]
>	at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final]
>	at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation.jar:1.1.1.Final]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final]
>	at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.1.Final]
>	at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final]
>	at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final]
>	at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final]
>	at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final]
>	at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final]
>	at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final]
>	at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final]
>	at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final]
>	at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final]
>	at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.1.Final]
>	at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final]
>	at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final]
>	at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.1.Final]
>	at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final]
>	at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view9.runInternalAction(Unknown Source) [engine-bll.jar:]
>	at org.ovirt.engine.core.bll.storage.BaseFsStorageHelper.RunConnectionStorageToDomain(BaseFsStorageHelper.java:31) [engine-bll.jar:]
>	at org.ovirt.engine.core.bll.storage.StorageHelperBase.ConnectStorageToDomainByVdsId(StorageHelperBase.java:68) [engine-bll.jar:]
>	at org.ovirt.engine.core.bll.storage.ConnectSingleAsyncOperation.Execute(ConnectSingleAsyncOperation.java:18) [engine-bll.jar:]
>	at org.ovirt.engine.core.utils.SyncronizeNumberOfAsyncOperations$AsyncOpThread.run(SyncronizeNumberOfAsyncOperations.java:30) [engine-utils.jar:]
>	at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:64) [engine-utils.jar:]
>	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_09-icedtea]
>	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_09-icedtea]
>	at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_09-icedtea]
>	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [rt.jar:1.7.0_09-icedtea]
>	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [rt.jar:1.7.0_09-icedtea]
>	at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_09-icedtea]
>Caused by: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: Vds with id: 093d532e-2750-11e2-a9ed-002590730631 was not found
>	at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.initializeVdsBroker(VdsBrokerCommand.java:52) [engine-vdsbroker.jar:]
>	at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.<init>(VdsBrokerCommand.java:31) [engine-vdsbroker.jar:]
>	at org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand.<init>(ConnectStorageServerVDSCommand.java:24) [engine-vdsbroker.jar:]
>	at org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand.<init>(ValidateStorageServerConnectionVDSCommand.java:8) [engine-vdsbroker.jar:]
>	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) [rt.jar:1.7.0_09-icedtea]
>	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) [rt.jar:1.7.0_09-icedtea]
>	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [rt.jar:1.7.0_09-icedtea]
>	at java.lang.reflect.Constructor.newInstance(Constructor.java:525) [rt.jar:1.7.0_09-icedtea]
>	at org.ovirt.engine.core.vdsbroker.ResourceManager.CreateCommand(ResourceManager.java:367) [engine-vdsbroker.jar:]
>	... 54 more
>
>2012-11-07 08:40:00,365 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-2) [21ebdfba] ActivateStorage Domain. After Connect all hosts to pool. Time:11/7/12 8:40 AM
>2012-11-07 08:40:00,369 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (QuartzScheduler_Worker-2) [21ebdfba] START, ActivateStorageDomainVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, storageDomainId = b1658676-2dc3-4357-b112-3852a056c663), log id: 6d889d8b
>2012-11-07 08:40:00,386 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (QuartzScheduler_Worker-2) [21ebdfba] Cannot get vdsManager for vdsid=093d532e-2750-11e2-a9ed-002590730631
>2012-11-07 08:40:00,394 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-2) [21ebdfba] ERROR, ActivateStorageDomainVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, storageDomainId = b1658676-2dc3-4357-b112-3852a056c663), exception: RuntimeException: VdcBLLException: Vds with id: 093d532e-2750-11e2-a9ed-002590730631 was not found, log id: 6d889d8b: java.lang.RuntimeException: VdcBLLException: Vds with id: 093d532e-2750-11e2-a9ed-002590730631 was not found
>	at org.ovirt.engine.core.vdsbroker.ResourceManager.CreateCommand(ResourceManager.java:372) [engine-vdsbroker.jar:]
>	at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:410) [engine-vdsbroker.jar:]
>	at org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand$IrsProxyData.gethostFromVds(IrsBrokerCommand.java:660) [engine-vdsbroker.jar:]
>	at org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand$IrsProxyData.access$400(IrsBrokerCommand.java:132) [engine-vdsbroker.jar:]
>	at org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand$IrsProxyData$3.runInTransaction(IrsBrokerCommand.java:558) [engine-vdsbroker.jar:]
>	at org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand$IrsProxyData$3.runInTransaction(IrsBrokerCommand.java:555) [engine-vdsbroker.jar:]
>	at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:168) [engine-utils.jar:]
>	at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:107) [engine-utils.jar:]
>	at org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand$IrsProxyData.getIrsProxy(IrsBrokerCommand.java:555) [engine-vdsbroker.jar:]
>	at org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand.getIrsProxy(IrsBrokerCommand.java:1524) [engine-vdsbroker.jar:]
>	at org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand.ExecuteVDSCommand(IrsBrokerCommand.java:1532) [engine-vdsbroker.jar:]
>	at org.ovirt.engine.core.vdsbroker.VDSCommandBase.ExecuteCommand(VDSCommandBase.java:60) [engine-vdsbroker.jar:]
>	at org.ovirt.engine.core.dal.VdcCommandBase.Execute(VdcCommandBase.java:41) [engine-dal.jar:]
>	at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:413) [engine-vdsbroker.jar:]
>	at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:35) [engine-bll.jar:]
>	at org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand.executeCommand(ActivateStorageDomainCommand.java:67) [engine-bll.jar:]
>	at org.ovirt.engine.core.bll.CommandBase.ExecuteWithoutTransaction(CommandBase.java:804) [engine-bll.jar:]
>	at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:896) [engine-bll.jar:]
>	at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1203) [engine-bll.jar:]
>	at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:168) [engine-utils.jar:]
>	at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:107) [engine-utils.jar:]
>	at org.ovirt.engine.core.bll.CommandBase.Execute(CommandBase.java:911) [engine-bll.jar:]
>	at org.ovirt.engine.core.bll.CommandBase.ExecuteAction(CommandBase.java:268) [engine-bll.jar:]
>	at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:336) [engine-bll.jar:]
>	at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:289) [engine-bll.jar:]
>	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_09-icedtea]
>	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_09-icedtea]
>	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_09-icedtea]
>	at java.lang.reflect.Method.invoke(Method.java:601) [rt.jar:1.7.0_09-icedtea]
>	at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final]
>	at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.1.Final]
>	at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final]
>	at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation.jar:1.1.1.Final]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final]
>	at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.1.Final]
>	at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final]
>	at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final]
>	at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final]
>	at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final]
>	at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final]
>	at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final]
>	at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final]
>	at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final]
>	at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final]
>	at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.1.Final]
>	at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final]
>	at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final]
>	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final]
>	at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.1.Final]
>	at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final]
>	at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view9.runInternalAction(Unknown Source) [engine-bll.jar:]
>	at org.ovirt.engine.core.bll.AutoRecoveryManager.check(AutoRecoveryManager.java:96) [engine-bll.jar:]
>	at org.ovirt.engine.core.bll.AutoRecoveryManager.onTimer(AutoRecoveryManager.java:63) [engine-bll.jar:]
>	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_09-icedtea]
>	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_09-icedtea]
>	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_09-icedtea]
>	at java.lang.reflect.Method.invoke(Method.java:601) [rt.jar:1.7.0_09-icedtea]
>	at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:64) [engine-scheduler.jar:]
>	at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:]
>	at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:]
>Caused by: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: Vds with id: 093d532e-2750-11e2-a9ed-002590730631 was not found
>	at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.initializeVdsBroker(VdsBrokerCommand.java:52) [engine-vdsbroker.jar:]
>	at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.<init>(VdsBrokerCommand.java:45) [engine-vdsbroker.jar:]
>	at org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand.<init>(SpmStatusVDSCommand.java:20) [engine-vdsbroker.jar:]
>	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) [rt.jar:1.7.0_09-icedtea]
>	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) [rt.jar:1.7.0_09-icedtea]
>	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [rt.jar:1.7.0_09-icedtea]
>	at java.lang.reflect.Constructor.newInstance(Constructor.java:525) [rt.jar:1.7.0_09-icedtea]
>	at org.ovirt.engine.core.vdsbroker.ResourceManager.CreateCommand(ResourceManager.java:367) [engine-vdsbroker.jar:]
>	... 68 more
>
>2012-11-07 08:40:00,408 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (QuartzScheduler_Worker-2) [21ebdfba] FINISH, ActivateStorageDomainVDSCommand, log id: 6d889d8b
>2012-11-07 08:40:00,409 ERROR [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-2) [21ebdfba] Command org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.RuntimeException: VdcBLLException: Vds with id: 093d532e-2750-11e2-a9ed-002590730631 was not found
>2012-11-07 08:40:00,424 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-2) [21ebdfba] Command [id=19c85902-fd90-4b8a-8eb3-d8ef7ebcd425]: Compensating CHANGED_STATUS_ONLY of org.ovirt.engine.core.common.businessentities.storage_pool_iso_map; snapshot: EntityStatusSnapshot [id=storagePoolId = c02503a4-274e-11e2-b33c-002590730631, storageId = b1658676-2dc3-4357-b112-3852a056c663, status=InActive].
>2012-11-07 08:40:00,453 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-2) [21ebdfba] AuditLogType: UNASSIGNED not exist in string table
>2012-11-07 08:40:00,453 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-2) [21ebdfba] AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table
>2012-11-07 08:40:00,454 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-2) [21ebdfba] AuditLogType: DWH_STOPPED not exist in string table
>2012-11-07 08:40:00,454 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-2) [21ebdfba] AuditLogType: DWH_STARTED not exist in string table
>2012-11-07 08:40:00,454 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-2) [21ebdfba] AuditLogType: DWH_ERROR not exist in string table
>2012-11-07 08:40:00,490 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-2) [21ebdfba] Checking autorecoverable storage domains done
>2012-11-07 08:40:03,999 INFO  [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-5) Eneterd VdsManager:constructor
>2012-11-07 08:40:04,001 INFO  [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-5) vdsBroker(192.168.2.113,54,321)
>2012-11-07 08:40:04,042 INFO  [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-5) ResourceManager::AddVds - VDS 093d532e-2750-11e2-a9ed-002590730631 was added to the Resource Manager
>2012-11-07 08:40:04,056 INFO  [org.ovirt.engine.core.bll.AsyncTaskManager] (MSC service thread 1-5) AsyncTaskManager: Initialization of AsyncTaskManager completed successfully.
>2012-11-07 08:40:04,057 INFO  [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-5) AsyncTaskManager: 11/7/12 8:40 AM
>2012-11-07 08:40:04,058 INFO  [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-5) VdsLoadBalancer: 11/7/12 8:40 AM
>2012-11-07 08:40:04,066 INFO  [org.ovirt.engine.core.bll.TimeLeasedVmPoolManager] (MSC service thread 1-5) TimeLeasedVmPoolManager constractor entered
>2012-11-07 08:40:04,067 INFO  [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-5) TimeLeasedVmPoolManager: 11/7/12 8:40 AM
>2012-11-07 08:40:04,088 INFO  [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-5) MacPoolManager: 11/7/12 8:40 AM
>2012-11-07 08:40:06,711 INFO  [org.ovirt.engine.core.vdsbroker.VdsManager] (QuartzScheduler_Worker-6) Initializing Host: Ovirt
>2012-11-07 08:40:06,719 INFO  [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (QuartzScheduler_Worker-6) [2aa6d4c4] Running command: HandleVdsVersionCommand internal: true. Entities affected :  ID: 093d532e-2750-11e2-a9ed-002590730631 Type: VDS
>2012-11-07 08:40:06,741 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-6) [2aa6d4c4] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-07 08:40:10,420 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-14) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:40:10,492 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-14) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 7715687b
>2012-11-07 08:41:13,523 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-14) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:41:13,525 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-14) Vds: Ovirt
>2012-11-07 08:41:13,526 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-14) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:41:13,527 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-14) FINISH, ConnectStoragePoolVDSCommand, log id: 7715687b
>2012-11-07 08:41:13,528 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-14) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:41:13,528 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-14) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:41:13,548 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-14) [4666268b] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:41:13,672 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-14) [4666268b] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-07 08:41:13,726 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-14) [4666268b] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 60b24cc9
>2012-11-07 08:41:13,727 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-14) [4666268b] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-07 08:41:13,727 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-14) [4666268b] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 60b24cc9
>2012-11-07 08:41:18,257 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-25) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-07 08:41:23,838 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-34) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:41:23,878 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-34) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 648ddccb
>2012-11-07 08:42:19,384 INFO  [org.ovirt.engine.core.bll.LoginAdminUserCommand] (http--0.0.0.0-8443-4) Checking if user admin@internal is an admin, result true
>2012-11-07 08:42:19,385 INFO  [org.ovirt.engine.core.bll.LoginAdminUserCommand] (http--0.0.0.0-8443-4) Running command: LoginAdminUserCommand internal: false.
>2012-11-07 08:42:19,487 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (RedirectServletReportsPage) with null version, using default general for version
>2012-11-07 08:42:19,525 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:19,535 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:19,541 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:19,545 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:19,551 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:19,556 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:19,560 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:19,564 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:19,567 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:19,573 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:19,576 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (ManagementNetwork) with null version, using default general for version
>2012-11-07 08:42:19,581 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:19,585 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:19,589 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:19,592 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:19,595 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:19,600 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:19,606 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,158 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,158 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,284 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-7) calling GetConfigurationValueQuery (ManagementNetwork) with null version, using default general for version
>2012-11-07 08:42:23,348 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,348 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,348 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-7) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,349 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-8) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,351 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,353 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,353 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,355 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-7) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,358 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,359 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,359 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,363 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,363 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-8) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,365 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,368 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,368 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,370 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-8) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,370 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-7) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,371 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,372 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,373 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,373 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-8) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,374 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,380 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,381 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,381 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-7) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,381 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-8) calling GetConfigurationValueQuery (ManagementNetwork) with null version, using default general for version
>2012-11-07 08:42:23,384 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,384 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,384 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,387 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,387 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-8) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,388 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,391 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,391 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-8) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,393 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,395 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,396 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-8) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,396 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,398 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,399 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-8) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,398 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,399 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,402 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,403 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,404 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,405 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,406 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,406 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (ManagementNetwork) with null version, using default general for version
>2012-11-07 08:42:23,409 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,409 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,409 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-8) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,411 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,412 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,414 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-8) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,414 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,415 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,415 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-7) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,417 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-8) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,418 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,418 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-7) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,419 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,421 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,421 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,422 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-8) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,425 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,425 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,428 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,429 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,430 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-7) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,430 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-8) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,430 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,431 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,433 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,434 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-8) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,435 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,436 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-7) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,438 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,438 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-7) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,439 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,440 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,440 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,441 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-8) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,441 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-7) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,443 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,443 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,444 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-8) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,444 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-7) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,446 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,446 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,447 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,447 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-8) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,447 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-7) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,448 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,449 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,450 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,451 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-8) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,452 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,452 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,453 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-7) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,454 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,455 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,455 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-8) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,455 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,464 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-7) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,465 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,466 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,466 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-8) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,468 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,468 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,468 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,468 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-7) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,469 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,471 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,471 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,471 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-8) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,472 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-7) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,472 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,473 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,475 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-07 08:42:23,475 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (DocsURL) with null version, using default general for version
>2012-11-07 08:42:26,790 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-34) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:42:26,791 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-34) Vds: Ovirt
>2012-11-07 08:42:26,792 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-34) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:42:26,793 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-34) FINISH, ConnectStoragePoolVDSCommand, log id: 648ddccb
>2012-11-07 08:42:26,794 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-34) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:42:26,794 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-34) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:42:26,808 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-34) [5541f4b4] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:42:26,909 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-34) [5541f4b4] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-07 08:42:26,977 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-34) [5541f4b4] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 32fe03ae
>2012-11-07 08:42:26,978 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-34) [5541f4b4] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-07 08:42:26,979 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-34) [5541f4b4] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 32fe03ae
>2012-11-07 08:42:28,230 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-7) calling GetConfigurationValueQuery (SpiceUsbAutoShare) with null version, using default general for version
>2012-11-07 08:42:29,824 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-44) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-07 08:42:31,857 INFO  [org.ovirt.engine.core.bll.HotUnPlugDiskFromVmCommand] (pool-3-thread-50) [57cafee7] Running command: HotUnPlugDiskFromVmCommand internal: false. Entities affected :  ID: 0e64bf87-8996-41be-9f28-a7126c815d70 Type: VM
>2012-11-07 08:42:37,060 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-57) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:42:37,107 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-57) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 556e33f7
>2012-11-07 08:43:40,001 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-57) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:43:40,002 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-57) Vds: Ovirt
>2012-11-07 08:43:40,003 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-57) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:43:40,004 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-57) FINISH, ConnectStoragePoolVDSCommand, log id: 556e33f7
>2012-11-07 08:43:40,005 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-57) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:43:40,005 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-57) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:43:40,015 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-57) [3144a546] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:43:40,098 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-57) [3144a546] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-07 08:43:40,143 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-57) [3144a546] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 7f2ccf9f
>2012-11-07 08:43:40,144 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-57) [3144a546] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-07 08:43:40,145 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-57) [3144a546] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 7f2ccf9f
>2012-11-07 08:43:41,214 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-64) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-07 08:43:50,224 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-79) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:43:50,266 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-79) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 450883c8
>2012-11-07 08:44:22,185 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (pool-3-thread-49) [783c9758] Lock Acquired to object EngineLock [exclusiveLocks= key: b1658676-2dc3-4357-b112-3852a056c663 value: STORAGE
>, sharedLocks= ]
>2012-11-07 08:44:22,209 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (pool-3-thread-49) [783c9758] Running command: ActivateStorageDomainCommand internal: false. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-07 08:44:22,229 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (pool-3-thread-49) [783c9758] Lock freed to object EngineLock [exclusiveLocks= key: b1658676-2dc3-4357-b112-3852a056c663 value: STORAGE
>, sharedLocks= ]
>2012-11-07 08:44:22,230 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (pool-3-thread-49) [783c9758] ActivateStorage Domain. Before Connect all hosts to pool. Time:11/7/12 8:44 AM
>2012-11-07 08:44:22,247 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-3-thread-48) [29110f47] START, ValidateStorageServerConnectionVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 039341b1-a8c0-4003-845a-4ef53f74c7c8, connection: localhost:/mnt/data };]), log id: 6bf75e55
>2012-11-07 08:44:22,277 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-3-thread-48) [29110f47] FINISH, ValidateStorageServerConnectionVDSCommand, return: {039341b1-a8c0-4003-845a-4ef53f74c7c8=0}, log id: 6bf75e55
>2012-11-07 08:44:22,278 INFO  [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-3-thread-48) [29110f47] Running command: ConnectStorageToVdsCommand internal: true. Entities affected :  ID: aaa00000-0000-0000-0000-123456789aaa Type: System
>2012-11-07 08:44:22,282 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-3-thread-48) [29110f47] START, ConnectStorageServerVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 039341b1-a8c0-4003-845a-4ef53f74c7c8, connection: localhost:/mnt/data };]), log id: 3a2d3ca1
>2012-11-07 08:44:53,138 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-79) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:44:53,139 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-79) Vds: Ovirt
>2012-11-07 08:44:53,140 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-79) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:44:53,141 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-79) FINISH, ConnectStoragePoolVDSCommand, log id: 450883c8
>2012-11-07 08:44:53,141 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-79) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:44:53,141 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-79) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:44:53,148 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-79) [11114d97] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:44:53,196 WARN  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-79) [11114d97] CanDoAction of action ReconstructMasterDomain failed. Reasons:VAR__ACTION__RECONSTRUCT_MASTER,VAR__TYPE__STORAGE__DOMAIN,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL2,$status Locked
>2012-11-07 08:45:00,000 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-81) Checking autorecoverable hosts
>2012-11-07 08:45:00,005 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-81) Autorecovering 0 hosts
>2012-11-07 08:45:00,006 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-81) Checking autorecoverable hosts done
>2012-11-07 08:45:00,006 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-81) Checking autorecoverable storage domains
>2012-11-07 08:45:00,009 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-81) Autorecovering 0 storage domains
>2012-11-07 08:45:00,009 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-81) Checking autorecoverable storage domains done
>2012-11-07 08:45:03,246 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-86) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:45:03,266 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-86) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 63dac4b8
>2012-11-07 08:45:06,743 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-94) starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-07 08:45:06,765 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-94) domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-07 08:45:06,771 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-94) [3c853001] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:45:06,831 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-94) [3c853001] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-07 08:45:06,853 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-94) [3c853001] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-07 08:45:06,856 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-94) [3c853001] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-07 08:45:06,856 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-94) [3c853001] No string for UNASSIGNED type. Use default Log
>2012-11-07 08:46:06,096 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-86) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:46:06,097 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-86) Vds: Ovirt
>2012-11-07 08:46:06,097 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-86) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:46:06,098 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-86) FINISH, ConnectStoragePoolVDSCommand, log id: 63dac4b8
>2012-11-07 08:46:06,099 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-86) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:46:06,099 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-86) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:46:06,104 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-86) [3e26d130] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:46:06,151 WARN  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-86) [3e26d130] CanDoAction of action ReconstructMasterDomain failed. Reasons:VAR__ACTION__RECONSTRUCT_MASTER,VAR__TYPE__STORAGE__DOMAIN,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL2,$status Locked
>2012-11-07 08:46:14,308 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-75) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-07 08:46:16,195 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-81) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:46:16,210 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-81) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 292d095a
>2012-11-07 08:46:18,258 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-84) starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-07 08:46:18,262 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-84) domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-07 08:46:18,268 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-84) [1a42202] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:46:18,349 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-84) [1a42202] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-07 08:46:18,370 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-84) [1a42202] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-07 08:46:18,374 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-84) [1a42202] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-07 08:46:18,375 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-84) [1a42202] No string for UNASSIGNED type. Use default Log
>2012-11-07 08:47:19,122 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-81) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:47:19,123 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-81) Vds: Ovirt
>2012-11-07 08:47:19,123 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-81) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:47:19,123 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-81) FINISH, ConnectStoragePoolVDSCommand, log id: 292d095a
>2012-11-07 08:47:19,124 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-81) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:47:19,124 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-81) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:47:19,131 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-81) [5af61c63] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:47:19,180 WARN  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-81) [5af61c63] CanDoAction of action ReconstructMasterDomain failed. Reasons:VAR__ACTION__RECONSTRUCT_MASTER,VAR__TYPE__STORAGE__DOMAIN,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL2,$status Locked
>2012-11-07 08:47:22,286 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (pool-3-thread-48) [29110f47] XML RPC error in command ConnectStorageServerVDS ( Vds: Ovirt ), the error was: java.util.concurrent.TimeoutException, TimeoutException:  
>2012-11-07 08:47:22,286 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-3-thread-48) [29110f47] FINISH, ConnectStorageServerVDSCommand, log id: 3a2d3ca1
>2012-11-07 08:47:22,286 ERROR [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-3-thread-48) [29110f47] Command org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSNetworkException: 
>2012-11-07 08:47:22,289 ERROR [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-3-thread-48) [29110f47] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand.
>2012-11-07 08:47:22,290 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (pool-3-thread-49) [783c9758] ActivateStorage Domain. After Connect all hosts to pool. Time:11/7/12 8:47 AM
>2012-11-07 08:47:22,290 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (pool-3-thread-49) [783c9758] START, ActivateStorageDomainVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, storageDomainId = b1658676-2dc3-4357-b112-3852a056c663), log id: 144cdfb6
>2012-11-07 08:47:22,308 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-3-thread-49) [783c9758] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:47:22,341 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-3-thread-49) [783c9758] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 14570622
>2012-11-07 08:47:27,298 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-68) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-07 08:47:29,825 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-72) starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-07 08:47:29,841 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-72) domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-07 08:47:29,846 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-72) [1ef1299a] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:47:29,930 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-72) [1ef1299a] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-07 08:47:29,950 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-72) [1ef1299a] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-07 08:47:29,953 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-72) [1ef1299a] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-07 08:47:29,954 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-72) [1ef1299a] No string for UNASSIGNED type. Use default Log
>2012-11-07 08:47:37,501 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-93) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-07 08:48:25,184 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (pool-3-thread-49) [783c9758] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:48:25,185 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (pool-3-thread-49) [783c9758] Vds: Ovirt
>2012-11-07 08:48:25,186 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (pool-3-thread-49) [783c9758] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:48:25,187 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-3-thread-49) [783c9758] FINISH, ConnectStoragePoolVDSCommand, log id: 14570622
>2012-11-07 08:48:25,187 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-3-thread-49) [783c9758] IrsBroker::Failed::ActivateStorageDomainVDS
>2012-11-07 08:48:25,188 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-3-thread-49) [783c9758] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:48:25,196 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (pool-3-thread-49) [d50c53d] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:48:25,230 WARN  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (pool-3-thread-49) [d50c53d] CanDoAction of action ReconstructMasterDomain failed. Reasons:VAR__ACTION__RECONSTRUCT_MASTER,VAR__TYPE__STORAGE__DOMAIN,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL2,$status Locked
>2012-11-07 08:48:25,248 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (pool-3-thread-49) [d50c53d] FINISH, ActivateStorageDomainVDSCommand, log id: 144cdfb6
>2012-11-07 08:48:25,249 ERROR [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (pool-3-thread-49) [d50c53d] Command org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.irsbroker.IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:48:25,254 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (pool-3-thread-49) [d50c53d] Command [id=c21387ab-fc6a-43dc-a9fa-6d60df7498b7]: Compensating CHANGED_STATUS_ONLY of org.ovirt.engine.core.common.businessentities.storage_pool_iso_map; snapshot: EntityStatusSnapshot [id=storagePoolId = c02503a4-274e-11e2-b33c-002590730631, storageId = b1658676-2dc3-4357-b112-3852a056c663, status=InActive].
>2012-11-07 08:48:25,269 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-70) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:48:25,294 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-70) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 325d2b1f
>2012-11-07 08:49:28,134 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-70) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:49:28,134 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-70) Vds: Ovirt
>2012-11-07 08:49:28,135 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-70) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:49:28,135 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-70) FINISH, ConnectStoragePoolVDSCommand, log id: 325d2b1f
>2012-11-07 08:49:28,135 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-70) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:49:28,136 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-70) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:49:28,141 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-70) [9061500] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:49:28,208 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-70) [9061500] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-07 08:49:28,254 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-70) [9061500] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 666b4f45
>2012-11-07 08:49:28,254 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-70) [9061500] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-07 08:49:28,255 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-70) [9061500] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 666b4f45
>2012-11-07 08:49:29,861 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-73) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-07 08:49:38,344 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-93) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:49:38,381 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-93) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 36a7b584
>2012-11-07 08:50:00,001 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-27) Checking autorecoverable hosts
>2012-11-07 08:50:00,003 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-27) Autorecovering 0 hosts
>2012-11-07 08:50:00,004 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-27) Checking autorecoverable hosts done
>2012-11-07 08:50:00,004 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-27) Checking autorecoverable storage domains
>2012-11-07 08:50:00,042 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-27) Autorecovering 1 storage domains
>2012-11-07 08:50:00,042 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-27) Autorecovering storage domains: b1658676-2dc3-4357-b112-3852a056c663
>2012-11-07 08:50:00,044 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-27) [49d02692] Lock Acquired to object EngineLock [exclusiveLocks= key: b1658676-2dc3-4357-b112-3852a056c663 value: STORAGE
>, sharedLocks= ]
>2012-11-07 08:50:00,062 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-27) [49d02692] Running command: ActivateStorageDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-07 08:50:00,083 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-27) [49d02692] Lock freed to object EngineLock [exclusiveLocks= key: b1658676-2dc3-4357-b112-3852a056c663 value: STORAGE
>, sharedLocks= ]
>2012-11-07 08:50:00,084 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-27) [49d02692] ActivateStorage Domain. Before Connect all hosts to pool. Time:11/7/12 8:50 AM
>2012-11-07 08:50:00,099 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-3-thread-49) [40dcc0ff] START, ValidateStorageServerConnectionVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 039341b1-a8c0-4003-845a-4ef53f74c7c8, connection: localhost:/mnt/data };]), log id: 21bca09c
>2012-11-07 08:50:00,125 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-3-thread-49) [40dcc0ff] FINISH, ValidateStorageServerConnectionVDSCommand, return: {039341b1-a8c0-4003-845a-4ef53f74c7c8=0}, log id: 21bca09c
>2012-11-07 08:50:00,126 INFO  [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-3-thread-49) [40dcc0ff] Running command: ConnectStorageToVdsCommand internal: true. Entities affected :  ID: aaa00000-0000-0000-0000-123456789aaa Type: System
>2012-11-07 08:50:00,130 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-3-thread-49) [40dcc0ff] START, ConnectStorageServerVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 039341b1-a8c0-4003-845a-4ef53f74c7c8, connection: localhost:/mnt/data };]), log id: 1fb7570d
>2012-11-07 08:50:41,215 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-93) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:50:41,217 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-93) Vds: Ovirt
>2012-11-07 08:50:41,217 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-93) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:50:41,219 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-93) FINISH, ConnectStoragePoolVDSCommand, log id: 36a7b584
>2012-11-07 08:50:41,219 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-93) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:50:41,220 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-93) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:50:41,228 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-93) [66b33fab] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:50:41,255 WARN  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-93) [66b33fab] CanDoAction of action ReconstructMasterDomain failed. Reasons:VAR__ACTION__RECONSTRUCT_MASTER,VAR__TYPE__STORAGE__DOMAIN,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL2,$status Locked
>2012-11-07 08:50:51,303 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-95) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:50:51,339 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-95) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 24981bce
>2012-11-07 08:51:54,154 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-95) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:51:54,156 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-95) Vds: Ovirt
>2012-11-07 08:51:54,157 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-95) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:51:54,158 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-95) FINISH, ConnectStoragePoolVDSCommand, log id: 24981bce
>2012-11-07 08:51:54,158 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-95) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:51:54,159 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-95) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:51:54,167 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-95) [413ceadc] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:51:54,232 WARN  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-95) [413ceadc] CanDoAction of action ReconstructMasterDomain failed. Reasons:VAR__ACTION__RECONSTRUCT_MASTER,VAR__TYPE__STORAGE__DOMAIN,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL2,$status Locked
>2012-11-07 08:52:04,284 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-90) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:52:04,329 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-90) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 37fa0af0
>2012-11-07 08:52:37,502 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-31) starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-07 08:52:37,507 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-31) domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-07 08:52:37,512 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-31) [4312eebb] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:52:37,591 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-31) [4312eebb] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-07 08:52:37,611 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-31) [4312eebb] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-07 08:52:37,615 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-31) [4312eebb] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-07 08:52:37,615 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-31) [4312eebb] No string for UNASSIGNED type. Use default Log
>2012-11-07 08:53:00,135 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (pool-3-thread-49) [40dcc0ff] XML RPC error in command ConnectStorageServerVDS ( Vds: Ovirt ), the error was: java.util.concurrent.TimeoutException, TimeoutException:  
>2012-11-07 08:53:00,137 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-3-thread-49) [40dcc0ff] FINISH, ConnectStorageServerVDSCommand, log id: 1fb7570d
>2012-11-07 08:53:00,137 ERROR [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-3-thread-49) [40dcc0ff] Command org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSNetworkException: 
>2012-11-07 08:53:00,141 ERROR [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-3-thread-49) [40dcc0ff] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand.
>2012-11-07 08:53:00,142 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-27) [49d02692] ActivateStorage Domain. After Connect all hosts to pool. Time:11/7/12 8:53 AM
>2012-11-07 08:53:00,143 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (QuartzScheduler_Worker-27) [49d02692] START, ActivateStorageDomainVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, storageDomainId = b1658676-2dc3-4357-b112-3852a056c663), log id: a1873f2
>2012-11-07 08:53:00,257 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-88) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-07 08:53:07,159 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-90) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:53:07,161 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-90) Vds: Ovirt
>2012-11-07 08:53:07,161 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-90) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:53:07,162 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-90) FINISH, ConnectStoragePoolVDSCommand, log id: 37fa0af0
>2012-11-07 08:53:07,163 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-90) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:53:07,163 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-90) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:53:07,171 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-90) [4ad6dab2] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:53:07,204 WARN  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-90) [4ad6dab2] CanDoAction of action ReconstructMasterDomain failed. Reasons:VAR__ACTION__RECONSTRUCT_MASTER,VAR__TYPE__STORAGE__DOMAIN,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL2,$status Locked
>2012-11-07 08:53:07,248 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-27) [49d02692] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:53:07,268 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-27) [49d02692] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 4dded356
>2012-11-07 08:54:10,208 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-27) [49d02692] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:54:10,210 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-27) [49d02692] Vds: Ovirt
>2012-11-07 08:54:10,211 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-27) [49d02692] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:54:10,212 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-27) [49d02692] FINISH, ConnectStoragePoolVDSCommand, log id: 4dded356
>2012-11-07 08:54:10,213 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-27) [49d02692] IrsBroker::Failed::ActivateStorageDomainVDS
>2012-11-07 08:54:10,213 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-27) [49d02692] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:54:10,221 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-27) [55a3e3b4] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:54:10,254 WARN  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-27) [55a3e3b4] CanDoAction of action ReconstructMasterDomain failed. Reasons:VAR__ACTION__RECONSTRUCT_MASTER,VAR__TYPE__STORAGE__DOMAIN,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL2,$status Locked
>2012-11-07 08:54:10,271 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (QuartzScheduler_Worker-27) [55a3e3b4] FINISH, ActivateStorageDomainVDSCommand, log id: a1873f2
>2012-11-07 08:54:10,272 ERROR [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-27) [55a3e3b4] Command org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.irsbroker.IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:54:10,278 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-27) [55a3e3b4] Command [id=19428ce3-972a-45e3-a1d4-5a78fac1ff2d]: Compensating CHANGED_STATUS_ONLY of org.ovirt.engine.core.common.businessentities.storage_pool_iso_map; snapshot: EntityStatusSnapshot [id=storagePoolId = c02503a4-274e-11e2-b33c-002590730631, storageId = b1658676-2dc3-4357-b112-3852a056c663, status=InActive].
>2012-11-07 08:54:10,292 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-27) [55a3e3b4] Checking autorecoverable storage domains done
>2012-11-07 08:54:10,299 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-78) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:54:10,315 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-78) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 6c9f9b2c
>2012-11-07 08:55:00,001 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-83) Checking autorecoverable hosts
>2012-11-07 08:55:00,003 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-83) Autorecovering 0 hosts
>2012-11-07 08:55:00,004 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-83) Checking autorecoverable hosts done
>2012-11-07 08:55:00,004 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-83) Checking autorecoverable storage domains
>2012-11-07 08:55:00,032 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-83) Autorecovering 1 storage domains
>2012-11-07 08:55:00,033 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-83) Autorecovering storage domains: b1658676-2dc3-4357-b112-3852a056c663
>2012-11-07 08:55:00,034 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-83) [3bea8ed9] Lock Acquired to object EngineLock [exclusiveLocks= key: b1658676-2dc3-4357-b112-3852a056c663 value: STORAGE
>, sharedLocks= ]
>2012-11-07 08:55:00,053 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-83) [3bea8ed9] Running command: ActivateStorageDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-07 08:55:00,074 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-83) [3bea8ed9] Lock freed to object EngineLock [exclusiveLocks= key: b1658676-2dc3-4357-b112-3852a056c663 value: STORAGE
>, sharedLocks= ]
>2012-11-07 08:55:00,075 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-83) [3bea8ed9] ActivateStorage Domain. Before Connect all hosts to pool. Time:11/7/12 8:55 AM
>2012-11-07 08:55:00,089 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-3-thread-46) [3ac83d99] START, ValidateStorageServerConnectionVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 039341b1-a8c0-4003-845a-4ef53f74c7c8, connection: localhost:/mnt/data };]), log id: 762872ea
>2012-11-07 08:55:00,109 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-3-thread-46) [3ac83d99] FINISH, ValidateStorageServerConnectionVDSCommand, return: {039341b1-a8c0-4003-845a-4ef53f74c7c8=0}, log id: 762872ea
>2012-11-07 08:55:00,110 INFO  [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-3-thread-46) [3ac83d99] Running command: ConnectStorageToVdsCommand internal: true. Entities affected :  ID: aaa00000-0000-0000-0000-123456789aaa Type: System
>2012-11-07 08:55:00,112 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-3-thread-46) [3ac83d99] START, ConnectStorageServerVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 039341b1-a8c0-4003-845a-4ef53f74c7c8, connection: localhost:/mnt/data };]), log id: 61ffc843
>2012-11-07 08:55:13,254 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-78) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:55:13,255 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-78) Vds: Ovirt
>2012-11-07 08:55:13,255 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-78) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:55:13,255 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-78) FINISH, ConnectStoragePoolVDSCommand, log id: 6c9f9b2c
>2012-11-07 08:55:13,255 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-78) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:55:13,256 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-78) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:55:13,261 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-78) [3012157d] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:55:13,307 WARN  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-78) [3012157d] CanDoAction of action ReconstructMasterDomain failed. Reasons:VAR__ACTION__RECONSTRUCT_MASTER,VAR__TYPE__STORAGE__DOMAIN,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL2,$status Locked
>2012-11-07 08:55:23,340 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-85) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:55:23,373 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-85) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 7ffff345
>2012-11-07 08:56:26,242 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-85) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:56:26,244 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-85) Vds: Ovirt
>2012-11-07 08:56:26,244 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-85) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:56:26,245 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-85) FINISH, ConnectStoragePoolVDSCommand, log id: 7ffff345
>2012-11-07 08:56:26,245 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-85) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:56:26,245 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-85) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:56:26,250 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-85) [2dce66ea] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:56:26,293 WARN  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-85) [2dce66ea] CanDoAction of action ReconstructMasterDomain failed. Reasons:VAR__ACTION__RECONSTRUCT_MASTER,VAR__TYPE__STORAGE__DOMAIN,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL2,$status Locked
>2012-11-07 08:56:36,339 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-93) [66b33fab] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:56:36,375 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-93) [66b33fab] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 18df7131
>2012-11-07 08:56:41,032 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-93) [66b33fab] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:56:41,034 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-93) [66b33fab] Vds: Ovirt
>2012-11-07 08:56:41,034 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-93) [66b33fab] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:56:41,035 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-93) [66b33fab] FINISH, ConnectStoragePoolVDSCommand, log id: 18df7131
>2012-11-07 08:56:41,035 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-93) [66b33fab] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:56:41,036 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-93) [66b33fab] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:56:41,042 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-93) [a6073a] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:56:41,240 WARN  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-93) [a6073a] CanDoAction of action ReconstructMasterDomain failed. Reasons:VAR__ACTION__RECONSTRUCT_MASTER,VAR__TYPE__STORAGE__DOMAIN,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL2,$status Locked
>2012-11-07 08:56:41,293 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-3-thread-46) [3ac83d99] FINISH, ConnectStorageServerVDSCommand, return: {039341b1-a8c0-4003-845a-4ef53f74c7c8=477}, log id: 61ffc843
>2012-11-07 08:56:41,296 ERROR [org.ovirt.engine.core.bll.storage.NFSStorageHelper] (pool-3-thread-46) [3ac83d99] The connection with details localhost:/mnt/data failed because of error code 477 and error message is: 477
>2012-11-07 08:56:41,325 ERROR [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-3-thread-46) [3ac83d99] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand.
>2012-11-07 08:56:41,325 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-83) [3bea8ed9] ActivateStorage Domain. After Connect all hosts to pool. Time:11/7/12 8:56 AM
>2012-11-07 08:56:41,326 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (QuartzScheduler_Worker-83) [3bea8ed9] START, ActivateStorageDomainVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, storageDomainId = b1658676-2dc3-4357-b112-3852a056c663), log id: 9f995c0
>2012-11-07 08:56:41,342 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-83) [3bea8ed9] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:56:41,355 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-83) [3bea8ed9] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 1293ed43
>2012-11-07 08:56:42,196 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-83) [3bea8ed9] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:56:42,196 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-83) [3bea8ed9] Vds: Ovirt
>2012-11-07 08:56:42,196 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-83) [3bea8ed9] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:56:42,197 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-83) [3bea8ed9] FINISH, ConnectStoragePoolVDSCommand, log id: 1293ed43
>2012-11-07 08:56:42,197 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-83) [3bea8ed9] IrsBroker::Failed::ActivateStorageDomainVDS
>2012-11-07 08:56:42,197 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-83) [3bea8ed9] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:56:42,202 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-83) [12d3c9a5] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:56:42,294 WARN  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-83) [12d3c9a5] CanDoAction of action ReconstructMasterDomain failed. Reasons:VAR__ACTION__RECONSTRUCT_MASTER,VAR__TYPE__STORAGE__DOMAIN,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL2,$status Locked
>2012-11-07 08:56:42,306 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (QuartzScheduler_Worker-83) [12d3c9a5] FINISH, ActivateStorageDomainVDSCommand, log id: 9f995c0
>2012-11-07 08:56:42,306 ERROR [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-83) [12d3c9a5] Command org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.irsbroker.IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:56:42,310 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-83) [12d3c9a5] Command [id=5f200406-0351-4021-ac4a-39c486201948]: Compensating CHANGED_STATUS_ONLY of org.ovirt.engine.core.common.businessentities.storage_pool_iso_map; snapshot: EntityStatusSnapshot [id=storagePoolId = c02503a4-274e-11e2-b33c-002590730631, storageId = b1658676-2dc3-4357-b112-3852a056c663, status=InActive].
>2012-11-07 08:56:42,326 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-83) [12d3c9a5] Checking autorecoverable storage domains done
>2012-11-07 08:56:51,288 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-7) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:56:51,325 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-7) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 1b26f4ec
>2012-11-07 08:56:52,136 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-7) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:56:52,137 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-7) Vds: Ovirt
>2012-11-07 08:56:52,138 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-7) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:56:52,139 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-7) FINISH, ConnectStoragePoolVDSCommand, log id: 1b26f4ec
>2012-11-07 08:56:52,139 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-7) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:56:52,139 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-7) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:56:52,147 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-7) [569c906d] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:56:52,217 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-7) [569c906d] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-07 08:56:52,262 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-7) [569c906d] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 61f0fec
>2012-11-07 08:56:52,263 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-7) [569c906d] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-07 08:56:52,263 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-7) [569c906d] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 61f0fec
>2012-11-07 08:56:53,326 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-11) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-07 08:57:02,343 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-28) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:57:02,381 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-28) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 17d82839
>2012-11-07 08:57:03,201 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-28) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:57:03,202 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-28) Vds: Ovirt
>2012-11-07 08:57:03,203 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-28) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:57:03,204 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-28) FINISH, ConnectStoragePoolVDSCommand, log id: 17d82839
>2012-11-07 08:57:03,205 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-28) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:57:03,205 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-28) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:57:03,212 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-28) [74ded3c2] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:57:03,281 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-28) [74ded3c2] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-07 08:57:03,328 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-28) [74ded3c2] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 1f6d2a01
>2012-11-07 08:57:03,329 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-28) [74ded3c2] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-07 08:57:03,329 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-28) [74ded3c2] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 1f6d2a01
>2012-11-07 08:57:03,549 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-32) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-07 08:57:13,397 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-49) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:57:13,437 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-49) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 2992bc83
>2012-11-07 08:57:14,234 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-49) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:57:14,236 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-49) Vds: Ovirt
>2012-11-07 08:57:14,236 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-49) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:57:14,237 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-49) FINISH, ConnectStoragePoolVDSCommand, log id: 2992bc83
>2012-11-07 08:57:14,238 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-49) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:57:14,238 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-49) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:57:14,245 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-49) [1c4eabb0] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:57:14,317 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-49) [1c4eabb0] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-07 08:57:14,362 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-49) [1c4eabb0] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: a1f0589
>2012-11-07 08:57:14,363 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-49) [1c4eabb0] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-07 08:57:14,363 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-49) [1c4eabb0] FINISH, MarkPoolInReconstructModeVDSCommand, log id: a1f0589
>2012-11-07 08:57:24,061 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-64) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-07 08:57:24,450 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-65) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:57:24,473 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-65) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 31475960
>2012-11-07 08:57:25,276 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-65) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:57:25,278 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-65) Vds: Ovirt
>2012-11-07 08:57:25,278 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-65) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:57:25,279 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-65) FINISH, ConnectStoragePoolVDSCommand, log id: 31475960
>2012-11-07 08:57:25,280 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-65) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:57:25,280 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-65) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:57:25,288 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-65) [28f824f8] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:57:25,356 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-65) [28f824f8] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-07 08:57:25,396 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-65) [28f824f8] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 3175ad7c
>2012-11-07 08:57:25,397 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-65) [28f824f8] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-07 08:57:25,397 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-65) [28f824f8] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 3175ad7c
>2012-11-07 08:57:34,274 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-72) [1ef1299a] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-07 08:57:35,489 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-77) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:57:35,526 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-77) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 507882ff
>2012-11-07 08:57:36,323 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-77) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:57:36,324 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-77) Vds: Ovirt
>2012-11-07 08:57:36,325 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-77) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:57:36,326 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-77) FINISH, ConnectStoragePoolVDSCommand, log id: 507882ff
>2012-11-07 08:57:36,327 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-77) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:57:36,327 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-77) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:57:36,335 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-77) [28189138] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:57:36,413 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-77) [28189138] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-07 08:57:36,455 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-77) [28189138] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 4c2b44fc
>2012-11-07 08:57:36,456 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-77) [28189138] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-07 08:57:36,456 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-77) [28189138] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 4c2b44fc
>2012-11-07 08:57:44,472 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-100) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-07 08:57:46,530 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-3) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:57:46,565 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-3) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 2766a7aa
>2012-11-07 08:57:47,374 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-3) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:57:47,375 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-3) Vds: Ovirt
>2012-11-07 08:57:47,376 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-3) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:57:47,376 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-3) FINISH, ConnectStoragePoolVDSCommand, log id: 2766a7aa
>2012-11-07 08:57:47,377 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-3) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:57:47,377 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-3) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:57:47,385 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-3) [73ca4946] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:57:47,457 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-3) [73ca4946] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-07 08:57:47,502 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-3) [73ca4946] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 753d33ea
>2012-11-07 08:57:47,503 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-3) [73ca4946] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-07 08:57:47,504 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-3) [73ca4946] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 753d33ea
>2012-11-07 08:57:54,675 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-16) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-07 08:57:57,588 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-24) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:57:57,627 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-24) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 68e33988
>2012-11-07 08:57:58,441 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-24) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:57:58,442 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-24) Vds: Ovirt
>2012-11-07 08:57:58,442 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-24) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:57:58,443 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-24) FINISH, ConnectStoragePoolVDSCommand, log id: 68e33988
>2012-11-07 08:57:58,444 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-24) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:57:58,444 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-24) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:57:58,452 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-24) [69058b6b] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:57:58,509 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-24) [69058b6b] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-07 08:57:58,553 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-24) [69058b6b] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 752325b8
>2012-11-07 08:57:58,554 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-24) [69058b6b] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-07 08:57:58,554 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-24) [69058b6b] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 752325b8
>2012-11-07 08:58:04,879 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-43) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-07 08:58:08,635 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-48) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:58:08,670 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-48) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 633d663
>2012-11-07 08:58:09,473 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-48) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:58:09,475 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-48) Vds: Ovirt
>2012-11-07 08:58:09,476 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-48) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:58:09,477 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-48) FINISH, ConnectStoragePoolVDSCommand, log id: 633d663
>2012-11-07 08:58:09,477 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-48) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:58:09,478 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-48) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:58:09,485 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-48) [7ad129a1] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:58:09,556 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-48) [7ad129a1] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-07 08:58:09,624 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-48) [7ad129a1] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 115f7afd
>2012-11-07 08:58:09,625 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-48) [7ad129a1] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-07 08:58:09,626 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-48) [7ad129a1] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 115f7afd
>2012-11-07 08:58:15,081 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-56) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-07 08:58:19,705 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-64) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:58:19,746 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-64) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 1af27560
>2012-11-07 08:58:20,554 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-64) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:58:20,556 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-64) Vds: Ovirt
>2012-11-07 08:58:20,556 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-64) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:58:20,557 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-64) FINISH, ConnectStoragePoolVDSCommand, log id: 1af27560
>2012-11-07 08:58:20,558 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-64) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:58:20,558 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-64) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:58:20,566 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-64) [13dbf3fd] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:58:20,634 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-64) [13dbf3fd] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-07 08:58:20,675 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-64) [13dbf3fd] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 6c408be4
>2012-11-07 08:58:20,677 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-64) [13dbf3fd] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-07 08:58:20,677 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-64) [13dbf3fd] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 6c408be4
>2012-11-07 08:58:25,285 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-37) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-07 08:58:30,756 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-70) [9061500] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:58:30,798 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-70) [9061500] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 27802792
>2012-11-07 08:58:31,625 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-70) [9061500] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:58:31,627 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-70) [9061500] Vds: Ovirt
>2012-11-07 08:58:31,627 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-70) [9061500] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:58:31,628 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-70) [9061500] FINISH, ConnectStoragePoolVDSCommand, log id: 27802792
>2012-11-07 08:58:31,629 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-70) [9061500] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:58:31,629 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-70) [9061500] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:58:31,637 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-70) [1d948b14] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:58:31,697 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-70) [1d948b14] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-07 08:58:31,742 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-70) [1d948b14] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 60ca4e42
>2012-11-07 08:58:31,743 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-70) [1d948b14] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-07 08:58:31,744 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-70) [1d948b14] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 60ca4e42
>2012-11-07 08:58:35,520 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-93) [a6073a] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-07 08:58:41,823 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-100) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:58:41,866 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-100) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 6bcd164a
>2012-11-07 08:58:42,673 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-100) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:58:42,674 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-100) Vds: Ovirt
>2012-11-07 08:58:42,675 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-100) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:58:42,676 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-100) FINISH, ConnectStoragePoolVDSCommand, log id: 6bcd164a
>2012-11-07 08:58:42,676 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-100) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:58:42,677 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-100) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:58:42,684 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-100) [6409c929] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:58:42,750 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-100) [6409c929] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-07 08:58:42,793 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-100) [6409c929] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 3f4bd09f
>2012-11-07 08:58:42,794 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-100) [6409c929] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-07 08:58:42,794 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-100) [6409c929] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 3f4bd09f
>2012-11-07 08:58:45,732 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-8) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-07 08:58:52,873 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-19) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:58:52,913 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-19) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 69125953
>2012-11-07 08:58:53,742 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-19) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:58:53,743 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-19) Vds: Ovirt
>2012-11-07 08:58:53,743 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-19) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:58:53,744 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-19) FINISH, ConnectStoragePoolVDSCommand, log id: 69125953
>2012-11-07 08:58:53,745 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-19) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:58:53,745 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-19) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:58:53,753 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-19) [528bd565] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:58:53,821 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-19) [528bd565] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-07 08:58:53,864 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-19) [528bd565] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: fe879a9
>2012-11-07 08:58:53,865 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-19) [528bd565] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-07 08:58:53,865 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-19) [528bd565] FINISH, MarkPoolInReconstructModeVDSCommand, log id: fe879a9
>2012-11-07 08:58:55,940 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-25) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-07 08:59:03,956 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-40) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:59:03,994 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-40) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 16a017f
>2012-11-07 08:59:04,792 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-40) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:59:04,793 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-40) Vds: Ovirt
>2012-11-07 08:59:04,794 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-40) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:59:04,795 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-40) FINISH, ConnectStoragePoolVDSCommand, log id: 16a017f
>2012-11-07 08:59:04,795 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-40) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:59:04,795 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-40) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:59:04,803 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-40) [3b654743] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:59:04,869 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-40) [3b654743] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-07 08:59:04,912 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-40) [3b654743] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 69fbf01f
>2012-11-07 08:59:04,913 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-40) [3b654743] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-07 08:59:04,914 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-40) [3b654743] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 69fbf01f
>2012-11-07 08:59:06,126 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-47) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-07 08:59:14,999 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-63) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:59:15,044 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-63) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 393e5e82
>2012-11-07 08:59:15,854 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-63) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:59:15,855 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-63) Vds: Ovirt
>2012-11-07 08:59:15,856 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-63) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:59:15,857 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-63) FINISH, ConnectStoragePoolVDSCommand, log id: 393e5e82
>2012-11-07 08:59:15,857 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-63) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:59:15,857 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-63) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:59:15,865 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-63) [5a5aac88] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:59:15,944 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-63) [5a5aac88] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-07 08:59:15,983 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-63) [5a5aac88] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 1672718e
>2012-11-07 08:59:15,983 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-63) [5a5aac88] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-07 08:59:15,984 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-63) [5a5aac88] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 1672718e
>2012-11-07 08:59:16,303 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-87) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-07 08:59:26,068 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-78) [3012157d] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:59:26,104 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-78) [3012157d] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 1b888aeb
>2012-11-07 08:59:26,912 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-78) [3012157d] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:59:26,913 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-78) [3012157d] Vds: Ovirt
>2012-11-07 08:59:26,914 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-78) [3012157d] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:59:26,915 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-78) [3012157d] FINISH, ConnectStoragePoolVDSCommand, log id: 1b888aeb
>2012-11-07 08:59:26,915 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-78) [3012157d] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:59:26,916 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-78) [3012157d] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:59:26,923 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-78) [13fe2af2] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:59:26,991 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-78) [13fe2af2] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-07 08:59:27,034 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-78) [13fe2af2] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 43c2dce2
>2012-11-07 08:59:27,034 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-78) [13fe2af2] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-07 08:59:27,035 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-78) [13fe2af2] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 43c2dce2
>2012-11-07 08:59:36,731 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-94) [3c853001] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-07 08:59:37,126 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-92) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:59:37,143 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-92) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 52326a92
>2012-11-07 08:59:37,945 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-92) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:59:37,946 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-92) Vds: Ovirt
>2012-11-07 08:59:37,947 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-92) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:59:37,948 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-92) FINISH, ConnectStoragePoolVDSCommand, log id: 52326a92
>2012-11-07 08:59:37,948 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-92) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:59:37,949 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-92) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:59:37,956 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-92) [2298b6b4] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:59:38,027 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-92) [2298b6b4] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-07 08:59:38,072 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-92) [2298b6b4] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 576e1acb
>2012-11-07 08:59:38,073 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-92) [2298b6b4] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-07 08:59:38,074 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-92) [2298b6b4] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 576e1acb
>2012-11-07 08:59:46,926 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-15) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-07 08:59:48,143 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-18) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-07 08:59:48,161 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-18) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 26c4c6a0
>2012-11-07 08:59:48,990 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-18) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-07 08:59:48,991 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-18) Vds: Ovirt
>2012-11-07 08:59:48,992 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-18) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:59:48,993 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-18) FINISH, ConnectStoragePoolVDSCommand, log id: 26c4c6a0
>2012-11-07 08:59:48,994 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-18) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-07 08:59:48,994 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-18) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-07 08:59:49,002 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-18) [60840b16] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-07 08:59:49,061 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-18) [60840b16] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-07 08:59:49,104 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-18) [60840b16] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 252a8ecd
>2012-11-07 08:59:49,105 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-18) [60840b16] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-07 08:59:49,105 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-18) [60840b16] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 252a8ecd
>2012-11-09 05:49:05,789 WARN  [org.ovirt.engine.core.dal.dbbroker.DbFacadeLocator] (MSC service thread 1-3) /etc/engine/engine.conf file is not found. Please check your engine installation. Default values will be used
>2012-11-09 05:50:16,672 INFO  [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-3) Start time: 11/9/12 5:50 AM
>2012-11-09 05:50:16,723 WARN  [org.ovirt.engine.core.utils.ConfigUtilsBase] (MSC service thread 1-3) Could not find enum value for option: CbcCheckOnVdsChange
>2012-11-09 05:50:16,725 WARN  [org.ovirt.engine.core.utils.ConfigUtilsBase] (MSC service thread 1-3) Could not find enum value for option: CAEngineKey
>2012-11-09 05:50:16,729 WARN  [org.ovirt.engine.core.utils.ConfigUtilsBase] (MSC service thread 1-3) Could not find enum value for option: MinimalETLVersion
>2012-11-09 05:50:16,732 WARN  [org.ovirt.engine.core.utils.ConfigUtilsBase] (MSC service thread 1-3) Could not find enum value for option: SQLServerI18NPrefix
>2012-11-09 05:50:16,895 WARN  [org.ovirt.engine.core.utils.ConfigUtilsBase] (MSC service thread 1-3) Could not find enum value for option: ScriptsPath
>2012-11-09 05:50:16,945 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (MSC service thread 1-3) Failed to decryptData must start with zero
>2012-11-09 05:50:16,945 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-3) Failed to decrypt value for property TruststorePass will be used encrypted value
>2012-11-09 05:50:16,946 WARN  [org.ovirt.engine.core.utils.ConfigUtilsBase] (MSC service thread 1-3) Could not find enum value for option: ENGINEEARLib
>2012-11-09 05:50:16,987 INFO  [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-3) VDSBrokerFrontend: 11/9/12 5:50 AM
>2012-11-09 05:50:16,989 INFO  [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-3) CpuFlagsManager: 11/9/12 5:50 AM
>2012-11-09 05:50:16,990 INFO  [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-3) Setting audit clean up manager to run at: 35 35 3 * * ?
>2012-11-09 05:50:17,000 INFO  [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-3) AuditLogCleanupManager: 11/9/12 5:50 AM
>2012-11-09 05:50:17,002 INFO  [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-3) TagsDirector initialization
>2012-11-09 05:50:17,018 INFO  [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-3) Tag root added to tree
>2012-11-09 05:50:17,026 INFO  [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-3) TagsDirector: 11/9/12 5:50 AM
>2012-11-09 05:50:17,032 INFO  [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-3) IsoDomainListSyncronizer: 11/9/12 5:50 AM
>2012-11-09 05:50:17,040 INFO  [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-3) InitHandlers: 11/9/12 5:50 AM
>2012-11-09 05:50:17,045 WARN  [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-3) Code VDS_FENCING_OPERATION_FAILED appears more then once in string table.
>2012-11-09 05:50:17,046 INFO  [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-3) ErrorTranslator: 11/9/12 5:50 AM
>2012-11-09 05:50:17,046 INFO  [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-3) VdsErrorTranslator: 11/9/12 5:50 AM
>2012-11-09 05:50:17,047 INFO  [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-3) Mark uncompleted jobs as UNKNOWN: 11/9/12 5:50 AM
>2012-11-09 05:50:17,070 INFO  [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-3) JobRepositoryCleanupManager: 11/9/12 5:50 AM
>2012-11-09 05:50:17,072 INFO  [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-3) AutoRecoveryManager: 11/9/12 5:50 AM
>2012-11-09 05:50:17,075 INFO  [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-3) ExecutionMessageDirector: 11/9/12 5:50 AM
>2012-11-09 05:50:17,107 INFO  [org.ovirt.engine.core.bll.adbroker.UsersDomainsCacheManagerService] (MSC service thread 1-2) UsersDomainsCacheManager: 11/9/12 5:50 AM
>2012-11-09 05:50:17,109 INFO  [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-1) InitResourceManager: 11/9/12 5:50 AM
>2012-11-09 05:50:17,111 INFO  [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-1) ResourceManager::ResourceManager::Entered
>2012-11-09 05:50:17,200 INFO  [org.ovirt.engine.core.bll.adbroker.UsersDomainsCacheManagerService] (MSC service thread 1-2) DbUserCacheManager: 11/9/12 5:50 AM
>2012-11-09 05:50:17,225 INFO  [org.ovirt.engine.core.bll.DbUserCacheManager] (QuartzScheduler_Worker-1) DbUserCacheManager::refreshAllUserData() - entered
>2012-11-09 05:50:27,187 INFO  [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-1) Eneterd VdsManager:constructor
>2012-11-09 05:50:27,187 INFO  [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-1) vdsBroker(192.168.2.113,54,321)
>2012-11-09 05:50:27,314 INFO  [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-1) ResourceManager::AddVds - VDS 093d532e-2750-11e2-a9ed-002590730631 was added to the Resource Manager
>2012-11-09 05:50:27,345 INFO  [org.ovirt.engine.core.bll.AsyncTaskManager] (MSC service thread 1-1) AsyncTaskManager: Initialization of AsyncTaskManager completed successfully.
>2012-11-09 05:50:27,346 INFO  [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-1) AsyncTaskManager: 11/9/12 5:50 AM
>2012-11-09 05:50:27,348 INFO  [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-1) VdsLoadBalancer: 11/9/12 5:50 AM
>2012-11-09 05:50:27,361 INFO  [org.ovirt.engine.core.bll.TimeLeasedVmPoolManager] (MSC service thread 1-1) TimeLeasedVmPoolManager constractor entered
>2012-11-09 05:50:27,361 INFO  [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-1) TimeLeasedVmPoolManager: 11/9/12 5:50 AM
>2012-11-09 05:50:27,393 INFO  [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-1) MacPoolManager: 11/9/12 5:50 AM
>2012-11-09 05:50:30,257 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-5) AuditLogType: UNASSIGNED not exist in string table
>2012-11-09 05:50:30,258 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-5) AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table
>2012-11-09 05:50:30,259 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-5) AuditLogType: DWH_STOPPED not exist in string table
>2012-11-09 05:50:30,259 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-5) AuditLogType: DWH_STARTED not exist in string table
>2012-11-09 05:50:30,259 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-5) AuditLogType: DWH_ERROR not exist in string table
>2012-11-09 05:50:30,410 INFO  [org.ovirt.engine.core.vdsbroker.VdsManager] (QuartzScheduler_Worker-5) Initializing Host: Ovirt
>2012-11-09 05:50:30,471 INFO  [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (QuartzScheduler_Worker-5) [8079edb] Running command: HandleVdsVersionCommand internal: true. Entities affected :  ID: 093d532e-2750-11e2-a9ed-002590730631 Type: VDS
>2012-11-09 05:50:30,491 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-5) [8079edb] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:50:37,398 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-18) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:50:37,517 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-18) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 12850164
>2012-11-09 05:50:38,474 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-18) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:50:38,476 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-18) Vds: Ovirt
>2012-11-09 05:50:38,482 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-18) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:50:38,483 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-18) FINISH, ConnectStoragePoolVDSCommand, log id: 12850164
>2012-11-09 05:50:38,483 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-18) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:50:38,483 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-18) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:50:38,500 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-18) [337a664f] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:50:40,173 WARN  [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (QuartzScheduler_Worker-18) [337a664f] Adding new value to configuration cache.
>2012-11-09 05:50:40,173 WARN  [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (QuartzScheduler_Worker-18) [337a664f] Didnt find the value of DBI18NPrefix in DB for version general - using default: 
>2012-11-09 05:50:40,174 WARN  [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (QuartzScheduler_Worker-18) [337a664f] Adding new value to configuration cache.
>2012-11-09 05:50:40,175 WARN  [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (QuartzScheduler_Worker-18) [337a664f] Didnt find the value of DBPagingType in DB for version general - using default: Offset
>2012-11-09 05:50:40,175 WARN  [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (QuartzScheduler_Worker-18) [337a664f] Adding new value to configuration cache.
>2012-11-09 05:50:40,175 WARN  [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (QuartzScheduler_Worker-18) [337a664f] Didnt find the value of DBPagingSyntax in DB for version general - using default: OFFSET (%1$s -1) LIMIT %2$s
>2012-11-09 05:50:40,176 WARN  [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (QuartzScheduler_Worker-18) [337a664f] Adding new value to configuration cache.
>2012-11-09 05:50:40,176 WARN  [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (QuartzScheduler_Worker-18) [337a664f] Didnt find the value of DBSearchTemplate in DB for version general - using default: SELECT * FROM (%2$s) %1$s) as T1 %3$s
>2012-11-09 05:50:40,223 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-18) [337a664f] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:50:40,325 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-18) [337a664f] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 52f9edfe
>2012-11-09 05:50:40,326 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-18) [337a664f] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:50:40,326 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-18) [337a664f] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 52f9edfe
>2012-11-09 05:50:42,231 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-24) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:50:50,448 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-39) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:50:50,467 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-39) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 23ae749e
>2012-11-09 05:50:51,339 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-39) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:50:51,341 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-39) Vds: Ovirt
>2012-11-09 05:50:51,342 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-39) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:50:51,343 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-39) FINISH, ConnectStoragePoolVDSCommand, log id: 23ae749e
>2012-11-09 05:50:51,344 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-39) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:50:51,344 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-39) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:50:51,357 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-39) [752566a] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:50:51,428 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-39) [752566a] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:50:51,473 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-39) [752566a] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 52218151
>2012-11-09 05:50:51,475 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-39) [752566a] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:50:51,475 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-39) [752566a] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 52218151
>2012-11-09 05:50:52,463 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-43) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:51:01,555 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-59) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:51:01,573 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-59) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 5fc068c
>2012-11-09 05:51:02,368 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-59) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:51:02,370 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-59) Vds: Ovirt
>2012-11-09 05:51:02,370 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-59) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:51:02,371 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-59) FINISH, ConnectStoragePoolVDSCommand, log id: 5fc068c
>2012-11-09 05:51:02,372 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-59) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:51:02,372 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-59) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:51:02,382 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-59) [830214e] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:51:02,451 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-59) [830214e] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:51:02,494 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-59) [830214e] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 35a4becc
>2012-11-09 05:51:02,495 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-59) [830214e] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:51:02,496 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-59) [830214e] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 35a4becc
>2012-11-09 05:51:02,678 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-60) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:51:12,576 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-78) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:51:12,596 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-78) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 162c812b
>2012-11-09 05:51:13,442 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-78) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:51:13,444 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-78) Vds: Ovirt
>2012-11-09 05:51:13,444 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-78) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:51:13,445 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-78) FINISH, ConnectStoragePoolVDSCommand, log id: 162c812b
>2012-11-09 05:51:13,446 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-78) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:51:13,446 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-78) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:51:13,456 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-78) [4c8fde32] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:51:13,524 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-78) [4c8fde32] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:51:13,571 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-78) [4c8fde32] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 53db09a4
>2012-11-09 05:51:13,571 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-78) [4c8fde32] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:51:13,571 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-78) [4c8fde32] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 53db09a4
>2012-11-09 05:51:23,268 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-94) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:51:23,658 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-95) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:51:23,680 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-95) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 2dbeb1a1
>2012-11-09 05:51:24,494 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-95) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:51:24,495 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-95) Vds: Ovirt
>2012-11-09 05:51:24,496 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-95) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:51:24,497 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-95) FINISH, ConnectStoragePoolVDSCommand, log id: 2dbeb1a1
>2012-11-09 05:51:24,497 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-95) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:51:24,498 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-95) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:51:24,508 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-95) [5b17058] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:51:24,572 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-95) [5b17058] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:51:24,618 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-95) [5b17058] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 4986086f
>2012-11-09 05:51:24,619 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-95) [5b17058] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:51:24,620 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-95) [5b17058] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 4986086f
>2012-11-09 05:51:33,467 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-16) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:51:34,696 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-14) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:51:34,719 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-14) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 3b4e815d
>2012-11-09 05:51:35,577 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-14) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:51:35,578 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-14) Vds: Ovirt
>2012-11-09 05:51:35,579 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-14) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:51:35,580 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-14) FINISH, ConnectStoragePoolVDSCommand, log id: 3b4e815d
>2012-11-09 05:51:35,580 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-14) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:51:35,580 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-14) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:51:35,587 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-14) [371014e1] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:51:35,656 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-14) [371014e1] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:51:35,702 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-14) [371014e1] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 22e6a2af
>2012-11-09 05:51:35,704 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-14) [371014e1] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:51:35,704 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-14) [371014e1] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 22e6a2af
>2012-11-09 05:51:43,662 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-30) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:51:45,790 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-36) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:51:45,814 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-36) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 27c8126a
>2012-11-09 05:51:46,725 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-36) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:51:46,727 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-36) Vds: Ovirt
>2012-11-09 05:51:46,727 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-36) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:51:46,728 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-36) FINISH, ConnectStoragePoolVDSCommand, log id: 27c8126a
>2012-11-09 05:51:46,729 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-36) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:51:46,729 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-36) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:51:46,734 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-36) [566248a0] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:51:46,800 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-36) [566248a0] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:51:46,844 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-36) [566248a0] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 42ddafbe
>2012-11-09 05:51:46,845 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-36) [566248a0] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:51:46,845 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-36) [566248a0] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 42ddafbe
>2012-11-09 05:51:53,857 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-47) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:51:56,937 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-52) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:51:56,961 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-52) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 18e33ce2
>2012-11-09 05:51:57,751 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-52) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:51:57,753 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-52) Vds: Ovirt
>2012-11-09 05:51:57,754 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-52) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:51:57,755 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-52) FINISH, ConnectStoragePoolVDSCommand, log id: 18e33ce2
>2012-11-09 05:51:57,755 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-52) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:51:57,756 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-52) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:51:57,764 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-52) [56f07966] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:51:57,831 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-52) [56f07966] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:51:57,874 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-52) [56f07966] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 2eb34620
>2012-11-09 05:51:57,875 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-52) [56f07966] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:51:57,876 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-52) [56f07966] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 2eb34620
>2012-11-09 05:52:04,064 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-66) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:52:07,972 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-73) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:52:08,013 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-73) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 2b48c226
>2012-11-09 05:52:08,819 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-73) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:52:08,820 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-73) Vds: Ovirt
>2012-11-09 05:52:08,821 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-73) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:52:08,822 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-73) FINISH, ConnectStoragePoolVDSCommand, log id: 2b48c226
>2012-11-09 05:52:08,822 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-73) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:52:08,823 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-73) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:52:08,831 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-73) [5e80c1d3] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:52:08,899 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-73) [5e80c1d3] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:52:08,943 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-73) [5e80c1d3] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 332effbe
>2012-11-09 05:52:08,944 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-73) [5e80c1d3] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:52:08,944 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-73) [5e80c1d3] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 332effbe
>2012-11-09 05:52:14,262 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-83) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:52:19,027 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-93) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:52:19,049 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-93) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 7abf7d6b
>2012-11-09 05:52:19,851 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-93) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:52:19,853 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-93) Vds: Ovirt
>2012-11-09 05:52:19,853 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-93) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:52:19,854 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-93) FINISH, ConnectStoragePoolVDSCommand, log id: 7abf7d6b
>2012-11-09 05:52:19,855 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-93) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:52:19,855 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-93) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:52:19,864 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-93) [1a175520] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:52:19,930 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-93) [1a175520] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:52:19,973 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-93) [1a175520] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 7d3f00b6
>2012-11-09 05:52:19,974 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-93) [1a175520] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:52:19,975 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-93) [1a175520] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 7d3f00b6
>2012-11-09 05:52:24,545 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-3) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:52:30,056 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-11) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:52:30,095 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-11) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 48202dfc
>2012-11-09 05:52:30,970 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-11) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:52:30,972 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-11) Vds: Ovirt
>2012-11-09 05:52:30,972 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-11) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:52:30,973 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-11) FINISH, ConnectStoragePoolVDSCommand, log id: 48202dfc
>2012-11-09 05:52:30,974 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-11) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:52:30,974 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-11) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:52:30,983 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-11) [2de976ac] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:52:31,078 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-11) [2de976ac] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:52:31,122 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-11) [2de976ac] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: ed34327
>2012-11-09 05:52:31,123 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-11) [2de976ac] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:52:31,124 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-11) [2de976ac] FINISH, MarkPoolInReconstructModeVDSCommand, log id: ed34327
>2012-11-09 05:52:34,732 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-22) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:52:41,208 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-30) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:52:41,229 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-30) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 23a63500
>2012-11-09 05:52:42,077 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-30) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:52:42,079 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-30) Vds: Ovirt
>2012-11-09 05:52:42,079 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-30) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:52:42,080 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-30) FINISH, ConnectStoragePoolVDSCommand, log id: 23a63500
>2012-11-09 05:52:42,081 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-30) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:52:42,081 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-30) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:52:42,090 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-30) [50500ec2] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:52:42,162 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-30) [50500ec2] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:52:42,204 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-30) [50500ec2] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 19e63361
>2012-11-09 05:52:42,204 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-30) [50500ec2] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:52:42,205 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-30) [50500ec2] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 19e63361
>2012-11-09 05:52:44,959 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-36) [566248a0] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:52:52,285 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-49) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:52:52,307 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-49) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 2809934f
>2012-11-09 05:52:53,113 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-49) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:52:53,114 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-49) Vds: Ovirt
>2012-11-09 05:52:53,115 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-49) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:52:53,116 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-49) FINISH, ConnectStoragePoolVDSCommand, log id: 2809934f
>2012-11-09 05:52:53,117 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-49) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:52:53,117 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-49) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:52:53,125 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-49) [2d1d8b93] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:52:53,191 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-49) [2d1d8b93] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:52:53,233 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-49) [2d1d8b93] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 35f9d32
>2012-11-09 05:52:53,234 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-49) [2d1d8b93] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:52:53,235 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-49) [2d1d8b93] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 35f9d32
>2012-11-09 05:52:55,157 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-57) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:53:03,314 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-68) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:53:03,335 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-68) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 3ef96495
>2012-11-09 05:53:04,139 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-68) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:53:04,141 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-68) Vds: Ovirt
>2012-11-09 05:53:04,142 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-68) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:53:04,143 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-68) FINISH, ConnectStoragePoolVDSCommand, log id: 3ef96495
>2012-11-09 05:53:04,143 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-68) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:53:04,144 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-68) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:53:04,152 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-68) [7016f073] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:53:04,223 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-68) [7016f073] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:53:04,263 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-68) [7016f073] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 5f213e31
>2012-11-09 05:53:04,264 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-68) [7016f073] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:53:04,265 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-68) [7016f073] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 5f213e31
>2012-11-09 05:53:05,364 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-72) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:53:14,358 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-88) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:53:14,374 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-88) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 79554741
>2012-11-09 05:53:15,199 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-88) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:53:15,201 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-88) Vds: Ovirt
>2012-11-09 05:53:15,201 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-88) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:53:15,202 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-88) FINISH, ConnectStoragePoolVDSCommand, log id: 79554741
>2012-11-09 05:53:15,203 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-88) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:53:15,203 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-88) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:53:15,212 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-88) [21d03423] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:53:15,283 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-88) [21d03423] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:53:15,324 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-88) [21d03423] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 61482313
>2012-11-09 05:53:15,325 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-88) [21d03423] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:53:15,325 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-88) [21d03423] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 61482313
>2012-11-09 05:53:15,542 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-92) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:53:25,408 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-7) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:53:25,428 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-7) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 14b68073
>2012-11-09 05:53:26,229 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-7) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:53:26,230 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-7) Vds: Ovirt
>2012-11-09 05:53:26,231 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-7) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:53:26,232 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-7) FINISH, ConnectStoragePoolVDSCommand, log id: 14b68073
>2012-11-09 05:53:26,232 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-7) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:53:26,233 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-7) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:53:26,251 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-7) [33efb190] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:53:26,317 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-7) [33efb190] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:53:26,363 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-7) [33efb190] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: ab5670d
>2012-11-09 05:53:26,364 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-7) [33efb190] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:53:26,364 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-7) [33efb190] FINISH, MarkPoolInReconstructModeVDSCommand, log id: ab5670d
>2012-11-09 05:53:36,051 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-25) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:53:36,442 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-28) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:53:36,462 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-28) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 5f3d29cd
>2012-11-09 05:53:37,281 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-28) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:53:37,282 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-28) Vds: Ovirt
>2012-11-09 05:53:37,283 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-28) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:53:37,284 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-28) FINISH, ConnectStoragePoolVDSCommand, log id: 5f3d29cd
>2012-11-09 05:53:37,284 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-28) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:53:37,285 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-28) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:53:37,293 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-28) [28303158] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:53:37,362 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-28) [28303158] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:53:37,405 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-28) [28303158] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 2d517aa8
>2012-11-09 05:53:37,406 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-28) [28303158] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:53:37,406 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-28) [28303158] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 2d517aa8
>2012-11-09 05:53:46,245 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-43) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:53:47,506 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-45) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:53:47,530 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-45) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 1473cf53
>2012-11-09 05:53:48,329 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-45) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:53:48,329 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-45) Vds: Ovirt
>2012-11-09 05:53:48,330 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-45) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:53:48,330 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-45) FINISH, ConnectStoragePoolVDSCommand, log id: 1473cf53
>2012-11-09 05:53:48,330 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-45) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:53:48,330 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-45) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:53:48,338 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-45) [163cb8df] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:53:48,408 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-45) [163cb8df] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:53:48,451 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-45) [163cb8df] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 6f4a1327
>2012-11-09 05:53:48,452 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-45) [163cb8df] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:53:48,453 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-45) [163cb8df] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 6f4a1327
>2012-11-09 05:53:56,438 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-59) [830214e] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:53:58,550 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-65) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:53:58,615 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-65) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 2768700f
>2012-11-09 05:53:59,415 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-65) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:53:59,416 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-65) Vds: Ovirt
>2012-11-09 05:53:59,416 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-65) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:53:59,417 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-65) FINISH, ConnectStoragePoolVDSCommand, log id: 2768700f
>2012-11-09 05:53:59,417 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-65) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:53:59,417 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-65) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:53:59,424 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-65) [27bb6518] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:53:59,488 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-65) [27bb6518] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:53:59,532 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-65) [27bb6518] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 536c97b9
>2012-11-09 05:53:59,532 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-65) [27bb6518] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:53:59,533 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-65) [27bb6518] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 536c97b9
>2012-11-09 05:54:06,630 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-78) [4c8fde32] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:54:09,623 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-85) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:54:09,647 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-85) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 1edfae7d
>2012-11-09 05:54:10,489 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-85) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:54:10,491 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-85) Vds: Ovirt
>2012-11-09 05:54:10,491 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-85) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:54:10,492 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-85) FINISH, ConnectStoragePoolVDSCommand, log id: 1edfae7d
>2012-11-09 05:54:10,493 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-85) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:54:10,493 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-85) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:54:10,501 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-85) [27d80cd9] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:54:10,598 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-85) [27d80cd9] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:54:10,642 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-85) [27d80cd9] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 32df2348
>2012-11-09 05:54:10,643 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-85) [27d80cd9] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:54:10,644 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-85) [27d80cd9] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 32df2348
>2012-11-09 05:54:16,824 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-98) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:54:20,735 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-1) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:54:20,761 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-1) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 39eee27e
>2012-11-09 05:54:21,621 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-1) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:54:21,622 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-1) Vds: Ovirt
>2012-11-09 05:54:21,623 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-1) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:54:21,624 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-1) FINISH, ConnectStoragePoolVDSCommand, log id: 39eee27e
>2012-11-09 05:54:21,624 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-1) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:54:21,625 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-1) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:54:21,633 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-1) [74d9341e] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:54:21,746 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-1) [74d9341e] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:54:21,804 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-1) [74d9341e] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 606523b2
>2012-11-09 05:54:21,804 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-1) [74d9341e] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:54:21,804 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-1) [74d9341e] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 606523b2
>2012-11-09 05:54:27,043 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-12) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:54:31,858 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-18) [337a664f] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:54:31,876 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-18) [337a664f] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 1f2a9760
>2012-11-09 05:54:32,690 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-18) [337a664f] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:54:32,692 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-18) [337a664f] Vds: Ovirt
>2012-11-09 05:54:32,692 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-18) [337a664f] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:54:32,693 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-18) [337a664f] FINISH, ConnectStoragePoolVDSCommand, log id: 1f2a9760
>2012-11-09 05:54:32,694 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-18) [337a664f] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:54:32,694 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-18) [337a664f] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:54:32,702 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-18) [d729fff] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:54:32,773 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-18) [d729fff] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:54:32,814 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-18) [d729fff] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 1c0540fa
>2012-11-09 05:54:32,815 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-18) [d729fff] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:54:32,815 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-18) [d729fff] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 1c0540fa
>2012-11-09 05:54:37,242 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-32) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:54:42,918 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-39) [752566a] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:54:42,944 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-39) [752566a] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 5b5a6c61
>2012-11-09 05:54:43,781 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-39) [752566a] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:54:43,783 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-39) [752566a] Vds: Ovirt
>2012-11-09 05:54:43,783 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-39) [752566a] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:54:43,784 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-39) [752566a] FINISH, ConnectStoragePoolVDSCommand, log id: 5b5a6c61
>2012-11-09 05:54:43,785 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-39) [752566a] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:54:43,785 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-39) [752566a] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:54:43,794 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-39) [36b5dbe6] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:54:43,862 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-39) [36b5dbe6] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:54:43,904 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-39) [36b5dbe6] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 57a899c2
>2012-11-09 05:54:43,905 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-39) [36b5dbe6] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:54:43,906 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-39) [36b5dbe6] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 57a899c2
>2012-11-09 05:54:47,445 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-50) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:54:53,995 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-59) [830214e] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:54:54,017 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-59) [830214e] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 3fffa6be
>2012-11-09 05:54:54,819 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-59) [830214e] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:54:54,821 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-59) [830214e] Vds: Ovirt
>2012-11-09 05:54:54,821 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-59) [830214e] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:54:54,822 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-59) [830214e] FINISH, ConnectStoragePoolVDSCommand, log id: 3fffa6be
>2012-11-09 05:54:54,823 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-59) [830214e] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:54:54,823 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-59) [830214e] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:54:54,831 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-59) [23ec66eb] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:54:54,894 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-59) [23ec66eb] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:54:54,940 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-59) [23ec66eb] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 73ba1748
>2012-11-09 05:54:54,941 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-59) [23ec66eb] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:54:54,941 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-59) [23ec66eb] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 73ba1748
>2012-11-09 05:54:57,644 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-70) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:55:00,003 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-68) [7016f073] Checking autorecoverable hosts
>2012-11-09 05:55:00,012 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-68) [7016f073] Autorecovering 0 hosts
>2012-11-09 05:55:00,012 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-68) [7016f073] Checking autorecoverable hosts done
>2012-11-09 05:55:00,013 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-68) [7016f073] Checking autorecoverable storage domains
>2012-11-09 05:55:00,080 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-68) [7016f073] Autorecovering 1 storage domains
>2012-11-09 05:55:00,081 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-68) [7016f073] Autorecovering storage domains: b1658676-2dc3-4357-b112-3852a056c663
>2012-11-09 05:55:00,086 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-68) [76698ab3] Lock Acquired to object EngineLock [exclusiveLocks= key: b1658676-2dc3-4357-b112-3852a056c663 value: STORAGE
>, sharedLocks= ]
>2012-11-09 05:55:00,101 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-68) [76698ab3] Running command: ActivateStorageDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:55:00,221 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-68) [76698ab3] Lock freed to object EngineLock [exclusiveLocks= key: b1658676-2dc3-4357-b112-3852a056c663 value: STORAGE
>, sharedLocks= ]
>2012-11-09 05:55:00,222 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-68) [76698ab3] ActivateStorage Domain. Before Connect all hosts to pool. Time:11/9/12 5:55 AM
>2012-11-09 05:55:00,298 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-3-thread-50) [5e08b39f] START, ValidateStorageServerConnectionVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 039341b1-a8c0-4003-845a-4ef53f74c7c8, connection: localhost:/mnt/data };]), log id: 42a6a850
>2012-11-09 05:55:00,317 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-3-thread-50) [5e08b39f] FINISH, ValidateStorageServerConnectionVDSCommand, return: {039341b1-a8c0-4003-845a-4ef53f74c7c8=0}, log id: 42a6a850
>2012-11-09 05:55:00,318 INFO  [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-3-thread-50) [5e08b39f] Running command: ConnectStorageToVdsCommand internal: true. Entities affected :  ID: aaa00000-0000-0000-0000-123456789aaa Type: System
>2012-11-09 05:55:00,320 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-3-thread-50) [5e08b39f] START, ConnectStorageServerVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 039341b1-a8c0-4003-845a-4ef53f74c7c8, connection: localhost:/mnt/data };]), log id: 6719e9e6
>2012-11-09 05:55:00,440 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-3-thread-50) [5e08b39f] FINISH, ConnectStorageServerVDSCommand, return: {039341b1-a8c0-4003-845a-4ef53f74c7c8=0}, log id: 6719e9e6
>2012-11-09 05:55:00,443 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-68) [76698ab3] ActivateStorage Domain. After Connect all hosts to pool. Time:11/9/12 5:55 AM
>2012-11-09 05:55:00,445 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (QuartzScheduler_Worker-68) [76698ab3] START, ActivateStorageDomainVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, storageDomainId = b1658676-2dc3-4357-b112-3852a056c663), log id: 6146477c
>2012-11-09 05:55:00,470 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-68) [76698ab3] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:55:00,510 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-68) [76698ab3] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 18e42e61
>2012-11-09 05:55:01,576 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-68) [76698ab3] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:55:01,577 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-68) [76698ab3] Vds: Ovirt
>2012-11-09 05:55:01,577 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-68) [76698ab3] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:55:01,578 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-68) [76698ab3] FINISH, ConnectStoragePoolVDSCommand, log id: 18e42e61
>2012-11-09 05:55:01,578 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-68) [76698ab3] IrsBroker::Failed::ActivateStorageDomainVDS
>2012-11-09 05:55:01,578 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-68) [76698ab3] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:55:01,590 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-68) [5bb24693] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:55:01,635 WARN  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-68) [5bb24693] CanDoAction of action ReconstructMasterDomain failed. Reasons:VAR__ACTION__RECONSTRUCT_MASTER,VAR__TYPE__STORAGE__DOMAIN,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL2,$status Locked
>2012-11-09 05:55:01,654 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (QuartzScheduler_Worker-68) [5bb24693] FINISH, ActivateStorageDomainVDSCommand, log id: 6146477c
>2012-11-09 05:55:01,654 ERROR [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-68) [5bb24693] Command org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.irsbroker.IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:55:01,669 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-68) [5bb24693] Command [id=b14e3673-c990-4772-9335-8e1af7b9e449]: Compensating CHANGED_STATUS_ONLY of org.ovirt.engine.core.common.businessentities.storage_pool_iso_map; snapshot: EntityStatusSnapshot [id=storagePoolId = c02503a4-274e-11e2-b33c-002590730631, storageId = b1658676-2dc3-4357-b112-3852a056c663, status=InActive].
>2012-11-09 05:55:01,713 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-68) [5bb24693] Checking autorecoverable storage domains done
>2012-11-09 05:55:05,015 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-81) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:55:05,065 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-81) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 44122ebe
>2012-11-09 05:55:05,912 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-81) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:55:05,914 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-81) Vds: Ovirt
>2012-11-09 05:55:05,914 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-81) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:55:05,915 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-81) FINISH, ConnectStoragePoolVDSCommand, log id: 44122ebe
>2012-11-09 05:55:05,915 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-81) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:55:05,916 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-81) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:55:05,924 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-81) [26a17f04] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:55:05,987 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-81) [26a17f04] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:55:06,031 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-81) [26a17f04] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 2cd8f8d
>2012-11-09 05:55:06,032 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-81) [26a17f04] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:55:06,032 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-81) [26a17f04] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 2cd8f8d
>2012-11-09 05:55:07,870 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-85) [27d80cd9] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:55:16,114 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-100) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:55:16,135 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-100) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 353d8164
>2012-11-09 05:55:16,945 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-100) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:55:16,946 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-100) Vds: Ovirt
>2012-11-09 05:55:16,947 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-100) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:55:16,948 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-100) FINISH, ConnectStoragePoolVDSCommand, log id: 353d8164
>2012-11-09 05:55:16,948 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-100) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:55:16,948 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-100) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:55:16,956 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-100) [772b7c58] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:55:17,028 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-100) [772b7c58] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:55:17,163 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-100) [772b7c58] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 420a707
>2012-11-09 05:55:17,164 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-100) [772b7c58] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:55:17,165 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-100) [772b7c58] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 420a707
>2012-11-09 05:55:18,091 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-6) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:55:27,294 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-19) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:55:27,340 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-19) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 615cabd5
>2012-11-09 05:55:28,150 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-19) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:55:28,152 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-19) Vds: Ovirt
>2012-11-09 05:55:28,152 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-19) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:55:28,153 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-19) FINISH, ConnectStoragePoolVDSCommand, log id: 615cabd5
>2012-11-09 05:55:28,154 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-19) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:55:28,154 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-19) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:55:28,164 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-19) [40e0d013] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:55:28,230 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-19) [40e0d013] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:55:28,305 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-19) [40e0d013] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 6b85e5a8
>2012-11-09 05:55:28,306 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-19) [40e0d013] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:55:28,306 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-19) [40e0d013] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 6b85e5a8
>2012-11-09 05:55:38,395 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-42) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:55:38,413 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-42) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 450c08a1
>2012-11-09 05:55:38,818 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-43) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:55:39,223 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-42) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:55:39,225 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-42) Vds: Ovirt
>2012-11-09 05:55:39,225 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-42) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:55:39,226 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-42) FINISH, ConnectStoragePoolVDSCommand, log id: 450c08a1
>2012-11-09 05:55:39,227 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-42) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:55:39,227 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-42) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:55:39,235 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-42) [4242cb9f] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:55:39,306 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-42) [4242cb9f] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:55:39,351 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-42) [4242cb9f] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 44778588
>2012-11-09 05:55:39,352 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-42) [4242cb9f] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:55:39,353 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-42) [4242cb9f] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 44778588
>2012-11-09 05:55:49,012 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-62) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:55:49,446 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-59) [23ec66eb] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:55:49,469 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-59) [23ec66eb] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 730f6422
>2012-11-09 05:55:50,362 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-59) [23ec66eb] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:55:50,364 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-59) [23ec66eb] Vds: Ovirt
>2012-11-09 05:55:50,364 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-59) [23ec66eb] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:55:50,366 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-59) [23ec66eb] FINISH, ConnectStoragePoolVDSCommand, log id: 730f6422
>2012-11-09 05:55:50,366 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-59) [23ec66eb] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:55:50,367 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-59) [23ec66eb] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:55:50,374 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-59) [50c588] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:55:50,449 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-59) [50c588] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:55:50,492 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-59) [50c588] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 830d625
>2012-11-09 05:55:50,493 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-59) [50c588] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:55:50,494 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-59) [50c588] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 830d625
>2012-11-09 05:55:59,189 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-78) [4c8fde32] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:56:00,587 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-83) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:56:00,612 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-83) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 11f03809
>2012-11-09 05:56:01,418 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-83) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:56:01,419 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-83) Vds: Ovirt
>2012-11-09 05:56:01,419 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-83) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:56:01,420 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-83) FINISH, ConnectStoragePoolVDSCommand, log id: 11f03809
>2012-11-09 05:56:01,421 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-83) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:56:01,422 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-83) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:56:01,429 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-83) [6a411a6a] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:56:01,502 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-83) [6a411a6a] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:56:01,544 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-83) [6a411a6a] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 56b21322
>2012-11-09 05:56:01,545 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-83) [6a411a6a] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:56:01,546 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-83) [6a411a6a] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 56b21322
>2012-11-09 05:56:09,393 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-97) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:56:11,638 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-100) [772b7c58] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:56:11,677 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-100) [772b7c58] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 472b7373
>2012-11-09 05:56:12,481 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-100) [772b7c58] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:56:12,483 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-100) [772b7c58] Vds: Ovirt
>2012-11-09 05:56:12,483 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-100) [772b7c58] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:56:12,484 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-100) [772b7c58] FINISH, ConnectStoragePoolVDSCommand, log id: 472b7373
>2012-11-09 05:56:12,485 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-100) [772b7c58] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:56:12,485 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-100) [772b7c58] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:56:12,493 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-100) [3c5b21f6] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:56:12,563 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-100) [3c5b21f6] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:56:12,609 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-100) [3c5b21f6] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 1e20960f
>2012-11-09 05:56:12,610 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-100) [3c5b21f6] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:56:12,611 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-100) [3c5b21f6] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 1e20960f
>2012-11-09 05:56:19,585 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-17) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:56:22,686 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-22) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:56:22,709 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-22) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 405f7bf
>2012-11-09 05:56:23,269 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-20) starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 05:56:23,279 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-20) domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 05:56:23,285 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-20) [7a721712] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:56:23,349 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-20) [7a721712] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 05:56:23,378 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-20) [7a721712] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 05:56:23,384 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-20) [7a721712] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 05:56:23,385 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-20) [7a721712] No string for UNASSIGNED type. Use default Log
>2012-11-09 05:56:23,511 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-22) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:56:23,512 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-22) Vds: Ovirt
>2012-11-09 05:56:23,512 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-22) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:56:23,513 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-22) FINISH, ConnectStoragePoolVDSCommand, log id: 405f7bf
>2012-11-09 05:56:23,514 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-22) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:56:23,514 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-22) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:56:23,522 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-22) [6b02e781] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:56:23,590 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-22) [6b02e781] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:56:23,631 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-22) [6b02e781] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 53e23b08
>2012-11-09 05:56:23,631 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-22) [6b02e781] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:56:23,632 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-22) [6b02e781] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 53e23b08
>2012-11-09 05:56:29,871 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-32) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:56:33,468 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-39) [36b5dbe6] starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 05:56:33,487 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-39) [36b5dbe6] domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 05:56:33,493 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-39) [55d3e877] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:56:33,572 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-39) [55d3e877] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 05:56:33,593 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-39) [55d3e877] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 05:56:33,596 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-39) [55d3e877] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 05:56:33,597 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-39) [55d3e877] No string for UNASSIGNED type. Use default Log
>2012-11-09 05:56:33,716 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-44) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:56:33,753 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-44) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 6270778e
>2012-11-09 05:56:34,737 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-44) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:56:34,739 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-44) Vds: Ovirt
>2012-11-09 05:56:34,739 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-44) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:56:34,740 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-44) FINISH, ConnectStoragePoolVDSCommand, log id: 6270778e
>2012-11-09 05:56:34,741 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-44) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:56:34,741 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-44) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:56:34,750 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-44) [237aa2e7] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:56:34,812 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-44) [237aa2e7] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:56:34,854 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-44) [237aa2e7] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 2afcefc6
>2012-11-09 05:56:34,855 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-44) [237aa2e7] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:56:34,856 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-44) [237aa2e7] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 2afcefc6
>2012-11-09 05:56:40,085 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-57) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:56:43,663 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-61) starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 05:56:43,667 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-61) domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 05:56:43,672 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-61) [18dd424d] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:56:43,742 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-61) [18dd424d] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 05:56:43,762 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-61) [18dd424d] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 05:56:43,765 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-61) [18dd424d] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 05:56:43,766 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-61) [18dd424d] No string for UNASSIGNED type. Use default Log
>2012-11-09 05:56:44,936 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-62) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:56:44,954 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-62) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 147fecff
>2012-11-09 05:56:45,762 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-62) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:56:45,764 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-62) Vds: Ovirt
>2012-11-09 05:56:45,764 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-62) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:56:45,765 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-62) FINISH, ConnectStoragePoolVDSCommand, log id: 147fecff
>2012-11-09 05:56:45,766 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-62) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:56:45,766 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-62) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:56:45,774 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-62) [e47bdb5] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:56:45,848 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-62) [e47bdb5] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:56:45,892 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-62) [e47bdb5] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 7fc09c59
>2012-11-09 05:56:45,893 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-62) [e47bdb5] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:56:45,893 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-62) [e47bdb5] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 7fc09c59
>2012-11-09 05:56:50,287 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-72) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:56:53,858 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-76) starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 05:56:53,862 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-76) domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 05:56:53,867 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-76) [29cab4a] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:56:53,935 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-76) [29cab4a] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 05:56:53,956 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-76) [29cab4a] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 05:56:53,960 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-76) [29cab4a] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 05:56:53,960 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-76) [29cab4a] No string for UNASSIGNED type. Use default Log
>2012-11-09 05:56:55,974 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-82) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:56:55,998 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-82) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 205ee640
>2012-11-09 05:56:56,810 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-82) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:56:56,811 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-82) Vds: Ovirt
>2012-11-09 05:56:56,812 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-82) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:56:56,813 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-82) FINISH, ConnectStoragePoolVDSCommand, log id: 205ee640
>2012-11-09 05:56:56,813 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-82) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:56:56,814 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-82) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:56:56,821 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-82) [66035589] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:56:56,893 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-82) [66035589] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:56:56,934 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-82) [66035589] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 2e7d902b
>2012-11-09 05:56:56,935 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-82) [66035589] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:56:56,936 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-82) [66035589] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 2e7d902b
>2012-11-09 05:57:00,532 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-90) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:57:04,064 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-96) starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 05:57:04,068 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-96) domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 05:57:04,074 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-96) [72946543] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:57:04,140 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-96) [72946543] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 05:57:04,161 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-96) [72946543] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 05:57:04,164 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-96) [72946543] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 05:57:04,165 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-96) [72946543] No string for UNASSIGNED type. Use default Log
>2012-11-09 05:57:07,016 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-6) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:57:07,043 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-6) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 35fd1bd5
>2012-11-09 05:57:07,958 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-6) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:57:07,960 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-6) Vds: Ovirt
>2012-11-09 05:57:07,960 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-6) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:57:07,961 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-6) FINISH, ConnectStoragePoolVDSCommand, log id: 35fd1bd5
>2012-11-09 05:57:07,962 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-6) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:57:07,962 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-6) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:57:07,969 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-6) [41b005f9] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:57:08,053 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-6) [41b005f9] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:57:08,099 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-6) [41b005f9] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 63201ef7
>2012-11-09 05:57:08,100 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-6) [41b005f9] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:57:08,101 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-6) [41b005f9] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 63201ef7
>2012-11-09 05:57:10,743 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-9) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:57:14,263 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-16) starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 05:57:14,267 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-16) domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 05:57:14,272 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-16) [5ba61d90] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:57:14,342 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-16) [5ba61d90] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 05:57:14,363 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-16) [5ba61d90] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 05:57:14,366 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-16) [5ba61d90] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 05:57:14,367 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-16) [5ba61d90] No string for UNASSIGNED type. Use default Log
>2012-11-09 05:57:18,176 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-22) [6b02e781] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:57:18,203 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-22) [6b02e781] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 1c3b4c24
>2012-11-09 05:57:19,020 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-22) [6b02e781] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:57:19,022 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-22) [6b02e781] Vds: Ovirt
>2012-11-09 05:57:19,022 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-22) [6b02e781] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:57:19,023 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-22) [6b02e781] FINISH, ConnectStoragePoolVDSCommand, log id: 1c3b4c24
>2012-11-09 05:57:19,024 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-22) [6b02e781] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:57:19,024 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-22) [6b02e781] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:57:19,032 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-22) [61ecd57b] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:57:19,096 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-22) [61ecd57b] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:57:19,141 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-22) [61ecd57b] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 50213505
>2012-11-09 05:57:19,141 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-22) [61ecd57b] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:57:19,142 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-22) [61ecd57b] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 50213505
>2012-11-09 05:57:20,938 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-27) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:57:24,547 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-32) starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 05:57:24,550 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-32) domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 05:57:24,556 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-32) [154e3e6e] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:57:24,627 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-32) [154e3e6e] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 05:57:24,649 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-32) [154e3e6e] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 05:57:24,652 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-32) [154e3e6e] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 05:57:24,653 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-32) [154e3e6e] No string for UNASSIGNED type. Use default Log
>2012-11-09 05:57:29,221 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-44) [237aa2e7] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:57:29,242 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-44) [237aa2e7] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 634f8da5
>2012-11-09 05:57:30,057 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-44) [237aa2e7] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:57:30,058 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-44) [237aa2e7] Vds: Ovirt
>2012-11-09 05:57:30,059 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-44) [237aa2e7] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:57:30,060 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-44) [237aa2e7] FINISH, ConnectStoragePoolVDSCommand, log id: 634f8da5
>2012-11-09 05:57:30,060 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-44) [237aa2e7] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:57:30,061 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-44) [237aa2e7] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:57:30,086 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-44) [3aed7df5] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:57:30,164 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-44) [3aed7df5] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:57:30,210 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-44) [3aed7df5] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 5b0a0bf5
>2012-11-09 05:57:30,211 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-44) [3aed7df5] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:57:30,211 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-44) [3aed7df5] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 5b0a0bf5
>2012-11-09 05:57:31,252 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (QuartzScheduler_Worker-48) vds::refreshVdsStats Failed getVdsStats,  vds = 093d532e-2750-11e2-a9ed-002590730631 : Ovirt, error = VDSNetworkException: VDSNetworkException: 
>2012-11-09 05:57:31,420 WARN  [org.ovirt.engine.core.vdsbroker.VdsManager] (QuartzScheduler_Worker-48) ResourceManager::refreshVdsRunTimeInfo::Failed to refresh VDS , vds = 093d532e-2750-11e2-a9ed-002590730631 : Ovirt, VDS Network Error, continuing.
>VDSNetworkException: 
>2012-11-09 05:57:33,431 WARN  [org.ovirt.engine.core.vdsbroker.VdsManager] (QuartzScheduler_Worker-49) [2d1d8b93] ResourceManager::refreshVdsRunTimeInfo::Failed to refresh VDS , vds = 093d532e-2750-11e2-a9ed-002590730631 : Ovirt, VDS Network Error, continuing.
>VDSNetworkException: 
>2012-11-09 05:57:35,825 INFO  [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (QuartzScheduler_Worker-57) [145b9bc9] Running command: InitVdsOnUpCommand internal: true.
>2012-11-09 05:57:35,904 INFO  [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (QuartzScheduler_Worker-57) [17ae0bc8] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected :  ID: c02503a4-274e-11e2-b33c-002590730631 Type: StoragePool
>2012-11-09 05:57:35,906 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (QuartzScheduler_Worker-57) [17ae0bc8] START, ConnectStorageServerVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, storageType = NFS, connectionList = [{ id: 9e77b8b6-dad5-4695-ad63-ed0f14920402, connection: 192.168.2.113:/mnt/iso };]), log id: 3098197f
>2012-11-09 05:57:35,984 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (QuartzScheduler_Worker-57) [17ae0bc8] FINISH, ConnectStorageServerVDSCommand, return: {9e77b8b6-dad5-4695-ad63-ed0f14920402=0}, log id: 3098197f
>2012-11-09 05:57:35,985 INFO  [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (QuartzScheduler_Worker-57) [17ae0bc8] Host Ovirt storage connection was succeeded 
>2012-11-09 05:57:36,007 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-57) [17ae0bc8] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 220cbbb2
>2012-11-09 05:57:37,061 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-57) [17ae0bc8] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:57:37,062 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-57) [17ae0bc8] Vds: Ovirt
>2012-11-09 05:57:37,062 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-57) [17ae0bc8] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:57:37,062 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-57) [17ae0bc8] FINISH, ConnectStoragePoolVDSCommand, log id: 220cbbb2
>2012-11-09 05:57:37,063 ERROR [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (QuartzScheduler_Worker-57) [17ae0bc8] Could not connect host Ovirt to pool Default
>2012-11-09 05:57:37,136 INFO  [org.ovirt.engine.core.bll.MultipleActionsRunner] (pool-3-thread-50) [17ae0bc8] MultipleActionsRunner of type MigrateVmToServer invoked with no actions
>2012-11-09 05:57:37,145 INFO  [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (QuartzScheduler_Worker-57) [612243f4] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected :  ID: 093d532e-2750-11e2-a9ed-002590730631 Type: VDS
>2012-11-09 05:57:37,148 INFO  [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (QuartzScheduler_Worker-57) [25d14add] Running command: HandleVdsVersionCommand internal: true. Entities affected :  ID: 093d532e-2750-11e2-a9ed-002590730631 Type: VDS
>2012-11-09 05:57:40,296 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-64) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:57:40,337 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-64) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 76be3b05
>2012-11-09 05:57:41,172 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-64) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:57:41,172 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-64) Vds: Ovirt
>2012-11-09 05:57:41,173 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-64) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:57:41,173 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-64) FINISH, ConnectStoragePoolVDSCommand, log id: 76be3b05
>2012-11-09 05:57:41,173 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-64) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:57:41,173 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-64) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:57:41,179 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-64) [4ae70538] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:57:41,247 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-64) [4ae70538] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:57:41,291 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-64) [4ae70538] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 68d799ea
>2012-11-09 05:57:41,291 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-64) [4ae70538] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:57:41,292 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-64) [4ae70538] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 68d799ea
>2012-11-09 05:57:43,305 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-69) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:57:44,959 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-70) starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 05:57:44,963 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-70) domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 05:57:44,968 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-70) [4dccea71] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:57:45,038 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-70) [4dccea71] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 05:57:45,060 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-70) [4dccea71] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 05:57:45,063 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-70) [4dccea71] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 05:57:45,064 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-70) [4dccea71] No string for UNASSIGNED type. Use default Log
>2012-11-09 05:57:51,366 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-80) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:57:51,388 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-80) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 127e2927
>2012-11-09 05:57:52,199 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-80) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:57:52,200 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-80) Vds: Ovirt
>2012-11-09 05:57:52,200 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-80) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:57:52,201 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-80) FINISH, ConnectStoragePoolVDSCommand, log id: 127e2927
>2012-11-09 05:57:52,202 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-80) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:57:52,202 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-80) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:57:52,210 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-80) [3f100e7f] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:57:52,282 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-80) [3f100e7f] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:57:52,323 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-80) [3f100e7f] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 4bc8e2d9
>2012-11-09 05:57:52,324 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-80) [3f100e7f] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:57:52,324 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-80) [3f100e7f] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 4bc8e2d9
>2012-11-09 05:57:53,482 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-85) [27d80cd9] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:57:55,158 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-89) starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 05:57:55,193 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-89) domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 05:57:55,198 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-89) [595cb87d] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:57:55,262 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-89) [595cb87d] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 05:57:55,282 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-89) [595cb87d] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 05:57:55,286 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-89) [595cb87d] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 05:57:55,287 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-89) [595cb87d] No string for UNASSIGNED type. Use default Log
>2012-11-09 05:58:02,405 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-4) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:58:02,428 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-4) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 23118257
>2012-11-09 05:58:03,240 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-4) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:58:03,242 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-4) Vds: Ovirt
>2012-11-09 05:58:03,242 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-4) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:58:03,243 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-4) FINISH, ConnectStoragePoolVDSCommand, log id: 23118257
>2012-11-09 05:58:03,243 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-4) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:58:03,244 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-4) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:58:03,252 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-4) [1b554dfb] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:58:03,321 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-4) [1b554dfb] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:58:03,364 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-4) [1b554dfb] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 3654018d
>2012-11-09 05:58:03,365 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-4) [1b554dfb] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:58:03,365 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-4) [1b554dfb] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 3654018d
>2012-11-09 05:58:03,672 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-8) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:58:05,365 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-7) [33efb190] starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 05:58:05,369 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-7) [33efb190] domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 05:58:05,374 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-7) [330181b2] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:58:05,457 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-7) [330181b2] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 05:58:05,477 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-7) [330181b2] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 05:58:05,480 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-7) [330181b2] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 05:58:05,481 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-7) [330181b2] No string for UNASSIGNED type. Use default Log
>2012-11-09 05:58:13,445 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-25) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:58:13,471 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-25) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 141ad5ce
>2012-11-09 05:58:13,904 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-24) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:58:14,285 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-25) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:58:14,286 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-25) Vds: Ovirt
>2012-11-09 05:58:14,286 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-25) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:58:14,287 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-25) FINISH, ConnectStoragePoolVDSCommand, log id: 141ad5ce
>2012-11-09 05:58:14,287 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-25) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:58:14,288 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-25) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:58:14,295 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-25) [7a97957c] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:58:14,368 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-25) [7a97957c] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:58:14,412 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-25) [7a97957c] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 2281ecf8
>2012-11-09 05:58:14,413 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-25) [7a97957c] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:58:14,414 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-25) [7a97957c] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 2281ecf8
>2012-11-09 05:58:24,084 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-38) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:58:24,494 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-46) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:58:24,511 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-46) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 307f1844
>2012-11-09 05:58:25,353 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-46) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:58:25,355 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-46) Vds: Ovirt
>2012-11-09 05:58:25,355 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-46) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:58:25,356 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-46) FINISH, ConnectStoragePoolVDSCommand, log id: 307f1844
>2012-11-09 05:58:25,357 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-46) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:58:25,357 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-46) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:58:25,365 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-46) [7e4e4198] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:58:25,451 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-46) [7e4e4198] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:58:25,493 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-46) [7e4e4198] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 4ef305c0
>2012-11-09 05:58:25,494 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-46) [7e4e4198] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:58:25,495 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-46) [7e4e4198] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 4ef305c0
>2012-11-09 05:58:34,282 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-60) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:58:35,574 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-59) [50c588] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:58:35,593 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-59) [50c588] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 42e62520
>2012-11-09 05:58:36,053 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-63) starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 05:58:36,057 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-63) domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 05:58:36,062 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-63) [178cc463] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:58:36,124 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-63) [178cc463] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 05:58:36,144 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-63) [178cc463] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 05:58:36,148 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-63) [178cc463] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 05:58:36,148 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-63) [178cc463] No string for UNASSIGNED type. Use default Log
>2012-11-09 05:58:36,400 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-59) [50c588] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:58:36,402 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-59) [50c588] Vds: Ovirt
>2012-11-09 05:58:36,402 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-59) [50c588] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:58:36,403 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-59) [50c588] FINISH, ConnectStoragePoolVDSCommand, log id: 42e62520
>2012-11-09 05:58:36,404 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-59) [50c588] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:58:36,404 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-59) [50c588] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:58:36,411 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-59) [35c61356] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:58:36,498 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-59) [35c61356] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:58:36,542 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-59) [35c61356] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 3811b739
>2012-11-09 05:58:36,543 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-59) [35c61356] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:58:36,544 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-59) [35c61356] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 3811b739
>2012-11-09 05:58:44,505 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-77) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:58:46,246 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-82) [66035589] starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 05:58:46,250 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-82) [66035589] domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 05:58:46,255 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-82) [11fb36b] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:58:46,322 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-82) [11fb36b] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 05:58:46,342 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-82) [11fb36b] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 05:58:46,345 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-82) [11fb36b] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 05:58:46,346 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-82) [11fb36b] No string for UNASSIGNED type. Use default Log
>2012-11-09 05:58:46,630 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-83) [6a411a6a] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:58:46,649 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-83) [6a411a6a] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 559cd57e
>2012-11-09 05:58:47,480 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-83) [6a411a6a] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:58:47,482 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-83) [6a411a6a] Vds: Ovirt
>2012-11-09 05:58:47,482 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-83) [6a411a6a] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:58:47,483 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-83) [6a411a6a] FINISH, ConnectStoragePoolVDSCommand, log id: 559cd57e
>2012-11-09 05:58:47,484 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-83) [6a411a6a] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:58:47,484 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-83) [6a411a6a] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:58:47,492 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-83) [7cf1d1b9] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:58:47,558 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-83) [7cf1d1b9] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:58:47,602 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-83) [7cf1d1b9] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 12798d1
>2012-11-09 05:58:47,603 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-83) [7cf1d1b9] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:58:47,603 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-83) [7cf1d1b9] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 12798d1
>2012-11-09 05:58:54,691 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-96) [72946543] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:58:56,439 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-95) [5b17058] starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 05:58:56,442 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-95) [5b17058] domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 05:58:56,447 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-95) [1deac176] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:58:56,516 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-95) [1deac176] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 05:58:56,528 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-95) [1deac176] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 05:58:56,530 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-95) [1deac176] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 05:58:56,530 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-95) [1deac176] No string for UNASSIGNED type. Use default Log
>2012-11-09 05:58:57,697 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-4) [1b554dfb] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:58:57,717 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-4) [1b554dfb] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 37d62db9
>2012-11-09 05:58:58,499 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-4) [1b554dfb] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:58:58,501 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-4) [1b554dfb] Vds: Ovirt
>2012-11-09 05:58:58,501 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-4) [1b554dfb] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:58:58,502 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-4) [1b554dfb] FINISH, ConnectStoragePoolVDSCommand, log id: 37d62db9
>2012-11-09 05:58:58,503 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-4) [1b554dfb] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:58:58,503 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-4) [1b554dfb] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:58:58,511 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-4) [2081aed0] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:58:58,577 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-4) [2081aed0] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:58:58,621 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-4) [2081aed0] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 5b696582
>2012-11-09 05:58:58,622 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-4) [2081aed0] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:58:58,622 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-4) [2081aed0] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 5b696582
>2012-11-09 05:59:04,883 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-17) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:59:06,631 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-21) starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 05:59:06,634 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-21) domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 05:59:06,639 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-21) [169e632] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:59:06,708 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-21) [169e632] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 05:59:06,729 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-21) [169e632] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 05:59:06,732 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-21) [169e632] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 05:59:06,733 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-21) [169e632] No string for UNASSIGNED type. Use default Log
>2012-11-09 05:59:08,700 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-24) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:59:08,736 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-24) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 3231845b
>2012-11-09 05:59:09,547 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-24) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:59:09,548 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-24) Vds: Ovirt
>2012-11-09 05:59:09,548 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-24) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:59:09,549 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-24) FINISH, ConnectStoragePoolVDSCommand, log id: 3231845b
>2012-11-09 05:59:09,550 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-24) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:59:09,550 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-24) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:59:09,557 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-24) [54131934] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:59:09,629 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-24) [54131934] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:59:09,670 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-24) [54131934] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 1d3b7a34
>2012-11-09 05:59:09,670 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-24) [54131934] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:59:09,671 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-24) [54131934] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 1d3b7a34
>2012-11-09 05:59:15,086 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-33) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:59:16,824 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-37) starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 05:59:16,827 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-37) domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 05:59:16,833 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-37) [7774543f] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:59:16,902 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-37) [7774543f] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 05:59:16,922 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-37) [7774543f] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 05:59:16,925 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-37) [7774543f] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 05:59:16,926 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-37) [7774543f] No string for UNASSIGNED type. Use default Log
>2012-11-09 05:59:19,755 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-46) [7e4e4198] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:59:19,780 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-46) [7e4e4198] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 7d99502
>2012-11-09 05:59:20,609 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-46) [7e4e4198] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:59:20,610 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-46) [7e4e4198] Vds: Ovirt
>2012-11-09 05:59:20,611 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-46) [7e4e4198] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:59:20,612 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-46) [7e4e4198] FINISH, ConnectStoragePoolVDSCommand, log id: 7d99502
>2012-11-09 05:59:20,612 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-46) [7e4e4198] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:59:20,613 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-46) [7e4e4198] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:59:20,620 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-46) [45a55fee] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:59:20,686 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-46) [45a55fee] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:59:20,731 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-46) [45a55fee] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: f7fc3d5
>2012-11-09 05:59:20,732 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-46) [45a55fee] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:59:20,732 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-46) [45a55fee] FINISH, MarkPoolInReconstructModeVDSCommand, log id: f7fc3d5
>2012-11-09 05:59:25,267 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-49) [2d1d8b93] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:59:27,045 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-55) starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 05:59:27,060 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-55) domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 05:59:27,065 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-55) [7462212d] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:59:27,131 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-55) [7462212d] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 05:59:27,151 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-55) [7462212d] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 05:59:27,154 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-55) [7462212d] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 05:59:27,155 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-55) [7462212d] No string for UNASSIGNED type. Use default Log
>2012-11-09 05:59:30,812 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-65) [27bb6518] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:59:31,024 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-65) [27bb6518] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 62674850
>2012-11-09 05:59:32,021 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-65) [27bb6518] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:59:32,021 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-65) [27bb6518] Vds: Ovirt
>2012-11-09 05:59:32,022 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-65) [27bb6518] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:59:32,022 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-65) [27bb6518] FINISH, ConnectStoragePoolVDSCommand, log id: 62674850
>2012-11-09 05:59:32,022 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-65) [27bb6518] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:59:32,023 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-65) [27bb6518] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:59:32,027 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-65) [30ec6c4c] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:59:32,109 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-65) [30ec6c4c] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:59:32,153 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-65) [30ec6c4c] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 1547df50
>2012-11-09 05:59:32,154 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-65) [30ec6c4c] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:59:32,154 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-65) [30ec6c4c] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 1547df50
>2012-11-09 05:59:35,474 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-74) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:59:37,244 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-68) [5bb24693] starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 05:59:37,247 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-68) [5bb24693] domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 05:59:37,253 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-68) [79e52acc] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:59:37,316 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-68) [79e52acc] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 05:59:37,337 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-68) [79e52acc] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 05:59:37,340 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-68) [79e52acc] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 05:59:37,341 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-68) [79e52acc] No string for UNASSIGNED type. Use default Log
>2012-11-09 05:59:42,232 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-80) [3f100e7f] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:59:42,373 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-80) [3f100e7f] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 72a48fe4
>2012-11-09 05:59:43,162 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-80) [3f100e7f] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:59:43,163 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-80) [3f100e7f] Vds: Ovirt
>2012-11-09 05:59:43,164 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-80) [3f100e7f] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:59:43,165 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-80) [3f100e7f] FINISH, ConnectStoragePoolVDSCommand, log id: 72a48fe4
>2012-11-09 05:59:43,165 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-80) [3f100e7f] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:59:43,166 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-80) [3f100e7f] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:59:43,173 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-80) [7d80820a] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:59:43,239 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-80) [7d80820a] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:59:43,282 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-80) [7d80820a] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 298941eb
>2012-11-09 05:59:43,283 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-80) [7d80820a] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:59:43,284 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-80) [7d80820a] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 298941eb
>2012-11-09 05:59:45,723 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-90) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:59:47,446 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-93) [1a175520] starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 05:59:47,450 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-93) [1a175520] domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 05:59:47,455 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-93) [2cf94af7] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:59:47,526 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-93) [2cf94af7] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 05:59:47,548 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-93) [2cf94af7] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 05:59:47,551 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-93) [2cf94af7] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 05:59:47,551 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-93) [2cf94af7] No string for UNASSIGNED type. Use default Log
>2012-11-09 05:59:53,359 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-6) [41b005f9] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 05:59:53,403 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-6) [41b005f9] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 60ed9de0
>2012-11-09 05:59:54,281 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-6) [41b005f9] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 05:59:54,282 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-6) [41b005f9] Vds: Ovirt
>2012-11-09 05:59:54,282 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-6) [41b005f9] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:59:54,283 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-6) [41b005f9] FINISH, ConnectStoragePoolVDSCommand, log id: 60ed9de0
>2012-11-09 05:59:54,283 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-6) [41b005f9] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 05:59:54,284 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-6) [41b005f9] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 05:59:54,288 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-6) [2e1c60f4] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:59:54,487 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-6) [2e1c60f4] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 05:59:54,602 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-6) [2e1c60f4] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 54d60e1d
>2012-11-09 05:59:54,603 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-6) [2e1c60f4] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 05:59:54,604 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-6) [2e1c60f4] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 54d60e1d
>2012-11-09 05:59:55,971 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-10) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 05:59:57,644 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-16) [5ba61d90] starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 05:59:57,648 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-16) [5ba61d90] domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 05:59:57,655 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-16) [1b7f8ea7] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 05:59:57,762 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-16) [1b7f8ea7] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 05:59:57,784 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-16) [1b7f8ea7] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 05:59:57,787 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-16) [1b7f8ea7] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 05:59:57,788 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-16) [1b7f8ea7] No string for UNASSIGNED type. Use default Log
>2012-11-09 05:59:58,157 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-1) calling GetConfigurationValueQuery (ApplicationMode) with null version, using default general for version
>2012-11-09 06:00:00,001 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-14) [371014e1] Checking autorecoverable hosts
>2012-11-09 06:00:00,004 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-14) [371014e1] Autorecovering 0 hosts
>2012-11-09 06:00:00,004 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-14) [371014e1] Checking autorecoverable hosts done
>2012-11-09 06:00:00,004 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-14) [371014e1] Checking autorecoverable storage domains
>2012-11-09 06:00:00,042 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-14) [371014e1] Autorecovering 1 storage domains
>2012-11-09 06:00:00,043 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-14) [371014e1] Autorecovering storage domains: b1658676-2dc3-4357-b112-3852a056c663
>2012-11-09 06:00:00,044 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-14) [27187068] Lock Acquired to object EngineLock [exclusiveLocks= key: b1658676-2dc3-4357-b112-3852a056c663 value: STORAGE
>, sharedLocks= ]
>2012-11-09 06:00:00,063 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-14) [27187068] Running command: ActivateStorageDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:00:00,083 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-14) [27187068] Lock freed to object EngineLock [exclusiveLocks= key: b1658676-2dc3-4357-b112-3852a056c663 value: STORAGE
>, sharedLocks= ]
>2012-11-09 06:00:00,084 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-14) [27187068] ActivateStorage Domain. Before Connect all hosts to pool. Time:11/9/12 6:00 AM
>2012-11-09 06:00:00,095 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-3-thread-49) [1d7873eb] START, ValidateStorageServerConnectionVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 039341b1-a8c0-4003-845a-4ef53f74c7c8, connection: localhost:/mnt/data };]), log id: d7abbe
>2012-11-09 06:00:00,112 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-3-thread-49) [1d7873eb] FINISH, ValidateStorageServerConnectionVDSCommand, return: {039341b1-a8c0-4003-845a-4ef53f74c7c8=0}, log id: d7abbe
>2012-11-09 06:00:00,112 INFO  [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-3-thread-49) [1d7873eb] Running command: ConnectStorageToVdsCommand internal: true. Entities affected :  ID: aaa00000-0000-0000-0000-123456789aaa Type: System
>2012-11-09 06:00:00,114 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-3-thread-49) [1d7873eb] START, ConnectStorageServerVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 039341b1-a8c0-4003-845a-4ef53f74c7c8, connection: localhost:/mnt/data };]), log id: 608e7dd1
>2012-11-09 06:00:00,140 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-3-thread-49) [1d7873eb] FINISH, ConnectStorageServerVDSCommand, return: {039341b1-a8c0-4003-845a-4ef53f74c7c8=0}, log id: 608e7dd1
>2012-11-09 06:00:00,142 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-14) [27187068] ActivateStorage Domain. After Connect all hosts to pool. Time:11/9/12 6:00 AM
>2012-11-09 06:00:00,143 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (QuartzScheduler_Worker-14) [27187068] START, ActivateStorageDomainVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, storageDomainId = b1658676-2dc3-4357-b112-3852a056c663), log id: 61a3f58f
>2012-11-09 06:00:00,170 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-14) [27187068] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:00:00,214 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-14) [27187068] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: d87eca4
>2012-11-09 06:00:01,051 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-14) [27187068] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:00:01,052 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-14) [27187068] Vds: Ovirt
>2012-11-09 06:00:01,053 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-14) [27187068] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:00:01,054 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-14) [27187068] FINISH, ConnectStoragePoolVDSCommand, log id: d87eca4
>2012-11-09 06:00:01,054 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-14) [27187068] IrsBroker::Failed::ActivateStorageDomainVDS
>2012-11-09 06:00:01,055 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-14) [27187068] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:00:01,062 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-14) [47c997e0] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:00:01,093 WARN  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-14) [47c997e0] CanDoAction of action ReconstructMasterDomain failed. Reasons:VAR__ACTION__RECONSTRUCT_MASTER,VAR__TYPE__STORAGE__DOMAIN,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL2,$status Locked
>2012-11-09 06:00:01,111 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (QuartzScheduler_Worker-14) [47c997e0] FINISH, ActivateStorageDomainVDSCommand, log id: 61a3f58f
>2012-11-09 06:00:01,111 ERROR [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-14) [47c997e0] Command org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.irsbroker.IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:00:01,118 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-14) [47c997e0] Command [id=28af9773-9614-467f-8c10-587ab2228913]: Compensating CHANGED_STATUS_ONLY of org.ovirt.engine.core.common.businessentities.storage_pool_iso_map; snapshot: EntityStatusSnapshot [id=storagePoolId = c02503a4-274e-11e2-b33c-002590730631, storageId = b1658676-2dc3-4357-b112-3852a056c663, status=InActive].
>2012-11-09 06:00:01,132 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-14) [47c997e0] Checking autorecoverable storage domains done
>2012-11-09 06:00:04,217 INFO  [org.ovirt.engine.core.bll.LoginAdminUserCommand] (http--0.0.0.0-8443-1) Checking if user admin@internal is an admin, result true
>2012-11-09 06:00:04,217 INFO  [org.ovirt.engine.core.bll.LoginAdminUserCommand] (http--0.0.0.0-8443-1) Running command: LoginAdminUserCommand internal: false.
>2012-11-09 06:00:04,284 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-1) calling GetConfigurationValueQuery (RedirectServletReportsPage) with null version, using default general for version
>2012-11-09 06:00:04,301 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-1) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,318 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-1) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,323 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-1) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,328 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-1) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,333 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-1) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,337 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-1) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,344 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,348 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-1) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,675 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-24) [54131934] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:00:04,692 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-24) [54131934] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 6904d7a9
>2012-11-09 06:00:04,779 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-1) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,779 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (ManagementNetwork) with null version, using default general for version
>2012-11-09 06:00:04,780 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,808 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,913 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (HighUtilizationForPowerSave) with null version, using default general for version
>2012-11-09 06:00:04,915 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (LowUtilizationForPowerSave) with null version, using default general for version
>2012-11-09 06:00:04,915 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-1) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,916 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,918 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (HighUtilizationForEvenlyDistribute) with null version, using default general for version
>2012-11-09 06:00:04,920 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,921 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,921 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,923 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,924 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,925 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,928 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-1) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,928 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (ManagementNetwork) with null version, using default general for version
>2012-11-09 06:00:04,930 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,934 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,934 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,936 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,939 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,939 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,940 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,941 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,943 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,943 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,946 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,946 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,947 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,949 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,949 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,952 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (HighUtilizationForEvenlyDistribute) with null version, using default general for version
>2012-11-09 06:00:04,952 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (LowUtilizationForPowerSave) with null version, using default general for version
>2012-11-09 06:00:04,952 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,952 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,952 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (HighUtilizationForPowerSave) with null version, using default general for version
>2012-11-09 06:00:04,956 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,956 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,957 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,959 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,959 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,961 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,963 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,964 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-4) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,965 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (ManagementNetwork) with null version, using default general for version
>2012-11-09 06:00:04,966 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,967 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,967 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,970 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,971 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,971 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,972 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,975 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,975 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,976 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,978 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,979 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,981 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,981 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,982 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,984 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,985 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,985 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,987 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,987 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,991 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,991 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,991 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (ManagementNetwork) with null version, using default general for version
>2012-11-09 06:00:04,992 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,993 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,994 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,995 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,997 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,998 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:04,998 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,000 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,001 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,001 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,003 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,004 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,005 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,006 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,008 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,008 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,012 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,012 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,013 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,016 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,022 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,022 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,023 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,025 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,026 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,028 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,028 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,038 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,040 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,041 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,041 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,045 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,047 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,049 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,049 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,050 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,052 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,055 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,057 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,057 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,058 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,060 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,060 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,063 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,063 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,066 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,066 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,066 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,069 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,069 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,071 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,073 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,073 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,074 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,076 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,076 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,077 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,079 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,079 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,082 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,082 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,083 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,085 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,085 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,086 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,089 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,089 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,090 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,093 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-3) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,093 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-5) calling GetConfigurationValueQuery (SearchResultsLimit) with null version, using default general for version
>2012-11-09 06:00:05,094 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-2) calling GetConfigurationValueQuery (DocsURL) with null version, using default general for version
>2012-11-09 06:00:05,531 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-24) [54131934] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:00:05,533 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-24) [54131934] Vds: Ovirt
>2012-11-09 06:00:05,533 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-24) [54131934] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:00:05,534 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-24) [54131934] FINISH, ConnectStoragePoolVDSCommand, log id: 6904d7a9
>2012-11-09 06:00:05,535 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-24) [54131934] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:00:05,535 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-24) [54131934] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:00:05,543 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-24) [5a56435a] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:00:05,628 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-24) [5a56435a] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:00:05,668 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-24) [5a56435a] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 2d8acae0
>2012-11-09 06:00:05,668 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-24) [5a56435a] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:00:05,669 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-24) [5a56435a] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 2d8acae0
>2012-11-09 06:00:06,162 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-26) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:00:07,871 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-33) starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 06:00:07,888 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-33) domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 06:00:07,893 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-33) [5b989670] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:00:07,962 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-33) [5b989670] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 06:00:08,004 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-33) [5b989670] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 06:00:08,008 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-33) [5b989670] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 06:00:08,009 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-33) [5b989670] No string for UNASSIGNED type. Use default Log
>2012-11-09 06:00:15,754 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-42) [4242cb9f] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:00:15,777 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-42) [4242cb9f] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 650120ee
>2012-11-09 06:00:16,374 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-45) [163cb8df] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:00:16,633 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-42) [4242cb9f] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:00:16,635 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-42) [4242cb9f] Vds: Ovirt
>2012-11-09 06:00:16,636 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-42) [4242cb9f] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:00:16,637 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-42) [4242cb9f] FINISH, ConnectStoragePoolVDSCommand, log id: 650120ee
>2012-11-09 06:00:16,637 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-42) [4242cb9f] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:00:16,638 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-42) [4242cb9f] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:00:16,646 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-42) [53cf458a] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:00:16,704 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-42) [53cf458a] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:00:16,750 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-42) [53cf458a] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 940460
>2012-11-09 06:00:16,750 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-42) [53cf458a] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:00:16,751 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-42) [53cf458a] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 940460
>2012-11-09 06:00:26,567 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-59) [35c61356] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:00:26,827 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-65) [30ec6c4c] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:00:26,846 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-65) [30ec6c4c] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 427c9817
>2012-11-09 06:00:27,699 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-65) [30ec6c4c] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:00:27,700 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-65) [30ec6c4c] Vds: Ovirt
>2012-11-09 06:00:27,701 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-65) [30ec6c4c] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:00:27,701 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-65) [30ec6c4c] FINISH, ConnectStoragePoolVDSCommand, log id: 427c9817
>2012-11-09 06:00:27,702 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-65) [30ec6c4c] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:00:27,702 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-65) [30ec6c4c] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:00:27,709 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-65) [54340bfe] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:00:27,785 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-65) [54340bfe] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:00:27,831 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-65) [54340bfe] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 614149ff
>2012-11-09 06:00:27,832 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-65) [54340bfe] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:00:27,832 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-65) [54340bfe] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 614149ff
>2012-11-09 06:00:36,759 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-80) [7d80820a] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:00:37,902 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-86) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:00:37,929 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-86) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 32daf8bd
>2012-11-09 06:00:38,751 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-86) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:00:38,753 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-86) Vds: Ovirt
>2012-11-09 06:00:38,753 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-86) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:00:38,754 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-86) FINISH, ConnectStoragePoolVDSCommand, log id: 32daf8bd
>2012-11-09 06:00:38,755 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-86) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:00:38,755 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-86) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:00:38,766 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-86) [1717f3a9] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:00:38,819 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-92) starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 06:00:38,838 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-86) [1717f3a9] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:00:38,838 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-92) domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 06:00:38,843 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-92) [58a8ab41] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:00:38,921 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-86) [1717f3a9] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 65cdebc9
>2012-11-09 06:00:38,934 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-92) [58a8ab41] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 06:00:38,954 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-92) [58a8ab41] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 06:00:38,957 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-92) [58a8ab41] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 06:00:38,958 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-92) [58a8ab41] No string for UNASSIGNED type. Use default Log
>2012-11-09 06:00:38,976 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-86) [1717f3a9] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:00:38,976 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-86) [1717f3a9] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 65cdebc9
>2012-11-09 06:00:46,953 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-1) [74d9341e] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:00:49,013 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-12) starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 06:00:49,017 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-12) domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 06:00:49,021 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-12) [6daae201] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:00:49,024 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-9) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:00:49,050 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-9) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 6f883d4d
>2012-11-09 06:00:49,407 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-12) [6daae201] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 06:00:49,427 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-12) [6daae201] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 06:00:49,430 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-12) [6daae201] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 06:00:49,431 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-12) [6daae201] No string for UNASSIGNED type. Use default Log
>2012-11-09 06:00:49,841 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-9) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:00:49,843 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-9) Vds: Ovirt
>2012-11-09 06:00:49,843 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-9) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:00:49,844 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-9) FINISH, ConnectStoragePoolVDSCommand, log id: 6f883d4d
>2012-11-09 06:00:49,845 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-9) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:00:49,845 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-9) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:00:49,853 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-9) [15914def] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:00:49,914 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-9) [15914def] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:00:49,960 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-9) [15914def] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 52ea508c
>2012-11-09 06:00:49,961 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-9) [15914def] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:00:49,962 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-9) [15914def] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 52ea508c
>2012-11-09 06:00:56,100 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-6) calling GetConfigurationValueQuery (SpiceUsbAutoShare) with null version, using default general for version
>2012-11-09 06:00:56,101 WARN  [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8443-1) calling GetConfigurationValueQuery (EnableUSBAsDefault) with null version, using default general for version
>2012-11-09 06:00:57,144 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-14) [47c997e0] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:00:59,190 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-26) starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 06:00:59,194 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-26) domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 06:00:59,199 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-26) [10f41170] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:00:59,260 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-26) [10f41170] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 06:00:59,280 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-26) [10f41170] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 06:00:59,283 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-26) [10f41170] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 06:00:59,284 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-26) [10f41170] No string for UNASSIGNED type. Use default Log
>2012-11-09 06:01:00,035 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-28) [28303158] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:01:00,056 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-28) [28303158] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 69a502c9
>2012-11-09 06:01:00,897 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-28) [28303158] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:01:00,899 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-28) [28303158] Vds: Ovirt
>2012-11-09 06:01:00,899 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-28) [28303158] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:01:00,900 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-28) [28303158] FINISH, ConnectStoragePoolVDSCommand, log id: 69a502c9
>2012-11-09 06:01:00,901 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-28) [28303158] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:01:00,901 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-28) [28303158] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:01:00,909 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-28) [3969a9ab] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:01:00,979 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-28) [3969a9ab] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:01:01,022 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-28) [3969a9ab] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 5426135e
>2012-11-09 06:01:01,023 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-28) [3969a9ab] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:01:01,023 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-28) [3969a9ab] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 5426135e
>2012-11-09 06:01:07,376 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-39) [55d3e877] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:01:09,393 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-44) [3aed7df5] starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 06:01:09,397 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-44) [3aed7df5] domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 06:01:09,402 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-44) [2abc5f3b] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:01:09,466 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-44) [2abc5f3b] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 06:01:09,487 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-44) [2abc5f3b] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 06:01:09,490 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-44) [2abc5f3b] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 06:01:09,490 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-44) [2abc5f3b] No string for UNASSIGNED type. Use default Log
>2012-11-09 06:01:11,098 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-48) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:01:11,120 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-48) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 67bed8b8
>2012-11-09 06:01:11,951 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-48) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:01:11,953 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-48) Vds: Ovirt
>2012-11-09 06:01:11,953 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-48) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:01:11,954 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-48) FINISH, ConnectStoragePoolVDSCommand, log id: 67bed8b8
>2012-11-09 06:01:11,955 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-48) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:01:11,955 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-48) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:01:11,962 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-48) [6dd3e912] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:01:12,033 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-48) [6dd3e912] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:01:12,073 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-48) [6dd3e912] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 6eaf5f45
>2012-11-09 06:01:12,074 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-48) [6dd3e912] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:01:12,075 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-48) [6dd3e912] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 6eaf5f45
>2012-11-09 06:01:17,563 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-61) [18dd424d] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:01:22,156 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-71) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:01:22,183 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-71) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 39a44468
>2012-11-09 06:01:23,279 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-71) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:01:23,281 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-71) Vds: Ovirt
>2012-11-09 06:01:23,281 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-71) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:01:23,282 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-71) FINISH, ConnectStoragePoolVDSCommand, log id: 39a44468
>2012-11-09 06:01:23,282 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-71) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:01:23,283 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-71) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:01:23,290 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-71) [7d10ffa4] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:01:23,363 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-71) [7d10ffa4] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:01:23,404 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-71) [7d10ffa4] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 208e42ea
>2012-11-09 06:01:23,405 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-71) [7d10ffa4] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:01:23,406 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-71) [7d10ffa4] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 208e42ea
>2012-11-09 06:01:27,761 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-81) [26a17f04] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:01:33,486 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-92) [58a8ab41] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:01:33,509 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-92) [58a8ab41] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 709d7d14
>2012-11-09 06:01:34,314 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-92) [58a8ab41] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:01:34,316 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-92) [58a8ab41] Vds: Ovirt
>2012-11-09 06:01:34,316 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-92) [58a8ab41] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:01:34,317 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-92) [58a8ab41] FINISH, ConnectStoragePoolVDSCommand, log id: 709d7d14
>2012-11-09 06:01:34,318 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-92) [58a8ab41] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:01:34,318 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-92) [58a8ab41] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:01:34,326 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-92) [4178a315] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:01:34,394 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-92) [4178a315] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:01:34,440 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-92) [4178a315] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 6f5892e
>2012-11-09 06:01:34,440 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-92) [4178a315] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:01:34,441 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-92) [4178a315] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 6f5892e
>2012-11-09 06:01:37,951 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-96) [72946543] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:01:44,517 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-7) [330181b2] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:01:44,540 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-7) [330181b2] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 77109735
>2012-11-09 06:01:45,350 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-7) [330181b2] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:01:45,352 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-7) [330181b2] Vds: Ovirt
>2012-11-09 06:01:45,352 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-7) [330181b2] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:01:45,353 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-7) [330181b2] FINISH, ConnectStoragePoolVDSCommand, log id: 77109735
>2012-11-09 06:01:45,354 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-7) [330181b2] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:01:45,354 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-7) [330181b2] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:01:45,363 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-7) [b37f953] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:01:45,438 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-7) [b37f953] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:01:45,481 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-7) [b37f953] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 7e790631
>2012-11-09 06:01:45,482 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-7) [b37f953] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:01:45,483 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-7) [b37f953] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 7e790631
>2012-11-09 06:01:48,153 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-15) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:01:55,568 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-27) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:01:55,582 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-27) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: a8c392c
>2012-11-09 06:01:56,424 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-27) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:01:56,425 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-27) Vds: Ovirt
>2012-11-09 06:01:56,426 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-27) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:01:56,426 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-27) FINISH, ConnectStoragePoolVDSCommand, log id: a8c392c
>2012-11-09 06:01:56,426 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-27) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:01:56,427 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-27) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:01:56,433 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-27) [1e482fc7] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:01:56,501 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-27) [1e482fc7] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:01:56,542 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-27) [1e482fc7] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 656d3b16
>2012-11-09 06:01:56,543 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-27) [1e482fc7] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:01:56,543 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-27) [1e482fc7] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 656d3b16
>2012-11-09 06:01:58,352 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-33) [5b989670] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:02:06,625 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-42) [53cf458a] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:02:06,647 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-42) [53cf458a] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 69f64b70
>2012-11-09 06:02:07,460 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-42) [53cf458a] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:02:07,461 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-42) [53cf458a] Vds: Ovirt
>2012-11-09 06:02:07,462 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-42) [53cf458a] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:02:07,463 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-42) [53cf458a] FINISH, ConnectStoragePoolVDSCommand, log id: 69f64b70
>2012-11-09 06:02:07,463 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-42) [53cf458a] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:02:07,464 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-42) [53cf458a] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:02:07,471 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-42) [2e24deb8] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:02:07,541 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-42) [2e24deb8] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:02:07,583 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-42) [2e24deb8] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 75e01c92
>2012-11-09 06:02:07,584 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-42) [2e24deb8] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:02:07,584 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-42) [2e24deb8] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 75e01c92
>2012-11-09 06:02:08,542 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-50) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:02:17,665 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-62) [e47bdb5] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:02:17,687 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-62) [e47bdb5] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 2b487860
>2012-11-09 06:02:18,502 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-62) [e47bdb5] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:02:18,504 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-62) [e47bdb5] Vds: Ovirt
>2012-11-09 06:02:18,504 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-62) [e47bdb5] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:02:18,505 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-62) [e47bdb5] FINISH, ConnectStoragePoolVDSCommand, log id: 2b487860
>2012-11-09 06:02:18,506 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-62) [e47bdb5] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:02:18,506 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-62) [e47bdb5] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:02:18,513 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-62) [54520a5b] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:02:18,587 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-62) [54520a5b] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:02:18,631 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-62) [54520a5b] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 7ae514dd
>2012-11-09 06:02:18,631 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-62) [54520a5b] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:02:18,632 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-62) [54520a5b] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 7ae514dd
>2012-11-09 06:02:18,721 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-69) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:02:28,708 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-83) [7cf1d1b9] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:02:28,728 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-83) [7cf1d1b9] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 43b9cf3
>2012-11-09 06:02:29,589 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-83) [7cf1d1b9] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:02:29,591 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-83) [7cf1d1b9] Vds: Ovirt
>2012-11-09 06:02:29,591 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-83) [7cf1d1b9] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:02:29,592 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-83) [7cf1d1b9] FINISH, ConnectStoragePoolVDSCommand, log id: 43b9cf3
>2012-11-09 06:02:29,593 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-83) [7cf1d1b9] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:02:29,593 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-83) [7cf1d1b9] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:02:29,618 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-83) [704409ec] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:02:29,688 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-83) [704409ec] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:02:29,733 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-83) [704409ec] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 457312ef
>2012-11-09 06:02:29,734 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-83) [704409ec] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:02:29,734 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-83) [704409ec] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 457312ef
>2012-11-09 06:02:37,302 INFO  [org.ovirt.engine.core.bll.HotPlugDiskToVmCommand] (pool-3-thread-49) [3addb7bb] Running command: HotPlugDiskToVmCommand internal: false. Entities affected :  ID: 0e64bf87-8996-41be-9f28-a7126c815d70 Type: VM
>2012-11-09 06:02:39,376 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-3) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:02:39,814 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-1) [74d9341e] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:02:39,830 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-1) [74d9341e] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 32685b2
>2012-11-09 06:02:40,642 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-1) [74d9341e] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:02:40,644 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-1) [74d9341e] Vds: Ovirt
>2012-11-09 06:02:40,644 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-1) [74d9341e] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:02:40,645 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-1) [74d9341e] FINISH, ConnectStoragePoolVDSCommand, log id: 32685b2
>2012-11-09 06:02:40,646 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-1) [74d9341e] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:02:40,646 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-1) [74d9341e] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:02:40,654 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-1) [2df3d062] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:02:40,727 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-1) [2df3d062] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:02:40,771 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-1) [2df3d062] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 495de463
>2012-11-09 06:02:40,772 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-1) [2df3d062] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:02:40,772 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-1) [2df3d062] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 495de463
>2012-11-09 06:02:44,806 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IsValidVDSCommand] (http--0.0.0.0-8443-6) START, IsValidVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null), log id: 68d43a97
>2012-11-09 06:02:44,807 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IsValidVDSCommand] (http--0.0.0.0-8443-6) FINISH, IsValidVDSCommand, return: false, log id: 68d43a97
>2012-11-09 06:02:44,808 WARN  [org.ovirt.engine.core.bll.RunVmCommand] (http--0.0.0.0-8443-6) CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,ACTION_TYPE_FAILED_IMAGE_REPOSITORY_NOT_FOUND
>2012-11-09 06:02:49,545 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-21) [169e632] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:02:50,853 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-14) [47c997e0] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:02:50,894 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-14) [47c997e0] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 7c8e6e24
>2012-11-09 06:02:51,703 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-14) [47c997e0] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:02:51,705 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-14) [47c997e0] Vds: Ovirt
>2012-11-09 06:02:51,705 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-14) [47c997e0] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:02:51,706 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-14) [47c997e0] FINISH, ConnectStoragePoolVDSCommand, log id: 7c8e6e24
>2012-11-09 06:02:51,707 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-14) [47c997e0] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:02:51,707 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-14) [47c997e0] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:02:51,714 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-14) [7f8a7815] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:02:51,779 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-14) [7f8a7815] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:02:51,822 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-14) [7f8a7815] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 3d96beb4
>2012-11-09 06:02:51,823 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-14) [7f8a7815] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:02:51,824 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-14) [7f8a7815] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 3d96beb4
>2012-11-09 06:02:59,740 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-35) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:03:01,902 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-43) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:03:01,920 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-43) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 57937126
>2012-11-09 06:03:02,723 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-43) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:03:02,724 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-43) Vds: Ovirt
>2012-11-09 06:03:02,724 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-43) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:03:02,725 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-43) FINISH, ConnectStoragePoolVDSCommand, log id: 57937126
>2012-11-09 06:03:02,726 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-43) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:03:02,726 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-43) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:03:02,734 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-43) [32075a2e] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:03:02,806 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-43) [32075a2e] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:03:02,846 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-43) [32075a2e] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 45b24b38
>2012-11-09 06:03:02,847 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-43) [32075a2e] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:03:02,847 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-43) [32075a2e] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 45b24b38
>2012-11-09 06:03:08,125 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (pool-3-thread-49) [20bb773f] Lock Acquired to object EngineLock [exclusiveLocks= key: b1658676-2dc3-4357-b112-3852a056c663 value: STORAGE
>, sharedLocks= ]
>2012-11-09 06:03:08,150 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (pool-3-thread-49) [20bb773f] Running command: ActivateStorageDomainCommand internal: false. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:03:08,171 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (pool-3-thread-49) [20bb773f] Lock freed to object EngineLock [exclusiveLocks= key: b1658676-2dc3-4357-b112-3852a056c663 value: STORAGE
>, sharedLocks= ]
>2012-11-09 06:03:08,171 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (pool-3-thread-49) [20bb773f] ActivateStorage Domain. Before Connect all hosts to pool. Time:11/9/12 6:03 AM
>2012-11-09 06:03:08,181 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-3-thread-50) [4235f367] START, ValidateStorageServerConnectionVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 039341b1-a8c0-4003-845a-4ef53f74c7c8, connection: localhost:/mnt/data };]), log id: 13cc048a
>2012-11-09 06:03:08,194 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-3-thread-50) [4235f367] FINISH, ValidateStorageServerConnectionVDSCommand, return: {039341b1-a8c0-4003-845a-4ef53f74c7c8=0}, log id: 13cc048a
>2012-11-09 06:03:08,195 INFO  [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-3-thread-50) [4235f367] Running command: ConnectStorageToVdsCommand internal: true. Entities affected :  ID: aaa00000-0000-0000-0000-123456789aaa Type: System
>2012-11-09 06:03:08,197 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-3-thread-50) [4235f367] START, ConnectStorageServerVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 039341b1-a8c0-4003-845a-4ef53f74c7c8, connection: localhost:/mnt/data };]), log id: 5bab35ca
>2012-11-09 06:03:08,215 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-3-thread-50) [4235f367] FINISH, ConnectStorageServerVDSCommand, return: {039341b1-a8c0-4003-845a-4ef53f74c7c8=0}, log id: 5bab35ca
>2012-11-09 06:03:08,216 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (pool-3-thread-49) [20bb773f] ActivateStorage Domain. After Connect all hosts to pool. Time:11/9/12 6:03 AM
>2012-11-09 06:03:08,217 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (pool-3-thread-49) [20bb773f] START, ActivateStorageDomainVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, storageDomainId = b1658676-2dc3-4357-b112-3852a056c663), log id: 1d8d6ff2
>2012-11-09 06:03:08,234 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-3-thread-49) [20bb773f] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:03:08,252 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-3-thread-49) [20bb773f] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 12431e8c
>2012-11-09 06:03:09,151 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (pool-3-thread-49) [20bb773f] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:03:09,153 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (pool-3-thread-49) [20bb773f] Vds: Ovirt
>2012-11-09 06:03:09,153 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (pool-3-thread-49) [20bb773f] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:03:09,154 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-3-thread-49) [20bb773f] FINISH, ConnectStoragePoolVDSCommand, log id: 12431e8c
>2012-11-09 06:03:09,155 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-3-thread-49) [20bb773f] IrsBroker::Failed::ActivateStorageDomainVDS
>2012-11-09 06:03:09,155 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-3-thread-49) [20bb773f] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:03:09,162 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (pool-3-thread-49) [7ff784d4] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:03:09,191 WARN  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (pool-3-thread-49) [7ff784d4] CanDoAction of action ReconstructMasterDomain failed. Reasons:VAR__ACTION__RECONSTRUCT_MASTER,VAR__TYPE__STORAGE__DOMAIN,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL2,$status Locked
>2012-11-09 06:03:09,209 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (pool-3-thread-49) [7ff784d4] FINISH, ActivateStorageDomainVDSCommand, log id: 1d8d6ff2
>2012-11-09 06:03:09,209 ERROR [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (pool-3-thread-49) [7ff784d4] Command org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.irsbroker.IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:03:09,215 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (pool-3-thread-49) [7ff784d4] Command [id=2c5eceaf-6d39-4a4c-94ce-1752696b66b1]: Compensating CHANGED_STATUS_ONLY of org.ovirt.engine.core.common.businessentities.storage_pool_iso_map; snapshot: EntityStatusSnapshot [id=storagePoolId = c02503a4-274e-11e2-b33c-002590730631, storageId = b1658676-2dc3-4357-b112-3852a056c663, status=InActive].
>2012-11-09 06:03:09,944 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-56) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:03:12,927 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-60) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:03:12,961 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-60) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 2a0dd7e
>2012-11-09 06:03:13,780 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-60) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:03:13,781 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-60) Vds: Ovirt
>2012-11-09 06:03:13,782 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-60) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:03:13,783 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-60) FINISH, ConnectStoragePoolVDSCommand, log id: 2a0dd7e
>2012-11-09 06:03:13,784 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-60) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:03:13,784 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-60) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:03:13,791 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-60) [715b2491] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:03:13,854 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-60) [715b2491] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:03:13,900 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-60) [715b2491] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 7324cccc
>2012-11-09 06:03:13,900 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-60) [715b2491] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:03:13,901 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-60) [715b2491] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 7324cccc
>2012-11-09 06:03:20,142 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-72) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:03:23,975 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-77) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:03:23,992 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-77) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 1d828e25
>2012-11-09 06:03:24,084 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-79) starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 06:03:24,088 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-79) domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 06:03:24,093 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-79) [771cdbe9] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:03:24,424 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-79) [771cdbe9] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 06:03:24,445 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-79) [771cdbe9] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 06:03:24,448 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-79) [771cdbe9] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 06:03:24,449 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-79) [771cdbe9] No string for UNASSIGNED type. Use default Log
>2012-11-09 06:03:24,798 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-77) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:03:24,800 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-77) Vds: Ovirt
>2012-11-09 06:03:24,800 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-77) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:03:24,801 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-77) FINISH, ConnectStoragePoolVDSCommand, log id: 1d828e25
>2012-11-09 06:03:24,801 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-77) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:03:24,802 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-77) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:03:24,809 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-77) [1ef92680] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:03:24,881 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-77) [1ef92680] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:03:24,923 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-77) [1ef92680] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 627f316
>2012-11-09 06:03:24,924 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-77) [1ef92680] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:03:24,925 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-77) [1ef92680] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 627f316
>2012-11-09 06:03:30,341 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-90) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:03:35,006 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-3) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:03:35,053 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-3) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 69c605fe
>2012-11-09 06:03:35,878 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-3) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:03:35,879 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-3) Vds: Ovirt
>2012-11-09 06:03:35,879 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-3) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:03:35,880 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-3) FINISH, ConnectStoragePoolVDSCommand, log id: 69c605fe
>2012-11-09 06:03:35,880 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-3) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:03:35,880 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-3) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:03:35,885 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-3) [60b43093] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:03:35,955 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-3) [60b43093] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:03:36,000 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-3) [60b43093] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 5badce01
>2012-11-09 06:03:36,001 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-3) [60b43093] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:03:36,001 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-3) [60b43093] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 5badce01
>2012-11-09 06:03:37,199 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (pool-3-thread-49) [1efd6270] Lock Acquired to object EngineLock [exclusiveLocks= key: b1658676-2dc3-4357-b112-3852a056c663 value: STORAGE
>, sharedLocks= ]
>2012-11-09 06:03:37,224 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (pool-3-thread-49) [1efd6270] Running command: ActivateStorageDomainCommand internal: false. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:03:37,245 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (pool-3-thread-49) [1efd6270] Lock freed to object EngineLock [exclusiveLocks= key: b1658676-2dc3-4357-b112-3852a056c663 value: STORAGE
>, sharedLocks= ]
>2012-11-09 06:03:37,245 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (pool-3-thread-49) [1efd6270] ActivateStorage Domain. Before Connect all hosts to pool. Time:11/9/12 6:03 AM
>2012-11-09 06:03:37,257 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-3-thread-50) [451d8b1d] START, ValidateStorageServerConnectionVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 039341b1-a8c0-4003-845a-4ef53f74c7c8, connection: localhost:/mnt/data };]), log id: 302353f2
>2012-11-09 06:03:37,277 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-3-thread-50) [451d8b1d] FINISH, ValidateStorageServerConnectionVDSCommand, return: {039341b1-a8c0-4003-845a-4ef53f74c7c8=0}, log id: 302353f2
>2012-11-09 06:03:37,278 INFO  [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-3-thread-50) [451d8b1d] Running command: ConnectStorageToVdsCommand internal: true. Entities affected :  ID: aaa00000-0000-0000-0000-123456789aaa Type: System
>2012-11-09 06:03:37,280 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-3-thread-50) [451d8b1d] START, ConnectStorageServerVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 039341b1-a8c0-4003-845a-4ef53f74c7c8, connection: localhost:/mnt/data };]), log id: 85ccf39
>2012-11-09 06:03:37,310 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-3-thread-50) [451d8b1d] FINISH, ConnectStorageServerVDSCommand, return: {039341b1-a8c0-4003-845a-4ef53f74c7c8=0}, log id: 85ccf39
>2012-11-09 06:03:37,311 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (pool-3-thread-49) [1efd6270] ActivateStorage Domain. After Connect all hosts to pool. Time:11/9/12 6:03 AM
>2012-11-09 06:03:37,312 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (pool-3-thread-49) [1efd6270] START, ActivateStorageDomainVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, storageDomainId = b1658676-2dc3-4357-b112-3852a056c663), log id: 1908ed7a
>2012-11-09 06:03:37,336 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-3-thread-49) [1efd6270] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:03:37,358 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-3-thread-49) [1efd6270] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 60ad902
>2012-11-09 06:03:38,231 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (pool-3-thread-49) [1efd6270] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:03:38,232 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (pool-3-thread-49) [1efd6270] Vds: Ovirt
>2012-11-09 06:03:38,233 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (pool-3-thread-49) [1efd6270] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:03:38,234 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-3-thread-49) [1efd6270] FINISH, ConnectStoragePoolVDSCommand, log id: 60ad902
>2012-11-09 06:03:38,234 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-3-thread-49) [1efd6270] IrsBroker::Failed::ActivateStorageDomainVDS
>2012-11-09 06:03:38,235 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-3-thread-49) [1efd6270] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:03:38,242 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (pool-3-thread-49) [66df56b] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:03:38,291 WARN  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (pool-3-thread-49) [66df56b] CanDoAction of action ReconstructMasterDomain failed. Reasons:VAR__ACTION__RECONSTRUCT_MASTER,VAR__TYPE__STORAGE__DOMAIN,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL2,$status Locked
>2012-11-09 06:03:38,310 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (pool-3-thread-49) [66df56b] FINISH, ActivateStorageDomainVDSCommand, log id: 1908ed7a
>2012-11-09 06:03:38,311 ERROR [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (pool-3-thread-49) [66df56b] Command org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.irsbroker.IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:03:38,318 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (pool-3-thread-49) [66df56b] Command [id=e63e7ebd-81c8-4cfb-8f9d-9940b68068b8]: Compensating CHANGED_STATUS_ONLY of org.ovirt.engine.core.common.businessentities.storage_pool_iso_map; snapshot: EntityStatusSnapshot [id=storagePoolId = c02503a4-274e-11e2-b33c-002590730631, storageId = b1658676-2dc3-4357-b112-3852a056c663, status=InActive].
>2012-11-09 06:03:40,527 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-15) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:03:46,091 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-21) [169e632] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:03:46,116 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-21) [169e632] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 52738a9f
>2012-11-09 06:03:46,939 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-21) [169e632] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:03:46,941 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-21) [169e632] Vds: Ovirt
>2012-11-09 06:03:46,941 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-21) [169e632] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:03:46,943 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-21) [169e632] FINISH, ConnectStoragePoolVDSCommand, log id: 52738a9f
>2012-11-09 06:03:46,943 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-21) [169e632] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:03:46,944 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-21) [169e632] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:03:46,951 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-21) [44ad2b88] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:03:47,019 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-21) [44ad2b88] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:03:47,062 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-21) [44ad2b88] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 32f0a4aa
>2012-11-09 06:03:47,063 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-21) [44ad2b88] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:03:47,064 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-21) [44ad2b88] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 32f0a4aa
>2012-11-09 06:03:50,709 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-27) [1e482fc7] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:03:57,145 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-35) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:03:57,166 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-35) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: d301799
>2012-11-09 06:03:57,982 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-35) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:03:57,984 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-35) Vds: Ovirt
>2012-11-09 06:03:57,984 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-35) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:03:57,985 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-35) FINISH, ConnectStoragePoolVDSCommand, log id: d301799
>2012-11-09 06:03:57,986 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-35) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:03:57,986 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-35) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:03:57,994 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-35) [410047da] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:03:58,059 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-35) [410047da] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:03:58,113 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-35) [410047da] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 5450e42c
>2012-11-09 06:03:58,113 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-35) [410047da] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:03:58,114 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-35) [410047da] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 5450e42c
>2012-11-09 06:04:00,892 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-44) [2abc5f3b] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:04:08,216 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-57) [25d14add] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:04:08,250 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-57) [25d14add] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 9d4282c
>2012-11-09 06:04:09,061 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-57) [25d14add] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:04:09,063 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-57) [25d14add] Vds: Ovirt
>2012-11-09 06:04:09,063 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-57) [25d14add] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:04:09,064 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-57) [25d14add] FINISH, ConnectStoragePoolVDSCommand, log id: 9d4282c
>2012-11-09 06:04:09,065 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-57) [25d14add] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:04:09,065 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-57) [25d14add] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:04:09,072 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-57) [297a1f8c] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:04:09,135 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-57) [297a1f8c] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:04:09,180 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-57) [297a1f8c] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 24b36b36
>2012-11-09 06:04:09,181 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-57) [297a1f8c] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:04:09,182 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-57) [297a1f8c] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 24b36b36
>2012-11-09 06:04:11,095 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-67) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:04:19,258 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-78) [4c8fde32] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:04:19,282 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-78) [4c8fde32] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 2fbb8ac9
>2012-11-09 06:04:20,131 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-78) [4c8fde32] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:04:20,132 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-78) [4c8fde32] Vds: Ovirt
>2012-11-09 06:04:20,132 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-78) [4c8fde32] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:04:20,133 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-78) [4c8fde32] FINISH, ConnectStoragePoolVDSCommand, log id: 2fbb8ac9
>2012-11-09 06:04:20,134 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-78) [4c8fde32] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:04:20,134 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-78) [4c8fde32] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:04:20,141 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-78) [61465af5] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:04:20,208 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-78) [61465af5] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:04:20,252 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-78) [61465af5] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 3d649d03
>2012-11-09 06:04:20,253 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-78) [61465af5] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:04:20,254 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-78) [61465af5] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 3d649d03
>2012-11-09 06:04:21,280 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-85) [27d80cd9] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:04:30,345 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-94) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:04:30,380 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-94) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 35cd510e
>2012-11-09 06:04:31,309 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-94) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:04:31,311 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-94) Vds: Ovirt
>2012-11-09 06:04:31,311 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-94) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:04:31,312 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-94) FINISH, ConnectStoragePoolVDSCommand, log id: 35cd510e
>2012-11-09 06:04:31,313 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-94) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:04:31,313 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-94) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:04:31,320 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-94) [34001dac] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:04:31,407 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-94) [34001dac] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:04:31,462 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-94) [34001dac] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 3472c41c
>2012-11-09 06:04:31,463 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-94) [34001dac] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:04:31,464 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-94) [34001dac] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 3472c41c
>2012-11-09 06:04:31,492 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-99) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:04:41,531 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-16) [1b7f8ea7] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:04:41,557 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-16) [1b7f8ea7] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 6a2b0af1
>2012-11-09 06:04:42,373 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-16) [1b7f8ea7] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:04:42,375 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-16) [1b7f8ea7] Vds: Ovirt
>2012-11-09 06:04:42,376 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-16) [1b7f8ea7] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:04:42,377 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-16) [1b7f8ea7] FINISH, ConnectStoragePoolVDSCommand, log id: 6a2b0af1
>2012-11-09 06:04:42,377 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-16) [1b7f8ea7] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:04:42,378 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-16) [1b7f8ea7] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:04:42,385 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-16) [66a59e71] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:04:42,459 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-16) [66a59e71] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:04:42,502 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-16) [66a59e71] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 7b581be5
>2012-11-09 06:04:42,503 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-16) [66a59e71] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:04:42,504 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-16) [66a59e71] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 7b581be5
>2012-11-09 06:04:52,193 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-34) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:04:52,579 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-32) [154e3e6e] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:04:52,596 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-32) [154e3e6e] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 315165f9
>2012-11-09 06:04:53,410 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-32) [154e3e6e] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:04:53,412 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-32) [154e3e6e] Vds: Ovirt
>2012-11-09 06:04:53,412 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-32) [154e3e6e] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:04:53,413 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-32) [154e3e6e] FINISH, ConnectStoragePoolVDSCommand, log id: 315165f9
>2012-11-09 06:04:53,414 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-32) [154e3e6e] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:04:53,414 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-32) [154e3e6e] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:04:53,421 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-32) [7178c267] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:04:53,490 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-32) [7178c267] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:04:53,532 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-32) [7178c267] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 2a6650e9
>2012-11-09 06:04:53,533 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-32) [7178c267] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:04:53,534 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-32) [7178c267] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 2a6650e9
>2012-11-09 06:05:00,000 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-44) [2abc5f3b] Checking autorecoverable hosts
>2012-11-09 06:05:00,004 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-44) [2abc5f3b] Autorecovering 0 hosts
>2012-11-09 06:05:00,005 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-44) [2abc5f3b] Checking autorecoverable hosts done
>2012-11-09 06:05:00,005 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-44) [2abc5f3b] Checking autorecoverable storage domains
>2012-11-09 06:05:00,025 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-44) [2abc5f3b] Autorecovering 1 storage domains
>2012-11-09 06:05:00,025 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-44) [2abc5f3b] Autorecovering storage domains: b1658676-2dc3-4357-b112-3852a056c663
>2012-11-09 06:05:00,026 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-44) [3e2a49eb] Lock Acquired to object EngineLock [exclusiveLocks= key: b1658676-2dc3-4357-b112-3852a056c663 value: STORAGE
>, sharedLocks= ]
>2012-11-09 06:05:00,046 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-44) [3e2a49eb] Running command: ActivateStorageDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:05:00,067 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-44) [3e2a49eb] Lock freed to object EngineLock [exclusiveLocks= key: b1658676-2dc3-4357-b112-3852a056c663 value: STORAGE
>, sharedLocks= ]
>2012-11-09 06:05:00,067 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-44) [3e2a49eb] ActivateStorage Domain. Before Connect all hosts to pool. Time:11/9/12 6:05 AM
>2012-11-09 06:05:00,083 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-3-thread-49) [47d4153c] START, ValidateStorageServerConnectionVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 039341b1-a8c0-4003-845a-4ef53f74c7c8, connection: localhost:/mnt/data };]), log id: 7d3c19d5
>2012-11-09 06:05:00,105 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-3-thread-49) [47d4153c] FINISH, ValidateStorageServerConnectionVDSCommand, return: {039341b1-a8c0-4003-845a-4ef53f74c7c8=0}, log id: 7d3c19d5
>2012-11-09 06:05:00,106 INFO  [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-3-thread-49) [47d4153c] Running command: ConnectStorageToVdsCommand internal: true. Entities affected :  ID: aaa00000-0000-0000-0000-123456789aaa Type: System
>2012-11-09 06:05:00,109 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-3-thread-49) [47d4153c] START, ConnectStorageServerVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 039341b1-a8c0-4003-845a-4ef53f74c7c8, connection: localhost:/mnt/data };]), log id: 6173309
>2012-11-09 06:05:00,140 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-3-thread-49) [47d4153c] FINISH, ConnectStorageServerVDSCommand, return: {039341b1-a8c0-4003-845a-4ef53f74c7c8=0}, log id: 6173309
>2012-11-09 06:05:00,141 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-44) [3e2a49eb] ActivateStorage Domain. After Connect all hosts to pool. Time:11/9/12 6:05 AM
>2012-11-09 06:05:00,142 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (QuartzScheduler_Worker-44) [3e2a49eb] START, ActivateStorageDomainVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, storageDomainId = b1658676-2dc3-4357-b112-3852a056c663), log id: 375867b0
>2012-11-09 06:05:00,166 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-44) [3e2a49eb] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:05:00,180 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-44) [3e2a49eb] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 34ac9e96
>2012-11-09 06:05:01,042 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-44) [3e2a49eb] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:05:01,043 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-44) [3e2a49eb] Vds: Ovirt
>2012-11-09 06:05:01,044 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-44) [3e2a49eb] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:05:01,045 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-44) [3e2a49eb] FINISH, ConnectStoragePoolVDSCommand, log id: 34ac9e96
>2012-11-09 06:05:01,045 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-44) [3e2a49eb] IrsBroker::Failed::ActivateStorageDomainVDS
>2012-11-09 06:05:01,046 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-44) [3e2a49eb] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:05:01,053 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-44) [541bd727] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:05:01,101 WARN  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-44) [541bd727] CanDoAction of action ReconstructMasterDomain failed. Reasons:VAR__ACTION__RECONSTRUCT_MASTER,VAR__TYPE__STORAGE__DOMAIN,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL2,$status Locked
>2012-11-09 06:05:01,119 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (QuartzScheduler_Worker-44) [541bd727] FINISH, ActivateStorageDomainVDSCommand, log id: 375867b0
>2012-11-09 06:05:01,120 ERROR [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-44) [541bd727] Command org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.irsbroker.IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:05:01,125 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-44) [541bd727] Command [id=0e9227d7-516b-4fe5-8d23-8ea2548ad769]: Compensating CHANGED_STATUS_ONLY of org.ovirt.engine.core.common.businessentities.storage_pool_iso_map; snapshot: EntityStatusSnapshot [id=storagePoolId = c02503a4-274e-11e2-b33c-002590730631, storageId = b1658676-2dc3-4357-b112-3852a056c663, status=InActive].
>2012-11-09 06:05:01,143 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-44) [541bd727] Checking autorecoverable storage domains done
>2012-11-09 06:05:02,381 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-54) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:05:03,614 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-56) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:05:03,641 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-56) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 361ad1a1
>2012-11-09 06:05:04,452 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-56) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:05:04,454 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-56) Vds: Ovirt
>2012-11-09 06:05:04,454 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-56) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:05:04,455 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-56) FINISH, ConnectStoragePoolVDSCommand, log id: 361ad1a1
>2012-11-09 06:05:04,456 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-56) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:05:04,456 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-56) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:05:04,463 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-56) [684da669] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:05:04,534 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-56) [684da669] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:05:04,580 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-56) [684da669] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 5c393fd3
>2012-11-09 06:05:04,581 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-56) [684da669] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:05:04,581 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-56) [684da669] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 5c393fd3
>2012-11-09 06:05:12,563 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-74) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:05:14,657 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-72) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:05:14,697 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-72) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 282ce970
>2012-11-09 06:05:15,506 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-72) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:05:15,508 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-72) Vds: Ovirt
>2012-11-09 06:05:15,509 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-72) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:05:15,510 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-72) FINISH, ConnectStoragePoolVDSCommand, log id: 282ce970
>2012-11-09 06:05:15,510 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-72) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:05:15,511 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-72) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:05:15,517 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-72) [4381d444] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:05:15,590 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-72) [4381d444] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:05:15,633 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-72) [4381d444] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 744d497e
>2012-11-09 06:05:15,634 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-72) [4381d444] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:05:15,634 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-72) [4381d444] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 744d497e
>2012-11-09 06:05:22,723 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-89) [595cb87d] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:05:25,714 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-93) [2cf94af7] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:05:25,732 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-93) [2cf94af7] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 41fa7ee8
>2012-11-09 06:05:26,542 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-93) [2cf94af7] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:05:26,543 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-93) [2cf94af7] Vds: Ovirt
>2012-11-09 06:05:26,544 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-93) [2cf94af7] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:05:26,545 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-93) [2cf94af7] FINISH, ConnectStoragePoolVDSCommand, log id: 41fa7ee8
>2012-11-09 06:05:26,545 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-93) [2cf94af7] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:05:26,546 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-93) [2cf94af7] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:05:26,553 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-93) [b625f1d] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:05:26,569 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-98) starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 06:05:26,572 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-98) domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 06:05:26,577 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-98) [5c9bb2b3] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:05:26,630 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-93) [b625f1d] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:05:26,680 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-98) [5c9bb2b3] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 06:05:26,700 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-93) [b625f1d] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 51218880
>2012-11-09 06:05:26,725 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-98) [5c9bb2b3] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 06:05:26,729 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-98) [5c9bb2b3] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 06:05:26,730 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-98) [5c9bb2b3] No string for UNASSIGNED type. Use default Log
>2012-11-09 06:05:26,746 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-93) [b625f1d] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:05:26,746 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-93) [b625f1d] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 51218880
>2012-11-09 06:05:32,960 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-10) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:05:36,782 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-11) [2de976ac] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:05:36,821 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-11) [2de976ac] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 592c9670
>2012-11-09 06:05:37,656 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-11) [2de976ac] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:05:37,658 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-11) [2de976ac] Vds: Ovirt
>2012-11-09 06:05:37,658 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-11) [2de976ac] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:05:37,659 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-11) [2de976ac] FINISH, ConnectStoragePoolVDSCommand, log id: 592c9670
>2012-11-09 06:05:37,660 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-11) [2de976ac] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:05:37,660 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-11) [2de976ac] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:05:37,670 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-11) [55182a8c] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:05:37,740 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-11) [55182a8c] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:05:37,786 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-11) [55182a8c] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 40c44b0a
>2012-11-09 06:05:37,787 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-11) [55182a8c] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:05:37,787 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-11) [55182a8c] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 40c44b0a
>2012-11-09 06:05:43,158 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-27) [1e482fc7] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:05:47,856 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-34) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:05:47,881 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-34) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 6eb724a1
>2012-11-09 06:05:48,708 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-34) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:05:48,710 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-34) Vds: Ovirt
>2012-11-09 06:05:48,710 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-34) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:05:48,711 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-34) FINISH, ConnectStoragePoolVDSCommand, log id: 6eb724a1
>2012-11-09 06:05:48,711 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-34) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:05:48,712 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-34) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:05:48,719 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-34) [6f2e29a6] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:05:48,791 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-34) [6f2e29a6] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:05:48,833 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-34) [6f2e29a6] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 3fe8c967
>2012-11-09 06:05:48,834 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-34) [6f2e29a6] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:05:48,835 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-34) [6f2e29a6] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 3fe8c967
>2012-11-09 06:05:53,338 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-47) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:05:58,924 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-53) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:05:58,951 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-53) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 704568cf
>2012-11-09 06:05:59,798 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-53) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:05:59,799 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-53) Vds: Ovirt
>2012-11-09 06:05:59,800 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-53) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:05:59,801 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-53) FINISH, ConnectStoragePoolVDSCommand, log id: 704568cf
>2012-11-09 06:05:59,801 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-53) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:05:59,801 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-53) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:05:59,808 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-53) [2c08e011] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:05:59,878 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-53) [2c08e011] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:05:59,921 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-53) [2c08e011] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: ea62959
>2012-11-09 06:05:59,922 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-53) [2c08e011] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:05:59,923 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-53) [2c08e011] FINISH, MarkPoolInReconstructModeVDSCommand, log id: ea62959
>2012-11-09 06:06:03,526 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-63) [178cc463] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:06:10,003 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-73) [5e80c1d3] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:06:10,024 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-73) [5e80c1d3] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 293f25fc
>2012-11-09 06:06:10,839 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-73) [5e80c1d3] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:06:10,841 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-73) [5e80c1d3] Vds: Ovirt
>2012-11-09 06:06:10,841 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-73) [5e80c1d3] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:06:10,842 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-73) [5e80c1d3] FINISH, ConnectStoragePoolVDSCommand, log id: 293f25fc
>2012-11-09 06:06:10,843 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-73) [5e80c1d3] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:06:10,843 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-73) [5e80c1d3] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:06:10,850 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-73) [7c3694e6] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:06:10,917 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-73) [7c3694e6] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:06:10,961 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-73) [7c3694e6] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 56cd2ce7
>2012-11-09 06:06:10,962 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-73) [7c3694e6] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:06:10,963 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-73) [7c3694e6] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 56cd2ce7
>2012-11-09 06:06:13,709 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-85) [27d80cd9] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:06:17,563 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-87) starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 06:06:17,579 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-87) domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 06:06:17,583 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-87) [223b58cb] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:06:17,648 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-87) [223b58cb] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 06:06:17,669 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-87) [223b58cb] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 06:06:17,672 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-87) [223b58cb] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 06:06:17,672 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-87) [223b58cb] No string for UNASSIGNED type. Use default Log
>2012-11-09 06:06:21,054 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-92) [4178a315] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:06:21,097 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-92) [4178a315] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 609fb4dd
>2012-11-09 06:06:21,908 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-92) [4178a315] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:06:21,910 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-92) [4178a315] Vds: Ovirt
>2012-11-09 06:06:21,910 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-92) [4178a315] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:06:21,911 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-92) [4178a315] FINISH, ConnectStoragePoolVDSCommand, log id: 609fb4dd
>2012-11-09 06:06:21,912 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-92) [4178a315] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:06:21,912 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-92) [4178a315] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:06:21,919 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-92) [1d18b37] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:06:21,985 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-92) [1d18b37] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:06:22,030 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-92) [1d18b37] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 616d437
>2012-11-09 06:06:22,031 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-92) [1d18b37] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:06:22,032 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-92) [1d18b37] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 616d437
>2012-11-09 06:06:23,880 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-96) [72946543] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:06:27,762 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-6) [2e1c60f4] starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 06:06:27,766 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-6) [2e1c60f4] domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 06:06:27,771 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-6) [1ed33020] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:06:27,838 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-6) [1ed33020] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 06:06:27,860 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-6) [1ed33020] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 06:06:27,862 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-6) [1ed33020] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 06:06:27,863 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-6) [1ed33020] No string for UNASSIGNED type. Use default Log
>2012-11-09 06:06:32,123 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-5) [8079edb] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:06:32,143 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-5) [8079edb] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 62486ebc
>2012-11-09 06:06:32,989 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-5) [8079edb] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:06:32,990 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-5) [8079edb] Vds: Ovirt
>2012-11-09 06:06:32,990 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-5) [8079edb] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:06:32,991 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-5) [8079edb] FINISH, ConnectStoragePoolVDSCommand, log id: 62486ebc
>2012-11-09 06:06:32,992 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-5) [8079edb] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:06:32,992 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-5) [8079edb] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:06:32,999 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-5) [5876d2b3] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:06:33,083 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-5) [5876d2b3] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:06:33,124 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-5) [5876d2b3] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 54bd13e5
>2012-11-09 06:06:33,125 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-5) [5876d2b3] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:06:33,125 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-5) [5876d2b3] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 54bd13e5
>2012-11-09 06:06:34,060 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-11) [55182a8c] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:06:37,952 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-19) [40e0d013] starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 06:06:37,956 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-19) [40e0d013] domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 06:06:37,961 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-19) [451bc9b5] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:06:38,044 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-19) [451bc9b5] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 06:06:38,066 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-19) [451bc9b5] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 06:06:38,069 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-19) [451bc9b5] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 06:06:38,069 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-19) [451bc9b5] No string for UNASSIGNED type. Use default Log
>2012-11-09 06:06:43,198 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-32) [7178c267] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:06:43,218 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-32) [7178c267] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 4cee8dc0
>2012-11-09 06:06:44,056 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-32) [7178c267] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:06:44,058 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-32) [7178c267] Vds: Ovirt
>2012-11-09 06:06:44,058 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-32) [7178c267] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:06:44,059 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-32) [7178c267] FINISH, ConnectStoragePoolVDSCommand, log id: 4cee8dc0
>2012-11-09 06:06:44,060 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-32) [7178c267] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:06:44,060 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-32) [7178c267] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:06:44,067 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-32) [38a65488] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:06:44,136 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-32) [38a65488] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:06:44,176 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-32) [38a65488] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 27ef3264
>2012-11-09 06:06:44,177 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-32) [38a65488] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:06:44,177 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-32) [38a65488] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 27ef3264
>2012-11-09 06:06:44,279 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-37) [7774543f] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:06:48,154 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-43) [32075a2e] starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 06:06:48,158 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-43) [32075a2e] domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 06:06:48,163 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-43) [2926073a] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:06:48,257 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-43) [2926073a] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 06:06:48,277 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-43) [2926073a] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 06:06:48,280 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-43) [2926073a] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 06:06:48,281 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-43) [2926073a] No string for UNASSIGNED type. Use default Log
>2012-11-09 06:06:54,257 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-44) [541bd727] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:06:54,278 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-44) [541bd727] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 514b19b4
>2012-11-09 06:06:54,761 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-54) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:06:55,152 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-44) [541bd727] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:06:55,154 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-44) [541bd727] Vds: Ovirt
>2012-11-09 06:06:55,155 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-44) [541bd727] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:06:55,156 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-44) [541bd727] FINISH, ConnectStoragePoolVDSCommand, log id: 514b19b4
>2012-11-09 06:06:55,157 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-44) [541bd727] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:06:55,157 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-44) [541bd727] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:06:55,164 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-44) [1bd08cb7] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:06:55,225 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-44) [1bd08cb7] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:06:55,270 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-44) [1bd08cb7] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 11311369
>2012-11-09 06:06:55,271 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-44) [1bd08cb7] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:06:55,272 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-44) [1bd08cb7] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 11311369
>2012-11-09 06:07:04,947 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-65) [54340bfe] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:07:05,371 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-75) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:07:05,388 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-75) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 10a5d51d
>2012-11-09 06:07:06,190 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-75) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:07:06,192 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-75) Vds: Ovirt
>2012-11-09 06:07:06,192 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-75) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:07:06,193 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-75) FINISH, ConnectStoragePoolVDSCommand, log id: 10a5d51d
>2012-11-09 06:07:06,194 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-75) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:07:06,194 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-75) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:07:06,201 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-75) [607b6583] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:07:06,290 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-75) [607b6583] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:07:06,323 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-75) [607b6583] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 3a72ed9e
>2012-11-09 06:07:06,324 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-75) [607b6583] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:07:06,324 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-75) [607b6583] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 3a72ed9e
>2012-11-09 06:07:15,146 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-89) [595cb87d] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:07:16,417 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-98) [5c9bb2b3] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:07:16,439 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-98) [5c9bb2b3] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 4b3755bc
>2012-11-09 06:07:17,251 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-98) [5c9bb2b3] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:07:17,253 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-98) [5c9bb2b3] Vds: Ovirt
>2012-11-09 06:07:17,253 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-98) [5c9bb2b3] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:07:17,254 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-98) [5c9bb2b3] FINISH, ConnectStoragePoolVDSCommand, log id: 4b3755bc
>2012-11-09 06:07:17,255 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-98) [5c9bb2b3] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:07:17,255 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-98) [5c9bb2b3] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:07:17,262 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-98) [64eac4aa] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:07:17,337 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-98) [64eac4aa] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:07:17,381 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-98) [64eac4aa] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 30c813bd
>2012-11-09 06:07:17,382 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-98) [64eac4aa] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:07:17,383 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-98) [64eac4aa] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 30c813bd
>2012-11-09 06:07:25,365 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-7) [b37f953] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:07:27,468 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-5) [5876d2b3] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:07:27,565 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-5) [5876d2b3] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 18acff88
>2012-11-09 06:07:28,360 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-5) [5876d2b3] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:07:28,362 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-5) [5876d2b3] Vds: Ovirt
>2012-11-09 06:07:28,362 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-5) [5876d2b3] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:07:28,363 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-5) [5876d2b3] FINISH, ConnectStoragePoolVDSCommand, log id: 18acff88
>2012-11-09 06:07:28,363 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-5) [5876d2b3] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:07:28,364 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-5) [5876d2b3] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:07:28,371 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-5) [73c71ec7] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:07:28,456 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-5) [73c71ec7] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:07:28,500 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-5) [73c71ec7] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 48b7976c
>2012-11-09 06:07:28,501 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-5) [73c71ec7] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:07:28,502 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-5) [73c71ec7] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 48b7976c
>2012-11-09 06:07:35,574 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-27) [1e482fc7] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:07:38,589 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-33) [5b989670] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:07:38,607 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-33) [5b989670] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 5a5b360a
>2012-11-09 06:07:39,377 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-32) [38a65488] starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 06:07:39,380 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-32) [38a65488] domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 06:07:39,385 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-32) [2361575e] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:07:39,420 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-33) [5b989670] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:07:39,421 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-33) [5b989670] Vds: Ovirt
>2012-11-09 06:07:39,422 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-33) [5b989670] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:07:39,423 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-33) [5b989670] FINISH, ConnectStoragePoolVDSCommand, log id: 5a5b360a
>2012-11-09 06:07:39,423 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-33) [5b989670] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:07:39,424 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-33) [5b989670] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:07:39,431 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-33) [778cf5bb] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:07:39,467 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-32) [2361575e] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 06:07:39,492 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-32) [2361575e] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 06:07:39,495 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-32) [2361575e] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 06:07:39,495 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-32) [2361575e] No string for UNASSIGNED type. Use default Log
>2012-11-09 06:07:39,529 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-33) [778cf5bb] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:07:39,575 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-33) [778cf5bb] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 6c1428ad
>2012-11-09 06:07:39,576 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-33) [778cf5bb] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:07:39,576 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-33) [778cf5bb] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 6c1428ad
>2012-11-09 06:07:45,754 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-42) [2e24deb8] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:07:49,546 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-53) [2c08e011] starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 06:07:49,550 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-53) [2c08e011] domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 06:07:49,554 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-53) [62a33474] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:07:49,619 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-53) [62a33474] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 06:07:49,641 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-53) [62a33474] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 06:07:49,644 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-53) [62a33474] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 06:07:49,644 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-53) [62a33474] No string for UNASSIGNED type. Use default Log
>2012-11-09 06:07:49,665 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-54) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:07:49,683 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-54) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 75b3ccad
>2012-11-09 06:07:50,555 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-54) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:07:50,557 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-54) Vds: Ovirt
>2012-11-09 06:07:50,557 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-54) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:07:50,558 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-54) FINISH, ConnectStoragePoolVDSCommand, log id: 75b3ccad
>2012-11-09 06:07:50,559 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-54) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:07:50,559 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-54) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:07:50,566 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-54) [162be1fd] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:07:50,626 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-54) [162be1fd] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:07:50,671 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-54) [162be1fd] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 43bc55a0
>2012-11-09 06:07:50,672 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-54) [162be1fd] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:07:50,672 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-54) [162be1fd] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 43bc55a0
>2012-11-09 06:07:55,932 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-59) [35c61356] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:07:59,741 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-74) starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 06:07:59,744 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-74) domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 06:07:59,749 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-74) [4972d75] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:07:59,814 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-74) [4972d75] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 06:07:59,834 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-74) [4972d75] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 06:07:59,836 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-74) [4972d75] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 06:07:59,837 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-74) [4972d75] No string for UNASSIGNED type. Use default Log
>2012-11-09 06:08:00,751 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-72) [4381d444] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:08:00,785 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-72) [4381d444] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 13e05fd
>2012-11-09 06:08:01,598 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-72) [4381d444] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:08:01,600 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-72) [4381d444] Vds: Ovirt
>2012-11-09 06:08:01,601 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-72) [4381d444] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:08:01,602 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-72) [4381d444] FINISH, ConnectStoragePoolVDSCommand, log id: 13e05fd
>2012-11-09 06:08:01,603 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-72) [4381d444] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:08:01,603 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-72) [4381d444] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:08:01,611 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-72) [7fa7598e] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:08:01,678 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-72) [7fa7598e] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:08:01,722 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-72) [7fa7598e] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 47f1aaec
>2012-11-09 06:08:01,723 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-72) [7fa7598e] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:08:01,723 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-72) [7fa7598e] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 47f1aaec
>2012-11-09 06:08:06,113 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-81) [26a17f04] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:08:09,945 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-90) starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 06:08:09,948 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-90) domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 06:08:09,953 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-90) [286004d] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:08:10,020 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-90) [286004d] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 06:08:10,041 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-90) [286004d] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 06:08:10,044 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-90) [286004d] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 06:08:10,045 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-90) [286004d] No string for UNASSIGNED type. Use default Log
>2012-11-09 06:08:11,810 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-93) [b625f1d] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:08:11,833 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-93) [b625f1d] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 5cca0800
>2012-11-09 06:08:12,633 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-93) [b625f1d] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:08:12,634 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-93) [b625f1d] Vds: Ovirt
>2012-11-09 06:08:12,635 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-93) [b625f1d] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:08:12,636 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-93) [b625f1d] FINISH, ConnectStoragePoolVDSCommand, log id: 5cca0800
>2012-11-09 06:08:12,636 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-93) [b625f1d] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:08:12,637 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-93) [b625f1d] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:08:12,643 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-93) [503538e1] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:08:12,705 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-93) [503538e1] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:08:12,750 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-93) [503538e1] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 42128019
>2012-11-09 06:08:12,751 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-93) [503538e1] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:08:12,752 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-93) [503538e1] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 42128019
>2012-11-09 06:08:16,316 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-2) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:08:22,838 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-9) [15914def] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:08:22,861 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-9) [15914def] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 2f94d4ec
>2012-11-09 06:08:23,682 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-9) [15914def] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:08:23,683 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-9) [15914def] Vds: Ovirt
>2012-11-09 06:08:23,683 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-9) [15914def] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:08:23,684 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-9) [15914def] FINISH, ConnectStoragePoolVDSCommand, log id: 2f94d4ec
>2012-11-09 06:08:23,685 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-9) [15914def] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:08:23,685 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-9) [15914def] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:08:23,692 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-9) [66879a04] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:08:23,753 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-9) [66879a04] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:08:23,794 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-9) [66879a04] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 77217463
>2012-11-09 06:08:23,795 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-9) [66879a04] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:08:23,795 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-9) [66879a04] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 77217463
>2012-11-09 06:08:26,522 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-11) [55182a8c] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:08:30,343 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-24) [5a56435a] starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 06:08:30,346 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-24) [5a56435a] domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 06:08:30,351 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-24) [1a8915ff] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:08:30,439 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-24) [1a8915ff] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 06:08:30,460 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-24) [1a8915ff] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 06:08:30,463 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-24) [1a8915ff] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 06:08:30,463 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-24) [1a8915ff] No string for UNASSIGNED type. Use default Log
>2012-11-09 06:08:33,887 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-34) [6f2e29a6] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:08:33,911 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-34) [6f2e29a6] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 724daec4
>2012-11-09 06:08:34,729 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-34) [6f2e29a6] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:08:34,731 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-34) [6f2e29a6] Vds: Ovirt
>2012-11-09 06:08:34,731 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-34) [6f2e29a6] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:08:34,732 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-34) [6f2e29a6] FINISH, ConnectStoragePoolVDSCommand, log id: 724daec4
>2012-11-09 06:08:34,733 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-34) [6f2e29a6] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:08:34,733 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-34) [6f2e29a6] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:08:34,740 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-34) [533b4d72] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:08:34,820 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-34) [533b4d72] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:08:34,862 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-34) [533b4d72] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 6112372d
>2012-11-09 06:08:34,863 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-34) [533b4d72] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:08:34,864 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-34) [533b4d72] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 6112372d
>2012-11-09 06:08:36,746 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-43) [2926073a] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:08:40,527 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-39) [55d3e877] starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 06:08:40,531 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-39) [55d3e877] domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 06:08:40,535 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-39) [7e0aba3c] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:08:40,606 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-39) [7e0aba3c] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 06:08:40,626 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-39) [7e0aba3c] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 06:08:40,629 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-39) [7e0aba3c] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 06:08:40,630 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-39) [7e0aba3c] No string for UNASSIGNED type. Use default Log
>2012-11-09 06:08:44,955 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-53) [62a33474] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:08:44,993 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-53) [62a33474] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 2410d91
>2012-11-09 06:08:45,813 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-53) [62a33474] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:08:45,815 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-53) [62a33474] Vds: Ovirt
>2012-11-09 06:08:45,815 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-53) [62a33474] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:08:45,816 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-53) [62a33474] FINISH, ConnectStoragePoolVDSCommand, log id: 2410d91
>2012-11-09 06:08:45,817 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-53) [62a33474] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:08:45,817 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-53) [62a33474] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:08:45,824 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-53) [265a0609] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:08:45,897 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-53) [265a0609] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:08:45,941 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-53) [265a0609] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 7aceae2d
>2012-11-09 06:08:45,942 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-53) [265a0609] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:08:45,942 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-53) [265a0609] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 7aceae2d
>2012-11-09 06:08:46,950 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-56) [684da669] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:08:50,710 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-63) [178cc463] starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 06:08:50,714 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-63) [178cc463] domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 06:08:50,718 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-63) [51f00e85] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:08:50,802 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-63) [51f00e85] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 06:08:50,822 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-63) [51f00e85] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 06:08:50,825 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-63) [51f00e85] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 06:08:50,826 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-63) [51f00e85] No string for UNASSIGNED type. Use default Log
>2012-11-09 06:08:56,029 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-75) [607b6583] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:08:56,051 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-75) [607b6583] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 466de7ee
>2012-11-09 06:08:56,891 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-75) [607b6583] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:08:56,893 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-75) [607b6583] Vds: Ovirt
>2012-11-09 06:08:56,893 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-75) [607b6583] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:08:56,894 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-75) [607b6583] FINISH, ConnectStoragePoolVDSCommand, log id: 466de7ee
>2012-11-09 06:08:56,895 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-75) [607b6583] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:08:56,896 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-75) [607b6583] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:08:56,903 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-75) [257ea93] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:08:56,985 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-75) [257ea93] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:08:57,030 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-75) [257ea93] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 794cc474
>2012-11-09 06:08:57,031 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-75) [257ea93] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:08:57,032 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-75) [257ea93] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 794cc474
>2012-11-09 06:08:57,135 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-73) [7c3694e6] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:09:00,894 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-84) starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 06:09:00,897 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-84) domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 06:09:00,901 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-84) [73bfb79b] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:09:00,990 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-84) [73bfb79b] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 06:09:01,010 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-84) [73bfb79b] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 06:09:01,013 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-84) [73bfb79b] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 06:09:01,014 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-84) [73bfb79b] No string for UNASSIGNED type. Use default Log
>2012-11-09 06:09:07,116 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-98) [64eac4aa] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:09:07,142 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-98) [64eac4aa] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 56f7c93d
>2012-11-09 06:09:07,571 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-96) [72946543] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:09:07,960 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-98) [64eac4aa] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:09:07,961 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-98) [64eac4aa] Vds: Ovirt
>2012-11-09 06:09:07,962 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-98) [64eac4aa] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:09:07,963 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-98) [64eac4aa] FINISH, ConnectStoragePoolVDSCommand, log id: 56f7c93d
>2012-11-09 06:09:07,963 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-98) [64eac4aa] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:09:07,964 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-98) [64eac4aa] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:09:07,970 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-98) [4947c0c5] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:09:08,044 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-98) [4947c0c5] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:09:08,089 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-98) [4947c0c5] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 328435ce
>2012-11-09 06:09:08,090 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-98) [4947c0c5] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:09:08,091 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-98) [4947c0c5] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 328435ce
>2012-11-09 06:09:17,758 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-13) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:09:18,165 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-9) [66879a04] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:09:18,177 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-9) [66879a04] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 16f9e052
>2012-11-09 06:09:19,021 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-9) [66879a04] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:09:19,022 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-9) [66879a04] Vds: Ovirt
>2012-11-09 06:09:19,023 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-9) [66879a04] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:09:19,024 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-9) [66879a04] FINISH, ConnectStoragePoolVDSCommand, log id: 16f9e052
>2012-11-09 06:09:19,024 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-9) [66879a04] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:09:19,025 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-9) [66879a04] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:09:19,032 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-9) [4bf1771a] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:09:19,103 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-9) [4bf1771a] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:09:19,144 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-9) [4bf1771a] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 7384f42e
>2012-11-09 06:09:19,145 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-9) [4bf1771a] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:09:19,145 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-9) [4bf1771a] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 7384f42e
>2012-11-09 06:09:27,944 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-36) [566248a0] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:09:29,218 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-32) [2361575e] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:09:29,242 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-32) [2361575e] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 4d14a060
>2012-11-09 06:09:30,046 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-32) [2361575e] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:09:30,048 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-32) [2361575e] Vds: Ovirt
>2012-11-09 06:09:30,048 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-32) [2361575e] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:09:30,049 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-32) [2361575e] FINISH, ConnectStoragePoolVDSCommand, log id: 4d14a060
>2012-11-09 06:09:30,050 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-32) [2361575e] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:09:30,050 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-32) [2361575e] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:09:30,057 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-32) [ca2ec9a] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:09:30,126 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-32) [ca2ec9a] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:09:30,170 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-32) [ca2ec9a] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: fa59e34
>2012-11-09 06:09:30,171 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-32) [ca2ec9a] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:09:30,172 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-32) [ca2ec9a] FINISH, MarkPoolInReconstructModeVDSCommand, log id: fa59e34
>2012-11-09 06:09:38,134 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-55) [7462212d] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:09:40,246 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-53) [265a0609] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:09:40,285 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-53) [265a0609] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: e068d92
>2012-11-09 06:09:41,101 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-53) [265a0609] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:09:41,103 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-53) [265a0609] Vds: Ovirt
>2012-11-09 06:09:41,104 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-53) [265a0609] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:09:41,105 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-53) [265a0609] FINISH, ConnectStoragePoolVDSCommand, log id: e068d92
>2012-11-09 06:09:41,105 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-53) [265a0609] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:09:41,106 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-53) [265a0609] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:09:41,112 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-53) [289159bb] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:09:41,176 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-53) [289159bb] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:09:41,220 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-53) [289159bb] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 4568e159
>2012-11-09 06:09:41,221 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-53) [289159bb] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:09:41,221 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-53) [289159bb] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 4568e159
>2012-11-09 06:09:48,328 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-62) [54520a5b] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:09:51,297 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-65) [54340bfe] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:09:51,321 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-65) [54340bfe] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 5d298bd2
>2012-11-09 06:09:52,141 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-65) [54340bfe] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:09:52,142 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-65) [54340bfe] Vds: Ovirt
>2012-11-09 06:09:52,143 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-65) [54340bfe] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:09:52,144 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-65) [54340bfe] FINISH, ConnectStoragePoolVDSCommand, log id: 5d298bd2
>2012-11-09 06:09:52,144 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-65) [54340bfe] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:09:52,145 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-65) [54340bfe] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:09:52,152 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-65) [52187354] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:09:52,194 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-72) [7fa7598e] starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 06:09:52,196 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-72) [7fa7598e] domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 06:09:52,200 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-72) [14ec0d95] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:09:52,233 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-65) [52187354] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:09:52,294 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-65) [52187354] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 5e4e29ca
>2012-11-09 06:09:52,307 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-72) [14ec0d95] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 06:09:52,327 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-72) [14ec0d95] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 06:09:52,330 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-72) [14ec0d95] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 06:09:52,330 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-72) [14ec0d95] No string for UNASSIGNED type. Use default Log
>2012-11-09 06:09:52,347 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-65) [52187354] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:09:52,348 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-65) [52187354] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 5e4e29ca
>2012-11-09 06:09:58,511 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-88) [21d03423] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:10:00,000 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-89) [595cb87d] Checking autorecoverable hosts
>2012-11-09 06:10:00,004 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-89) [595cb87d] Autorecovering 0 hosts
>2012-11-09 06:10:00,005 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-89) [595cb87d] Checking autorecoverable hosts done
>2012-11-09 06:10:00,005 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-89) [595cb87d] Checking autorecoverable storage domains
>2012-11-09 06:10:00,027 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-89) [595cb87d] Autorecovering 1 storage domains
>2012-11-09 06:10:00,028 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-89) [595cb87d] Autorecovering storage domains: b1658676-2dc3-4357-b112-3852a056c663
>2012-11-09 06:10:00,029 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-89) [5d28fdc] Lock Acquired to object EngineLock [exclusiveLocks= key: b1658676-2dc3-4357-b112-3852a056c663 value: STORAGE
>, sharedLocks= ]
>2012-11-09 06:10:00,049 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-89) [5d28fdc] Running command: ActivateStorageDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:10:00,069 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-89) [5d28fdc] Lock freed to object EngineLock [exclusiveLocks= key: b1658676-2dc3-4357-b112-3852a056c663 value: STORAGE
>, sharedLocks= ]
>2012-11-09 06:10:00,069 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-89) [5d28fdc] ActivateStorage Domain. Before Connect all hosts to pool. Time:11/9/12 6:10 AM
>2012-11-09 06:10:00,085 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-3-thread-49) [30110607] START, ValidateStorageServerConnectionVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 039341b1-a8c0-4003-845a-4ef53f74c7c8, connection: localhost:/mnt/data };]), log id: 13b8bcb1
>2012-11-09 06:10:00,107 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-3-thread-49) [30110607] FINISH, ValidateStorageServerConnectionVDSCommand, return: {039341b1-a8c0-4003-845a-4ef53f74c7c8=0}, log id: 13b8bcb1
>2012-11-09 06:10:00,108 INFO  [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-3-thread-49) [30110607] Running command: ConnectStorageToVdsCommand internal: true. Entities affected :  ID: aaa00000-0000-0000-0000-123456789aaa Type: System
>2012-11-09 06:10:00,109 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-3-thread-49) [30110607] START, ConnectStorageServerVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 039341b1-a8c0-4003-845a-4ef53f74c7c8, connection: localhost:/mnt/data };]), log id: 72ebe769
>2012-11-09 06:10:00,127 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-3-thread-49) [30110607] FINISH, ConnectStorageServerVDSCommand, return: {039341b1-a8c0-4003-845a-4ef53f74c7c8=0}, log id: 72ebe769
>2012-11-09 06:10:00,128 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-89) [5d28fdc] ActivateStorage Domain. After Connect all hosts to pool. Time:11/9/12 6:10 AM
>2012-11-09 06:10:00,128 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (QuartzScheduler_Worker-89) [5d28fdc] START, ActivateStorageDomainVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, storageDomainId = b1658676-2dc3-4357-b112-3852a056c663), log id: 51dc8ae7
>2012-11-09 06:10:00,143 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-89) [5d28fdc] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:10:00,157 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-89) [5d28fdc] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 4106e23d
>2012-11-09 06:10:00,972 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-89) [5d28fdc] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:10:00,973 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-89) [5d28fdc] Vds: Ovirt
>2012-11-09 06:10:00,974 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-89) [5d28fdc] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:10:00,975 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-89) [5d28fdc] FINISH, ConnectStoragePoolVDSCommand, log id: 4106e23d
>2012-11-09 06:10:00,975 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-89) [5d28fdc] IrsBroker::Failed::ActivateStorageDomainVDS
>2012-11-09 06:10:00,976 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-89) [5d28fdc] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:10:00,983 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-89) [4572a9a8] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:10:01,014 WARN  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-89) [4572a9a8] CanDoAction of action ReconstructMasterDomain failed. Reasons:VAR__ACTION__RECONSTRUCT_MASTER,VAR__TYPE__STORAGE__DOMAIN,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL2,$status Locked
>2012-11-09 06:10:01,032 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (QuartzScheduler_Worker-89) [4572a9a8] FINISH, ActivateStorageDomainVDSCommand, log id: 51dc8ae7
>2012-11-09 06:10:01,033 ERROR [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-89) [4572a9a8] Command org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.irsbroker.IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:10:01,039 INFO  [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (QuartzScheduler_Worker-89) [4572a9a8] Command [id=a4e3508f-f8f6-4b54-9201-7983a3ed9e9a]: Compensating CHANGED_STATUS_ONLY of org.ovirt.engine.core.common.businessentities.storage_pool_iso_map; snapshot: EntityStatusSnapshot [id=storagePoolId = c02503a4-274e-11e2-b33c-002590730631, storageId = b1658676-2dc3-4357-b112-3852a056c663, status=InActive].
>2012-11-09 06:10:01,054 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-89) [4572a9a8] Checking autorecoverable storage domains done
>2012-11-09 06:10:02,381 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-96) [72946543] starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 06:10:02,384 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-96) [72946543] domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 06:10:02,389 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-96) [40c581e3] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:10:02,396 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-97) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:10:02,420 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-97) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 1227b22
>2012-11-09 06:10:02,467 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-96) [40c581e3] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 06:10:02,788 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-96) [40c581e3] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 06:10:02,791 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-96) [40c581e3] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 06:10:02,791 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-96) [40c581e3] No string for UNASSIGNED type. Use default Log
>2012-11-09 06:10:03,223 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-97) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:10:03,225 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-97) Vds: Ovirt
>2012-11-09 06:10:03,225 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-97) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:10:03,226 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-97) FINISH, ConnectStoragePoolVDSCommand, log id: 1227b22
>2012-11-09 06:10:03,226 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-97) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:10:03,227 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-97) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:10:03,234 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-97) [56bdb812] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:10:03,302 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-97) [56bdb812] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:10:03,343 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-97) [56bdb812] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 552f67f9
>2012-11-09 06:10:03,344 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-97) [56bdb812] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:10:03,345 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-97) [56bdb812] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 552f67f9
>2012-11-09 06:10:08,697 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-8) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:10:12,563 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-23) starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 06:10:12,567 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-23) domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 06:10:12,572 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-23) [254f5e35] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:10:12,640 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-23) [254f5e35] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 06:10:12,652 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-23) [254f5e35] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 06:10:12,654 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-23) [254f5e35] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 06:10:12,654 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-23) [254f5e35] No string for UNASSIGNED type. Use default Log
>2012-11-09 06:10:13,426 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-9) [4bf1771a] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:10:13,447 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-9) [4bf1771a] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 59e48854
>2012-11-09 06:10:14,275 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-9) [4bf1771a] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:10:14,277 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-9) [4bf1771a] Vds: Ovirt
>2012-11-09 06:10:14,277 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-9) [4bf1771a] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:10:14,278 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-9) [4bf1771a] FINISH, ConnectStoragePoolVDSCommand, log id: 59e48854
>2012-11-09 06:10:14,279 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-9) [4bf1771a] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:10:14,279 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-9) [4bf1771a] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:10:14,285 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-9) [42503c82] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:10:14,361 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-9) [42503c82] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:10:14,403 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-9) [42503c82] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 7305299c
>2012-11-09 06:10:14,404 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-9) [42503c82] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:10:14,404 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-9) [42503c82] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 7305299c
>2012-11-09 06:10:18,889 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-26) [10f41170] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:10:24,484 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-36) [566248a0] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:10:24,503 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-36) [566248a0] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 6bf85634
>2012-11-09 06:10:25,318 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-36) [566248a0] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:10:25,319 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-36) [566248a0] Vds: Ovirt
>2012-11-09 06:10:25,320 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-36) [566248a0] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:10:25,321 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-36) [566248a0] FINISH, ConnectStoragePoolVDSCommand, log id: 6bf85634
>2012-11-09 06:10:25,321 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-36) [566248a0] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:10:25,322 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-36) [566248a0] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:10:25,328 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-36) [6a1b94a3] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:10:25,395 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-36) [6a1b94a3] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:10:25,440 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-36) [6a1b94a3] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 509d869
>2012-11-09 06:10:25,441 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-36) [6a1b94a3] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:10:25,442 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-36) [6a1b94a3] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 509d869
>2012-11-09 06:10:29,066 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-39) [7e0aba3c] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:10:32,960 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-55) [7462212d] starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 06:10:32,964 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-55) [7462212d] domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 06:10:32,969 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-55) [37f9a932] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:10:33,054 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-55) [37f9a932] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 06:10:33,076 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-55) [37f9a932] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 06:10:33,079 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-55) [37f9a932] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 06:10:33,080 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-55) [37f9a932] No string for UNASSIGNED type. Use default Log
>2012-11-09 06:10:35,517 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-56) [684da669] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:10:35,557 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-56) [684da669] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 421dcba6
>2012-11-09 06:10:36,369 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-56) [684da669] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:10:36,370 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-56) [684da669] Vds: Ovirt
>2012-11-09 06:10:36,371 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-56) [684da669] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:10:36,372 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-56) [684da669] FINISH, ConnectStoragePoolVDSCommand, log id: 421dcba6
>2012-11-09 06:10:36,372 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-56) [684da669] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:10:36,373 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-56) [684da669] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:10:36,380 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-56) [31742c8e] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:10:36,501 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-56) [31742c8e] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:10:36,581 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-56) [31742c8e] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 2828ea62
>2012-11-09 06:10:36,582 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-56) [31742c8e] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:10:36,582 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-56) [31742c8e] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 2828ea62
>2012-11-09 06:10:39,247 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-67) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:10:43,159 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-74) [4972d75] starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 06:10:43,163 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-74) [4972d75] domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 06:10:43,168 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-74) [a5799a6] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:10:43,234 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-74) [a5799a6] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 06:10:43,256 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-74) [a5799a6] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 06:10:43,258 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-74) [a5799a6] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 06:10:43,259 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-74) [a5799a6] No string for UNASSIGNED type. Use default Log
>2012-11-09 06:10:46,657 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-68) [79e52acc] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:10:46,679 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-68) [79e52acc] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 1916a5b7
>2012-11-09 06:10:47,498 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-68) [79e52acc] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:10:47,500 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-68) [79e52acc] Vds: Ovirt
>2012-11-09 06:10:47,500 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-68) [79e52acc] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:10:47,501 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-68) [79e52acc] FINISH, ConnectStoragePoolVDSCommand, log id: 1916a5b7
>2012-11-09 06:10:47,502 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-68) [79e52acc] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:10:47,502 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-68) [79e52acc] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:10:47,509 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-68) [4d9d30b9] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:10:47,581 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-68) [4d9d30b9] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:10:47,623 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-68) [4d9d30b9] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 40bef1c
>2012-11-09 06:10:47,624 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-68) [4d9d30b9] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:10:47,624 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-68) [4d9d30b9] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 40bef1c
>2012-11-09 06:10:49,437 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-77) [1ef92680] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:10:53,339 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-88) [21d03423] starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 06:10:53,343 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-88) [21d03423] domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 06:10:53,347 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-88) [75521dac] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:10:53,412 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-88) [75521dac] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 06:10:53,434 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-88) [75521dac] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 06:10:53,437 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-88) [75521dac] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 06:10:53,437 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-88) [75521dac] No string for UNASSIGNED type. Use default Log
>2012-11-09 06:10:57,704 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-94) [34001dac] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:10:57,731 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-94) [34001dac] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 3906e8a3
>2012-11-09 06:10:58,542 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-94) [34001dac] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:10:58,543 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-94) [34001dac] Vds: Ovirt
>2012-11-09 06:10:58,544 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-94) [34001dac] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:10:58,545 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-94) [34001dac] FINISH, ConnectStoragePoolVDSCommand, log id: 3906e8a3
>2012-11-09 06:10:58,545 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-94) [34001dac] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:10:58,546 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-94) [34001dac] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:10:58,553 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-94) [3a9ebc03] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:10:58,626 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-94) [3a9ebc03] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:10:58,670 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-94) [3a9ebc03] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 4cf1055d
>2012-11-09 06:10:58,671 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-94) [3a9ebc03] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:10:58,672 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-94) [3a9ebc03] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 4cf1055d
>2012-11-09 06:10:59,617 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-3) [60b43093] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:11:03,526 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-10) starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 06:11:03,530 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-10) domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 06:11:03,534 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-10) [148d903] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:11:03,597 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-10) [148d903] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 06:11:03,619 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-10) [148d903] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 06:11:03,621 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-10) [148d903] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 06:11:03,622 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-10) [148d903] No string for UNASSIGNED type. Use default Log
>2012-11-09 06:11:08,748 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-5) [73c71ec7] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:11:08,788 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-5) [73c71ec7] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 4895e384
>2012-11-09 06:11:09,631 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-5) [73c71ec7] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:11:09,632 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-5) [73c71ec7] Vds: Ovirt
>2012-11-09 06:11:09,633 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-5) [73c71ec7] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:11:09,634 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-5) [73c71ec7] FINISH, ConnectStoragePoolVDSCommand, log id: 4895e384
>2012-11-09 06:11:09,634 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-5) [73c71ec7] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:11:09,635 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-5) [73c71ec7] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:11:09,641 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-5) [6fa95ce5] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:11:09,705 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-5) [6fa95ce5] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:11:09,750 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-5) [6fa95ce5] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 6a79a593
>2012-11-09 06:11:09,750 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-5) [6fa95ce5] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:11:09,750 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-5) [6fa95ce5] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 6a79a593
>2012-11-09 06:11:09,851 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-21) [44ad2b88] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:11:19,854 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-40) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:11:19,877 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-40) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 16764b06
>2012-11-09 06:11:20,693 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-40) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:11:20,695 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-40) Vds: Ovirt
>2012-11-09 06:11:20,695 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-40) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:11:20,696 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-40) FINISH, ConnectStoragePoolVDSCommand, log id: 16764b06
>2012-11-09 06:11:20,697 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-40) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:11:20,697 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-40) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:11:20,704 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-40) [6de45c34] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:11:20,772 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-40) [6de45c34] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:11:20,813 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-40) [6de45c34] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 3ce2b0d7
>2012-11-09 06:11:20,814 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-40) [6de45c34] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:11:20,815 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-40) [6de45c34] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 3ce2b0d7
>2012-11-09 06:11:30,467 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-44) [1bd08cb7] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:11:30,907 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-57) [297a1f8c] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:11:30,946 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-57) [297a1f8c] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 300206ec
>2012-11-09 06:11:31,798 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-57) [297a1f8c] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:11:31,800 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-57) [297a1f8c] Vds: Ovirt
>2012-11-09 06:11:31,800 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-57) [297a1f8c] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:11:31,801 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-57) [297a1f8c] FINISH, ConnectStoragePoolVDSCommand, log id: 300206ec
>2012-11-09 06:11:31,801 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-57) [297a1f8c] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:11:31,802 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-57) [297a1f8c] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:11:31,809 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-57) [4e6aed1a] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:11:31,878 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-57) [4e6aed1a] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:11:31,921 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-57) [4e6aed1a] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 698de8b9
>2012-11-09 06:11:31,922 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-57) [4e6aed1a] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:11:31,923 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-57) [4e6aed1a] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 698de8b9
>2012-11-09 06:11:40,652 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-73) [7c3694e6] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:11:42,035 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-76) [29cab4a] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:11:42,050 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-76) [29cab4a] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 212de407
>2012-11-09 06:11:42,870 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-76) [29cab4a] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:11:42,872 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-76) [29cab4a] Vds: Ovirt
>2012-11-09 06:11:42,872 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-76) [29cab4a] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:11:42,873 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-76) [29cab4a] FINISH, ConnectStoragePoolVDSCommand, log id: 212de407
>2012-11-09 06:11:42,874 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-76) [29cab4a] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:11:42,874 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-76) [29cab4a] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:11:42,881 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-76) [d6dcb6b] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:11:42,950 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-76) [d6dcb6b] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:11:42,993 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-76) [d6dcb6b] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 4e11bea8
>2012-11-09 06:11:42,994 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-76) [d6dcb6b] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:11:42,994 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-76) [d6dcb6b] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 4e11bea8
>2012-11-09 06:11:50,835 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-86) [1717f3a9] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:11:53,075 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-98) [4947c0c5] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:11:53,111 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-98) [4947c0c5] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 6255031e
>2012-11-09 06:11:53,919 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-98) [4947c0c5] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:11:53,921 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-98) [4947c0c5] Vds: Ovirt
>2012-11-09 06:11:53,921 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-98) [4947c0c5] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:11:53,922 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-98) [4947c0c5] FINISH, ConnectStoragePoolVDSCommand, log id: 6255031e
>2012-11-09 06:11:53,923 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-98) [4947c0c5] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:11:53,923 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-98) [4947c0c5] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:11:53,930 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-98) [2e0c37a1] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:11:54,000 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-98) [2e0c37a1] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:11:54,042 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-98) [2e0c37a1] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 5aaaea14
>2012-11-09 06:11:54,043 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-98) [2e0c37a1] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:11:54,044 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-98) [2e0c37a1] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 5aaaea14
>2012-11-09 06:12:01,027 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-15) domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:12:04,116 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-20) [7a721712] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:12:04,144 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-20) [7a721712] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 1d0fbfa1
>2012-11-09 06:12:04,948 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-9) [42503c82] starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 06:12:04,950 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-9) [42503c82] domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 06:12:04,952 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-9) [2d6ddbdc] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:12:04,962 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-20) [7a721712] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:12:04,963 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-20) [7a721712] Vds: Ovirt
>2012-11-09 06:12:04,963 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-20) [7a721712] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:12:04,964 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-20) [7a721712] FINISH, ConnectStoragePoolVDSCommand, log id: 1d0fbfa1
>2012-11-09 06:12:04,965 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-20) [7a721712] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:12:04,965 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-20) [7a721712] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:12:04,971 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-20) [6ef094a9] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:12:05,079 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-9) [2d6ddbdc] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 06:12:05,104 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-20) [6ef094a9] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:12:05,116 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-9) [2d6ddbdc] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 06:12:05,118 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-9) [2d6ddbdc] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 06:12:05,119 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-9) [2d6ddbdc] No string for UNASSIGNED type. Use default Log
>2012-11-09 06:12:05,177 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-20) [6ef094a9] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 5f449324
>2012-11-09 06:12:05,178 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-20) [6ef094a9] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:12:05,179 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-20) [6ef094a9] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 5f449324
>2012-11-09 06:12:11,474 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-30) [50500ec2] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:12:15,147 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-28) [3969a9ab] starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 06:12:15,151 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-28) [3969a9ab] domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 06:12:15,156 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-28) [232bf9ad] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:12:15,231 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-37) [7774543f] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:12:15,278 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-37) [7774543f] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 73367957
>2012-11-09 06:12:15,325 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-28) [232bf9ad] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 06:12:15,646 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-28) [232bf9ad] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 06:12:15,649 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-28) [232bf9ad] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 06:12:15,650 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-28) [232bf9ad] No string for UNASSIGNED type. Use default Log
>2012-11-09 06:12:16,093 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-37) [7774543f] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:12:16,095 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-37) [7774543f] Vds: Ovirt
>2012-11-09 06:12:16,095 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-37) [7774543f] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:12:16,096 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-37) [7774543f] FINISH, ConnectStoragePoolVDSCommand, log id: 73367957
>2012-11-09 06:12:16,097 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-37) [7774543f] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:12:16,098 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-37) [7774543f] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:12:16,104 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-37) [54615510] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:12:16,165 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-37) [54615510] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:12:16,204 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-37) [54615510] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 7172ce1f
>2012-11-09 06:12:16,205 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-37) [54615510] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:12:16,205 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-37) [54615510] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 7172ce1f
>2012-11-09 06:12:21,737 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-42) [2e24deb8] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:12:25,365 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-49) [2d1d8b93] starting ProcessDomainRecovery for domain d1eaaed8-01da-4188-b54b-77175adebb90
>2012-11-09 06:12:25,368 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-49) [2d1d8b93] domain d1eaaed8-01da-4188-b54b-77175adebb90 was reported by all hosts in status UP as problematic. Moving the Domain to NonOperational.
>2012-11-09 06:12:25,373 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-49) [42cab6a8] The message key HandleFailedStorageDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:12:25,501 INFO  [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-49) [42cab6a8] Running command: HandleFailedStorageDomainCommand internal: true. Entities affected :  ID: d1eaaed8-01da-4188-b54b-77175adebb90 Type: Storage
>2012-11-09 06:12:25,523 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (QuartzScheduler_Worker-49) [42cab6a8] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL
>2012-11-09 06:12:25,525 ERROR [org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand] (QuartzScheduler_Worker-49) [42cab6a8] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.HandleFailedStorageDomainCommand.
>2012-11-09 06:12:25,526 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (QuartzScheduler_Worker-49) [42cab6a8] No string for UNASSIGNED type. Use default Log
>2012-11-09 06:12:26,279 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-44) [1bd08cb7] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:12:26,314 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-44) [1bd08cb7] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 4fc91cf0
>2012-11-09 06:12:27,165 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-44) [1bd08cb7] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:12:27,166 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-44) [1bd08cb7] Vds: Ovirt
>2012-11-09 06:12:27,166 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-44) [1bd08cb7] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:12:27,167 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-44) [1bd08cb7] FINISH, ConnectStoragePoolVDSCommand, log id: 4fc91cf0
>2012-11-09 06:12:27,167 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-44) [1bd08cb7] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:12:27,167 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-44) [1bd08cb7] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:12:27,172 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-44) [142e6ec0] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:12:27,322 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-44) [142e6ec0] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:12:27,450 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-44) [142e6ec0] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 5ead4c2c
>2012-11-09 06:12:27,450 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-44) [142e6ec0] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:12:27,451 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-44) [142e6ec0] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 5ead4c2c
>2012-11-09 06:12:32,008 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-64) [4ae70538] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:12:37,528 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-78) [61465af5] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:12:37,625 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-78) [61465af5] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 1d75880f
>2012-11-09 06:12:38,516 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-78) [61465af5] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:12:38,517 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-78) [61465af5] Vds: Ovirt
>2012-11-09 06:12:38,517 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-78) [61465af5] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:12:38,518 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-78) [61465af5] FINISH, ConnectStoragePoolVDSCommand, log id: 1d75880f
>2012-11-09 06:12:38,518 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-78) [61465af5] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:12:38,518 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-78) [61465af5] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:12:38,524 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-78) [96b56bb] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:12:38,798 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-78) [96b56bb] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:12:38,914 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-78) [96b56bb] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 1f564226
>2012-11-09 06:12:38,915 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-78) [96b56bb] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:12:38,915 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-78) [96b56bb] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 1f564226
>2012-11-09 06:12:42,269 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-81) [26a17f04] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:12:48,986 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-97) [56bdb812] hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:12:49,067 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-97) [56bdb812] START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 1bab2fb4
>2012-11-09 06:12:49,884 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-97) [56bdb812] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:12:49,886 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-97) [56bdb812] Vds: Ovirt
>2012-11-09 06:12:49,886 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-97) [56bdb812] Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:12:49,887 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-97) [56bdb812] FINISH, ConnectStoragePoolVDSCommand, log id: 1bab2fb4
>2012-11-09 06:12:49,888 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-97) [56bdb812] IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:12:49,888 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-97) [56bdb812] Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:12:49,895 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-97) [69f2af74] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:12:49,954 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-97) [69f2af74] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:12:49,999 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-97) [69f2af74] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 18aa1594
>2012-11-09 06:12:49,999 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-97) [69f2af74] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:12:50,000 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-97) [69f2af74] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 18aa1594
>2012-11-09 06:12:52,585 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-3) [60b43093] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt
>2012-11-09 06:13:00,076 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-17) hostFromVds::selectedVds - Ovirt, spmStatus Unknown_Pool, storage pool Default
>2012-11-09 06:13:00,221 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-17) START, ConnectStoragePoolVDSCommand(vdsId = 093d532e-2750-11e2-a9ed-002590730631, storagePoolId = c02503a4-274e-11e2-b33c-002590730631, vds_spm_id = 1, masterDomainId = b1658676-2dc3-4357-b112-3852a056c663, masterVersion = 1), log id: 18ce4e14
>2012-11-09 06:13:01,080 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-17) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value 
> Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>mStatus                       Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>mCode                         304
>mMessage                      Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>
>
>2012-11-09 06:13:01,082 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-17) Vds: Ovirt
>2012-11-09 06:13:01,082 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-17) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:13:01,083 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-17) FINISH, ConnectStoragePoolVDSCommand, log id: 18ce4e14
>2012-11-09 06:13:01,084 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-17) IrsBroker::Failed::GetStoragePoolInfoVDS
>2012-11-09 06:13:01,084 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-17) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=c02503a4-274e-11e2-b33c-002590730631, msdUUID=b1658676-2dc3-4357-b112-3852a056c663'
>2012-11-09 06:13:01,090 WARN  [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (QuartzScheduler_Worker-17) [63672619] The message key ReconstructMasterDomain is missing from bundles/ExecutionMessages
>2012-11-09 06:13:01,155 INFO  [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-17) [63672619] Running command: ReconstructMasterDomainCommand internal: true. Entities affected :  ID: b1658676-2dc3-4357-b112-3852a056c663 Type: Storage
>2012-11-09 06:13:01,200 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-17) [63672619] START, MarkPoolInReconstructModeVDSCommand(storagePoolId = c02503a4-274e-11e2-b33c-002590730631, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 5a8975c
>2012-11-09 06:13:01,201 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-17) [63672619] clearing cache for problematic entities in pool c02503a4-274e-11e2-b33c-002590730631
>2012-11-09 06:13:01,201 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-17) [63672619] FINISH, MarkPoolInReconstructModeVDSCommand, log id: 5a8975c
>2012-11-09 06:13:02,771 WARN  [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-18) [d729fff] domain d1eaaed8-01da-4188-b54b-77175adebb90 in problem. vds: Ovirt

Actions: View

Attachments on bug 874120: 640114 | 641526 | 641527