Attachment 853858 Details for Bug 1056544 – VDSM log

VDSM log

vdsm.log (text/plain), 55.16 KB, created by Shanzhi Yu on 2014-01-22 12:53:40 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Shanzhi Yu

Created: 2014-01-22 12:53:40 UTC

Size: 55.16 KB

patch

obsolete

>Thread-19::DEBUG::2014-01-22 20:51:30,111::domainMonitor::178::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 7af59253-11d4-43e4-8d3c-a68a4d2650c2
>Thread-19::ERROR::2014-01-22 20:51:30,111::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain 7af59253-11d4-43e4-8d3c-a68a4d2650c2
>Thread-19::ERROR::2014-01-22 20:51:30,112::sdc::153::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain 7af59253-11d4-43e4-8d3c-a68a4d2650c2
>Thread-19::DEBUG::2014-01-22 20:51:30,112::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend
>Thread-19::DEBUG::2014-01-22 20:51:30,112::misc::83::Storage.Misc.excCmd::(<lambda>) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/7af59253-11d4-43e4-8d3c-a68a4d2650c2/metadata count=1' (cwd None)
>Thread-19::DEBUG::2014-01-22 20:51:30,121::misc::83::Storage.Misc.excCmd::(<lambda>) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.00267727 s, 765 kB/s\n'; <rc> = 0
>Thread-19::DEBUG::2014-01-22 20:51:30,121::misc::384::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.00267727 s, 765 kB/s'], size: 2048
>Thread-19::DEBUG::2014-01-22 20:51:30,121::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[]
>Thread-19::WARNING::2014-01-22 20:51:30,122::persistentDict::256::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>Thread-19::DEBUG::2014-01-22 20:51:30,122::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend
>Thread-19::DEBUG::2014-01-22 20:51:30,122::lvm::491::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-19::DEBUG::2014-01-22 20:51:30,122::lvm::493::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-19::DEBUG::2014-01-22 20:51:30,123::lvm::501::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-19::DEBUG::2014-01-22 20:51:30,123::lvm::513::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-19::DEBUG::2014-01-22 20:51:30,123::lvm::374::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-19::DEBUG::2014-01-22 20:51:30,124::misc::83::Storage.Misc.excCmd::(<lambda>) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \'a|/dev/mapper/36090a038d0f751994bf9045fb249cf34|/dev/mapper/36090a038d0f7b16e4df9245fb249cf30|\', \'r|.*|\' ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 7af59253-11d4-43e4-8d3c-a68a4d2650c2' (cwd None)
>Thread-18::DEBUG::2014-01-22 20:51:30,250::domainMonitor::178::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf
>Thread-18::ERROR::2014-01-22 20:51:30,251::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf
>Thread-18::ERROR::2014-01-22 20:51:30,251::sdc::153::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf
>Thread-18::DEBUG::2014-01-22 20:51:30,251::lvm::374::OperationMutex::(_reloadvgs) Got the operational mutex
>Thread-18::DEBUG::2014-01-22 20:51:30,252::misc::83::Storage.Misc.excCmd::(<lambda>) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \'a|/dev/mapper/36090a038d0f751994bf9045fb249cf34|/dev/mapper/36090a038d0f7b16e4df9245fb249cf30|\', \'r|.*|\' ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf' (cwd None)
>Thread-19::DEBUG::2014-01-22 20:51:30,301::misc::83::Storage.Misc.excCmd::(<lambda>) SUCCESS: <err> = ''; <rc> = 0
>Thread-19::DEBUG::2014-01-22 20:51:30,303::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['DESCRIPTION=lsu-2', 'VERSION=3', 'TYPE=ISCSI', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKRENEWALINTERVALSEC=5', u'PV0=pv:36090a038d0f751994bf9045fb249cf34,uuid:TsWoLt-pVsc-eyKx-uHw1-BKvI-5MX3-RANJ7C,pestart:0,pecount:237,mapoffset:0', 'PHYBLKSIZE=512', 'SDUUID=7af59253-11d4-43e4-8d3c-a68a4d2650c2', 'IOOPTIMEOUTSEC=1', 'CLASS=Data', 'LOCKPOLICY=', 'VGUUID=6ZtLFc-0Ev3-qj0N-sJr8-Nm7Z-IGaV-GvZ6JG', 'POOL_UUID=c4777d10-e010-4a3a-8fe1-9149263e5bf4', 'MASTER_VERSION=2', 'ROLE=Master', 'POOL_DESCRIPTION=lsu', 'POOL_SPM_ID=2', 'POOL_SPM_LVER=0', '_SHA_CKSUM=9ab174805a61e644a5fd6a0b980b29f7eb54f427', u'POOL_DOMAINS=9b0af9c6-96a2-43c5-857a-61e9f45b6cdf:Active,868c427d-c6a7-43e9-bfa1-dc107ca69bec:Attached,7af59253-11d4-43e4-8d3c-a68a4d2650c2:Active']
>Thread-19::DEBUG::2014-01-22 20:51:30,303::lvm::340::OperationMutex::(_reloadpvs) Got the operational mutex
>Thread-19::DEBUG::2014-01-22 20:51:30,304::misc::83::Storage.Misc.excCmd::(<lambda>) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \'a|/dev/mapper/36090a038d0f751994bf9045fb249cf34|/dev/mapper/36090a038d0f7b16e4df9245fb249cf30|\', \'r|.*|\' ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | --ignoreskippedcluster -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None)
>Thread-18::DEBUG::2014-01-22 20:51:30,435::misc::83::Storage.Misc.excCmd::(<lambda>) FAILED: <err> = '  Volume group "9b0af9c6-96a2-43c5-857a-61e9f45b6cdf" not found\n  Skipping volume group 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf\n'; <rc> = 5
>Thread-18::WARNING::2014-01-22 20:51:30,436::lvm::379::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "9b0af9c6-96a2-43c5-857a-61e9f45b6cdf" not found', '  Skipping volume group 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf']
>Thread-18::ERROR::2014-01-22 20:51:30,437::sdc::143::Storage.StorageDomainCache::(_findDomain) domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf not found
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain
>    dom = findMethod(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 170, in _findUnfetchedDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'9b0af9c6-96a2-43c5-857a-61e9f45b6cdf',)
>Thread-18::ERROR::2014-01-22 20:51:30,437::domainMonitor::225::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 190, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 98, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain
>    dom = findMethod(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 170, in _findUnfetchedDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'9b0af9c6-96a2-43c5-857a-61e9f45b6cdf',)
>Thread-19::DEBUG::2014-01-22 20:51:30,458::misc::83::Storage.Misc.excCmd::(<lambda>) SUCCESS: <err> = ''; <rc> = 0
>Thread-19::DEBUG::2014-01-22 20:51:30,458::lvm::365::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex
>Thread-19::WARNING::2014-01-22 20:51:30,459::sd::370::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 7af59253-11d4-43e4-8d3c-a68a4d2650c2_imageNS already registered
>Thread-19::WARNING::2014-01-22 20:51:30,459::sd::378::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 7af59253-11d4-43e4-8d3c-a68a4d2650c2_volumeNS already registered
>Thread-19::WARNING::2014-01-22 20:51:30,459::blockSD::421::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 7af59253-11d4-43e4-8d3c-a68a4d2650c2_lvmActivationNS already registered
>Thread-19::DEBUG::2014-01-22 20:51:30,460::misc::83::Storage.Misc.excCmd::(<lambda>) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \'a|/dev/mapper/36090a038d0f751994bf9045fb249cf34|/dev/mapper/36090a038d0f7b16e4df9245fb249cf30|\', \'r|.*|\' ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 7af59253-11d4-43e4-8d3c-a68a4d2650c2' (cwd None)
>Thread-19::DEBUG::2014-01-22 20:51:30,612::misc::83::Storage.Misc.excCmd::(<lambda>) SUCCESS: <err> = ''; <rc> = 0
>Thread-19::DEBUG::2014-01-22 20:51:30,612::misc::83::Storage.Misc.excCmd::(<lambda>) '/bin/dd iflag=direct if=/dev/7af59253-11d4-43e4-8d3c-a68a4d2650c2/metadata bs=4096 count=1' (cwd None)
>Thread-19::DEBUG::2014-01-22 20:51:30,624::misc::83::Storage.Misc.excCmd::(<lambda>) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00429707 s, 953 kB/s\n'; <rc> = 0
>Thread-201::DEBUG::2014-01-22 20:51:32,022::task::579::TaskManager.Task::(_updateState) Task=`4d324c9e-fc14-4444-9f28-e762d95be9bd`::moving from state init -> state preparing
>Thread-201::INFO::2014-01-22 20:51:32,022::logUtils::40::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-201::INFO::2014-01-22 20:51:32,022::logUtils::42::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9b0af9c6-96a2-43c5-857a-61e9f45b6cdf': {'delay': '0', 'lastCheck': '1.6', 'code': 358, 'valid': False, 'version': -1}, u'7af59253-11d4-43e4-8d3c-a68a4d2650c2': {'delay': '0.00429707', 'lastCheck': '1.4', 'code': 0, 'valid': True, 'version': 3}}
>Thread-201::DEBUG::2014-01-22 20:51:32,022::task::1168::TaskManager.Task::(prepare) Task=`4d324c9e-fc14-4444-9f28-e762d95be9bd`::finished: {u'9b0af9c6-96a2-43c5-857a-61e9f45b6cdf': {'delay': '0', 'lastCheck': '1.6', 'code': 358, 'valid': False, 'version': -1}, u'7af59253-11d4-43e4-8d3c-a68a4d2650c2': {'delay': '0.00429707', 'lastCheck': '1.4', 'code': 0, 'valid': True, 'version': 3}}
>Thread-201::DEBUG::2014-01-22 20:51:32,022::task::579::TaskManager.Task::(_updateState) Task=`4d324c9e-fc14-4444-9f28-e762d95be9bd`::moving from state preparing -> state finished
>Thread-201::DEBUG::2014-01-22 20:51:32,023::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-201::DEBUG::2014-01-22 20:51:32,023::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-201::DEBUG::2014-01-22 20:51:32,023::task::974::TaskManager.Task::(_decref) Task=`4d324c9e-fc14-4444-9f28-e762d95be9bd`::ref 0 aborting False
>Thread-18::ERROR::2014-01-22 20:51:40,438::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf
>Thread-18::ERROR::2014-01-22 20:51:40,438::sdc::153::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf
>Thread-18::DEBUG::2014-01-22 20:51:40,438::lvm::374::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-18::DEBUG::2014-01-22 20:51:40,439::misc::83::Storage.Misc.excCmd::(<lambda>) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \'a|/dev/mapper/36090a038d0f751994bf9045fb249cf34|/dev/mapper/36090a038d0f7b16e4df9245fb249cf30|\', \'r|.*|\' ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf' (cwd None)
>Thread-18::DEBUG::2014-01-22 20:51:40,623::misc::83::Storage.Misc.excCmd::(<lambda>) FAILED: <err> = '  Volume group "9b0af9c6-96a2-43c5-857a-61e9f45b6cdf" not found\n  Skipping volume group 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf\n'; <rc> = 5
>Thread-18::WARNING::2014-01-22 20:51:40,624::lvm::379::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "9b0af9c6-96a2-43c5-857a-61e9f45b6cdf" not found', '  Skipping volume group 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf']
>Thread-18::DEBUG::2014-01-22 20:51:40,625::lvm::403::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-18::ERROR::2014-01-22 20:51:40,625::sdc::143::Storage.StorageDomainCache::(_findDomain) domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf not found
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain
>    dom = findMethod(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 170, in _findUnfetchedDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'9b0af9c6-96a2-43c5-857a-61e9f45b6cdf',)
>Thread-18::ERROR::2014-01-22 20:51:40,626::domainMonitor::225::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 190, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 98, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain
>    dom = findMethod(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 170, in _findUnfetchedDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'9b0af9c6-96a2-43c5-857a-61e9f45b6cdf',)
>Thread-19::DEBUG::2014-01-22 20:51:40,627::misc::83::Storage.Misc.excCmd::(<lambda>) '/bin/dd iflag=direct if=/dev/7af59253-11d4-43e4-8d3c-a68a4d2650c2/metadata bs=4096 count=1' (cwd None)
>Thread-19::DEBUG::2014-01-22 20:51:40,635::misc::83::Storage.Misc.excCmd::(<lambda>) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000651143 s, 6.3 MB/s\n'; <rc> = 0
>Thread-100::DEBUG::2014-01-22 20:51:42,888::task::579::TaskManager.Task::(_updateState) Task=`bca73306-be9f-4646-ae71-84cf5c78ad0f`::moving from state init -> state preparing
>Thread-100::INFO::2014-01-22 20:51:42,889::logUtils::40::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='7af59253-11d4-43e4-8d3c-a68a4d2650c2', spUUID='c4777d10-e010-4a3a-8fe1-9149263e5bf4', imgUUID='47ae4d42-7fda-4e86-ad21-942f802a3e7d', volUUID='bbd21e13-3b14-4592-b723-19032750a639', options=None)
>Thread-100::INFO::2014-01-22 20:51:42,889::logUtils::42::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1073741824', 'apparentsize': '1073741824'}
>Thread-100::DEBUG::2014-01-22 20:51:42,889::task::1168::TaskManager.Task::(prepare) Task=`bca73306-be9f-4646-ae71-84cf5c78ad0f`::finished: {'truesize': '1073741824', 'apparentsize': '1073741824'}
>Thread-100::DEBUG::2014-01-22 20:51:42,889::task::579::TaskManager.Task::(_updateState) Task=`bca73306-be9f-4646-ae71-84cf5c78ad0f`::moving from state preparing -> state finished
>Thread-100::DEBUG::2014-01-22 20:51:42,890::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-100::DEBUG::2014-01-22 20:51:42,890::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-100::DEBUG::2014-01-22 20:51:42,890::task::974::TaskManager.Task::(_decref) Task=`bca73306-be9f-4646-ae71-84cf5c78ad0f`::ref 0 aborting False
>VM Channels Listener::DEBUG::2014-01-22 20:51:43,377::vmChannels::90::vds::(_handle_timeouts) Timeout on fileno 16.
>Thread-207::DEBUG::2014-01-22 20:51:47,160::task::579::TaskManager.Task::(_updateState) Task=`4d52ee29-dafe-44ed-88d7-4f98b8ad80a1`::moving from state init -> state preparing
>Thread-207::INFO::2014-01-22 20:51:47,160::logUtils::40::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-207::INFO::2014-01-22 20:51:47,160::logUtils::42::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9b0af9c6-96a2-43c5-857a-61e9f45b6cdf': {'delay': '0', 'lastCheck': '6.5', 'code': 358, 'valid': False, 'version': -1}, u'7af59253-11d4-43e4-8d3c-a68a4d2650c2': {'delay': '0.000651143', 'lastCheck': '6.5', 'code': 0, 'valid': True, 'version': 3}}
>Thread-207::DEBUG::2014-01-22 20:51:47,160::task::1168::TaskManager.Task::(prepare) Task=`4d52ee29-dafe-44ed-88d7-4f98b8ad80a1`::finished: {u'9b0af9c6-96a2-43c5-857a-61e9f45b6cdf': {'delay': '0', 'lastCheck': '6.5', 'code': 358, 'valid': False, 'version': -1}, u'7af59253-11d4-43e4-8d3c-a68a4d2650c2': {'delay': '0.000651143', 'lastCheck': '6.5', 'code': 0, 'valid': True, 'version': 3}}
>Thread-207::DEBUG::2014-01-22 20:51:47,160::task::579::TaskManager.Task::(_updateState) Task=`4d52ee29-dafe-44ed-88d7-4f98b8ad80a1`::moving from state preparing -> state finished
>Thread-207::DEBUG::2014-01-22 20:51:47,161::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-207::DEBUG::2014-01-22 20:51:47,161::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-207::DEBUG::2014-01-22 20:51:47,161::task::974::TaskManager.Task::(_decref) Task=`4d52ee29-dafe-44ed-88d7-4f98b8ad80a1`::ref 0 aborting False
>Thread-18::ERROR::2014-01-22 20:51:50,627::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf
>Thread-18::ERROR::2014-01-22 20:51:50,627::sdc::153::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf
>Thread-18::DEBUG::2014-01-22 20:51:50,627::lvm::374::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-18::DEBUG::2014-01-22 20:51:50,628::misc::83::Storage.Misc.excCmd::(<lambda>) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \'a|/dev/mapper/36090a038d0f751994bf9045fb249cf34|/dev/mapper/36090a038d0f7b16e4df9245fb249cf30|\', \'r|.*|\' ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf' (cwd None)
>Thread-19::DEBUG::2014-01-22 20:51:50,637::misc::83::Storage.Misc.excCmd::(<lambda>) '/bin/dd iflag=direct if=/dev/7af59253-11d4-43e4-8d3c-a68a4d2650c2/metadata bs=4096 count=1' (cwd None)
>Thread-19::DEBUG::2014-01-22 20:51:50,645::misc::83::Storage.Misc.excCmd::(<lambda>) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000595041 s, 6.9 MB/s\n'; <rc> = 0
>Thread-18::DEBUG::2014-01-22 20:51:50,823::misc::83::Storage.Misc.excCmd::(<lambda>) FAILED: <err> = '  Volume group "9b0af9c6-96a2-43c5-857a-61e9f45b6cdf" not found\n  Skipping volume group 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf\n'; <rc> = 5
>Thread-18::WARNING::2014-01-22 20:51:50,825::lvm::379::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "9b0af9c6-96a2-43c5-857a-61e9f45b6cdf" not found', '  Skipping volume group 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf']
>Thread-18::DEBUG::2014-01-22 20:51:50,825::lvm::403::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-18::ERROR::2014-01-22 20:51:50,826::sdc::143::Storage.StorageDomainCache::(_findDomain) domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf not found
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain
>    dom = findMethod(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 170, in _findUnfetchedDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'9b0af9c6-96a2-43c5-857a-61e9f45b6cdf',)
>Thread-18::ERROR::2014-01-22 20:51:50,827::domainMonitor::225::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 190, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 98, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain
>    dom = findMethod(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 170, in _findUnfetchedDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'9b0af9c6-96a2-43c5-857a-61e9f45b6cdf',)
>Thread-212::DEBUG::2014-01-22 20:51:58,273::BindingXMLRPC::913::vds::(wrapper) client [10.66.7.235]::call vmMigrate with ({'src': '10.66.106.28', 'dst': '10.66.106.25:54321', 'vmId': 'a0be0e96-cdb9-4692-9e85-8ee55a4c2ecc', 'method': 'online'},) {} flowID [faf089e]
>Thread-212::DEBUG::2014-01-22 20:51:58,273::API::454::vds::(migrate) {'src': '10.66.106.28', 'dst': '10.66.106.25:54321', 'vmId': 'a0be0e96-cdb9-4692-9e85-8ee55a4c2ecc', 'method': 'online'}
>Thread-212::DEBUG::2014-01-22 20:51:58,274::BindingXMLRPC::920::vds::(wrapper) return vmMigrate with {'status': {'message': 'Migration process starting', 'code': 0}}
>Thread-213::DEBUG::2014-01-22 20:51:58,274::vm::140::vm.Vm::(_setupVdsConnection) vmId=`a0be0e96-cdb9-4692-9e85-8ee55a4c2ecc`::Destination server is: 10.66.106.25:54321
>Thread-213::DEBUG::2014-01-22 20:51:58,275::vm::142::vm.Vm::(_setupVdsConnection) vmId=`a0be0e96-cdb9-4692-9e85-8ee55a4c2ecc`::Initiating connection with destination
>Thread-213::DEBUG::2014-01-22 20:51:58,328::vm::191::vm.Vm::(_prepareGuest) vmId=`a0be0e96-cdb9-4692-9e85-8ee55a4c2ecc`::migration Process begins
>Thread-213::DEBUG::2014-01-22 20:51:58,369::vm::277::vm.Vm::(run) vmId=`a0be0e96-cdb9-4692-9e85-8ee55a4c2ecc`::migration semaphore acquired
>Thread-213::DEBUG::2014-01-22 20:51:59,004::libvirtvm::454::vm.Vm::(_startUnderlyingMigration) vmId=`a0be0e96-cdb9-4692-9e85-8ee55a4c2ecc`::starting migration to qemu+tls://10.66.106.25/system
>Thread-214::DEBUG::2014-01-22 20:51:59,004::libvirtvm::333::vm.Vm::(run) vmId=`a0be0e96-cdb9-4692-9e85-8ee55a4c2ecc`::migration downtime thread started
>Thread-215::DEBUG::2014-01-22 20:51:59,005::libvirtvm::369::vm.Vm::(run) vmId=`a0be0e96-cdb9-4692-9e85-8ee55a4c2ecc`::starting migration monitor thread
>Thread-213::DEBUG::2014-01-22 20:51:59,132::libvirtconnection::128::vds::(wrapper) Unknown libvirterror: ecode: 9 edom: 10 level: 2 message: operation failed: Failed to connect to remote libvirt URI qemu+tls://10.66.106.25/system
>Thread-213::DEBUG::2014-01-22 20:51:59,132::libvirtvm::348::vm.Vm::(cancel) vmId=`a0be0e96-cdb9-4692-9e85-8ee55a4c2ecc`::canceling migration downtime thread
>Thread-213::DEBUG::2014-01-22 20:51:59,132::libvirtvm::415::vm.Vm::(stop) vmId=`a0be0e96-cdb9-4692-9e85-8ee55a4c2ecc`::stopping migration monitor thread
>Thread-214::DEBUG::2014-01-22 20:51:59,132::libvirtvm::345::vm.Vm::(run) vmId=`a0be0e96-cdb9-4692-9e85-8ee55a4c2ecc`::migration downtime thread exiting
>Thread-213::ERROR::2014-01-22 20:51:59,133::vm::197::vm.Vm::(_recover) vmId=`a0be0e96-cdb9-4692-9e85-8ee55a4c2ecc`::operation failed: Failed to connect to remote libvirt URI qemu+tls://10.66.106.25/system
>Thread-213::ERROR::2014-01-22 20:51:59,685::vm::300::vm.Vm::(run) vmId=`a0be0e96-cdb9-4692-9e85-8ee55a4c2ecc`::Failed to migrate
>Traceback (most recent call last):
>  File "/usr/share/vdsm/vm.py", line 285, in run
>    self._startUnderlyingMigration()
>  File "/usr/share/vdsm/libvirtvm.py", line 479, in _startUnderlyingMigration
>    None, maxBandwidth)
>  File "/usr/share/vdsm/libvirtvm.py", line 517, in f
>    ret = attr(*args, **kwargs)
>  File "/usr/lib64/python2.6/site-packages/vdsm/libvirtconnection.py", line 104, in wrapper
>    ret = f(*args, **kwargs)
>  File "/usr/lib64/python2.6/site-packages/libvirt.py", line 1178, in migrateToURI2
>    if ret == -1: raise libvirtError ('virDomainMigrateToURI2() failed', dom=self)
>libvirtError: operation failed: Failed to connect to remote libvirt URI qemu+tls://10.66.106.25/system
>Thread-19::DEBUG::2014-01-22 20:52:00,647::misc::83::Storage.Misc.excCmd::(<lambda>) '/bin/dd iflag=direct if=/dev/7af59253-11d4-43e4-8d3c-a68a4d2650c2/metadata bs=4096 count=1' (cwd None)
>Thread-19::DEBUG::2014-01-22 20:52:00,656::misc::83::Storage.Misc.excCmd::(<lambda>) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00108833 s, 3.8 MB/s\n'; <rc> = 0
>Thread-18::ERROR::2014-01-22 20:52:00,827::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf
>Thread-18::ERROR::2014-01-22 20:52:00,828::sdc::153::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf
>Thread-18::DEBUG::2014-01-22 20:52:00,828::lvm::374::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-18::DEBUG::2014-01-22 20:52:00,828::misc::83::Storage.Misc.excCmd::(<lambda>) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \'a|/dev/mapper/36090a038d0f751994bf9045fb249cf34|/dev/mapper/36090a038d0f7b16e4df9245fb249cf30|\', \'r|.*|\' ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf' (cwd None)
>Thread-18::DEBUG::2014-01-22 20:52:01,025::misc::83::Storage.Misc.excCmd::(<lambda>) FAILED: <err> = '  Volume group "9b0af9c6-96a2-43c5-857a-61e9f45b6cdf" not found\n  Skipping volume group 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf\n'; <rc> = 5
>Thread-18::WARNING::2014-01-22 20:52:01,027::lvm::379::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "9b0af9c6-96a2-43c5-857a-61e9f45b6cdf" not found', '  Skipping volume group 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf']
>Thread-18::DEBUG::2014-01-22 20:52:01,027::lvm::403::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-18::ERROR::2014-01-22 20:52:01,028::sdc::143::Storage.StorageDomainCache::(_findDomain) domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf not found
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain
>    dom = findMethod(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 170, in _findUnfetchedDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'9b0af9c6-96a2-43c5-857a-61e9f45b6cdf',)
>Thread-18::ERROR::2014-01-22 20:52:01,028::domainMonitor::225::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 190, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 98, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain
>    dom = findMethod(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 170, in _findUnfetchedDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'9b0af9c6-96a2-43c5-857a-61e9f45b6cdf',)
>Thread-217::DEBUG::2014-01-22 20:52:02,330::task::579::TaskManager.Task::(_updateState) Task=`b27451a8-63b3-422a-9e1e-bb4cf7e5c181`::moving from state init -> state preparing
>Thread-217::INFO::2014-01-22 20:52:02,330::logUtils::40::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-217::INFO::2014-01-22 20:52:02,331::logUtils::42::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9b0af9c6-96a2-43c5-857a-61e9f45b6cdf': {'delay': '0', 'lastCheck': '1.3', 'code': 358, 'valid': False, 'version': -1}, u'7af59253-11d4-43e4-8d3c-a68a4d2650c2': {'delay': '0.00108833', 'lastCheck': '1.7', 'code': 0, 'valid': True, 'version': 3}}
>Thread-217::DEBUG::2014-01-22 20:52:02,331::task::1168::TaskManager.Task::(prepare) Task=`b27451a8-63b3-422a-9e1e-bb4cf7e5c181`::finished: {u'9b0af9c6-96a2-43c5-857a-61e9f45b6cdf': {'delay': '0', 'lastCheck': '1.3', 'code': 358, 'valid': False, 'version': -1}, u'7af59253-11d4-43e4-8d3c-a68a4d2650c2': {'delay': '0.00108833', 'lastCheck': '1.7', 'code': 0, 'valid': True, 'version': 3}}
>Thread-217::DEBUG::2014-01-22 20:52:02,331::task::579::TaskManager.Task::(_updateState) Task=`b27451a8-63b3-422a-9e1e-bb4cf7e5c181`::moving from state preparing -> state finished
>Thread-217::DEBUG::2014-01-22 20:52:02,331::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-217::DEBUG::2014-01-22 20:52:02,331::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-217::DEBUG::2014-01-22 20:52:02,332::task::974::TaskManager.Task::(_decref) Task=`b27451a8-63b3-422a-9e1e-bb4cf7e5c181`::ref 0 aborting False
>Thread-220::DEBUG::2014-01-22 20:52:02,709::BindingXMLRPC::913::vds::(wrapper) client [10.66.7.235]::call vmGetMigrationStatus with ('a0be0e96-cdb9-4692-9e85-8ee55a4c2ecc',) {} flowID [153b34d8]
>Thread-220::DEBUG::2014-01-22 20:52:02,709::BindingXMLRPC::920::vds::(wrapper) return vmGetMigrationStatus with {'status': {'message': 'Fatal error during migration', 'code': 12}}
>Thread-19::DEBUG::2014-01-22 20:52:10,659::misc::83::Storage.Misc.excCmd::(<lambda>) '/bin/dd iflag=direct if=/dev/7af59253-11d4-43e4-8d3c-a68a4d2650c2/metadata bs=4096 count=1' (cwd None)
>Thread-19::DEBUG::2014-01-22 20:52:10,667::misc::83::Storage.Misc.excCmd::(<lambda>) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000715862 s, 5.7 MB/s\n'; <rc> = 0
>Thread-18::ERROR::2014-01-22 20:52:11,028::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf
>Thread-18::ERROR::2014-01-22 20:52:11,029::sdc::153::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf
>Thread-18::DEBUG::2014-01-22 20:52:11,029::lvm::374::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-18::DEBUG::2014-01-22 20:52:11,030::misc::83::Storage.Misc.excCmd::(<lambda>) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \'a|/dev/mapper/36090a038d0f751994bf9045fb249cf34|/dev/mapper/36090a038d0f7b16e4df9245fb249cf30|\', \'r|.*|\' ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf' (cwd None)
>Thread-18::DEBUG::2014-01-22 20:52:11,180::misc::83::Storage.Misc.excCmd::(<lambda>) FAILED: <err> = '  Volume group "9b0af9c6-96a2-43c5-857a-61e9f45b6cdf" not found\n  Skipping volume group 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf\n'; <rc> = 5
>Thread-18::WARNING::2014-01-22 20:52:11,182::lvm::379::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "9b0af9c6-96a2-43c5-857a-61e9f45b6cdf" not found', '  Skipping volume group 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf']
>Thread-18::DEBUG::2014-01-22 20:52:11,182::lvm::403::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-18::ERROR::2014-01-22 20:52:11,183::sdc::143::Storage.StorageDomainCache::(_findDomain) domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf not found
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain
>    dom = findMethod(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 170, in _findUnfetchedDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'9b0af9c6-96a2-43c5-857a-61e9f45b6cdf',)
>Thread-18::ERROR::2014-01-22 20:52:11,183::domainMonitor::225::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 190, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 98, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain
>    dom = findMethod(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 170, in _findUnfetchedDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'9b0af9c6-96a2-43c5-857a-61e9f45b6cdf',)
>VM Channels Listener::DEBUG::2014-01-22 20:52:13,410::vmChannels::90::vds::(_handle_timeouts) Timeout on fileno 16.
>Thread-225::DEBUG::2014-01-22 20:52:17,776::task::579::TaskManager.Task::(_updateState) Task=`83591bc6-a065-42a2-b307-f6756b1cffb9`::moving from state init -> state preparing
>Thread-225::INFO::2014-01-22 20:52:17,776::logUtils::40::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-225::INFO::2014-01-22 20:52:17,776::logUtils::42::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9b0af9c6-96a2-43c5-857a-61e9f45b6cdf': {'delay': '0', 'lastCheck': '6.6', 'code': 358, 'valid': False, 'version': -1}, u'7af59253-11d4-43e4-8d3c-a68a4d2650c2': {'delay': '0.000715862', 'lastCheck': '7.1', 'code': 0, 'valid': True, 'version': 3}}
>Thread-225::DEBUG::2014-01-22 20:52:17,776::task::1168::TaskManager.Task::(prepare) Task=`83591bc6-a065-42a2-b307-f6756b1cffb9`::finished: {u'9b0af9c6-96a2-43c5-857a-61e9f45b6cdf': {'delay': '0', 'lastCheck': '6.6', 'code': 358, 'valid': False, 'version': -1}, u'7af59253-11d4-43e4-8d3c-a68a4d2650c2': {'delay': '0.000715862', 'lastCheck': '7.1', 'code': 0, 'valid': True, 'version': 3}}
>Thread-225::DEBUG::2014-01-22 20:52:17,776::task::579::TaskManager.Task::(_updateState) Task=`83591bc6-a065-42a2-b307-f6756b1cffb9`::moving from state preparing -> state finished
>Thread-225::DEBUG::2014-01-22 20:52:17,777::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-225::DEBUG::2014-01-22 20:52:17,777::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-225::DEBUG::2014-01-22 20:52:17,777::task::974::TaskManager.Task::(_decref) Task=`83591bc6-a065-42a2-b307-f6756b1cffb9`::ref 0 aborting False
>Thread-19::DEBUG::2014-01-22 20:52:20,669::misc::83::Storage.Misc.excCmd::(<lambda>) '/bin/dd iflag=direct if=/dev/7af59253-11d4-43e4-8d3c-a68a4d2650c2/metadata bs=4096 count=1' (cwd None)
>Thread-19::DEBUG::2014-01-22 20:52:20,677::misc::83::Storage.Misc.excCmd::(<lambda>) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000801678 s, 5.1 MB/s\n'; <rc> = 0
>Thread-18::ERROR::2014-01-22 20:52:21,183::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf
>Thread-18::ERROR::2014-01-22 20:52:21,184::sdc::153::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf
>Thread-18::DEBUG::2014-01-22 20:52:21,184::lvm::374::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-18::DEBUG::2014-01-22 20:52:21,185::misc::83::Storage.Misc.excCmd::(<lambda>) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \'a|/dev/mapper/36090a038d0f751994bf9045fb249cf34|/dev/mapper/36090a038d0f7b16e4df9245fb249cf30|\', \'r|.*|\' ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf' (cwd None)
>Thread-18::DEBUG::2014-01-22 20:52:21,336::misc::83::Storage.Misc.excCmd::(<lambda>) FAILED: <err> = '  Volume group "9b0af9c6-96a2-43c5-857a-61e9f45b6cdf" not found\n  Skipping volume group 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf\n'; <rc> = 5
>Thread-18::WARNING::2014-01-22 20:52:21,337::lvm::379::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "9b0af9c6-96a2-43c5-857a-61e9f45b6cdf" not found', '  Skipping volume group 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf']
>Thread-18::DEBUG::2014-01-22 20:52:21,338::lvm::403::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-18::ERROR::2014-01-22 20:52:21,338::sdc::143::Storage.StorageDomainCache::(_findDomain) domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf not found
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain
>    dom = findMethod(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 170, in _findUnfetchedDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'9b0af9c6-96a2-43c5-857a-61e9f45b6cdf',)
>Thread-18::ERROR::2014-01-22 20:52:21,339::domainMonitor::225::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 190, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 98, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain
>    dom = findMethod(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 170, in _findUnfetchedDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'9b0af9c6-96a2-43c5-857a-61e9f45b6cdf',)
>Thread-19::DEBUG::2014-01-22 20:52:30,680::misc::83::Storage.Misc.excCmd::(<lambda>) '/bin/dd iflag=direct if=/dev/7af59253-11d4-43e4-8d3c-a68a4d2650c2/metadata bs=4096 count=1' (cwd None)
>Thread-19::DEBUG::2014-01-22 20:52:30,690::misc::83::Storage.Misc.excCmd::(<lambda>) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00324796 s, 1.3 MB/s\n'; <rc> = 0
>Thread-18::ERROR::2014-01-22 20:52:31,339::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf
>Thread-18::ERROR::2014-01-22 20:52:31,339::sdc::153::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf
>Thread-18::DEBUG::2014-01-22 20:52:31,340::lvm::374::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-18::DEBUG::2014-01-22 20:52:31,340::misc::83::Storage.Misc.excCmd::(<lambda>) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \'a|/dev/mapper/36090a038d0f751994bf9045fb249cf34|/dev/mapper/36090a038d0f7b16e4df9245fb249cf30|\', \'r|.*|\' ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf' (cwd None)
>Thread-18::DEBUG::2014-01-22 20:52:31,490::misc::83::Storage.Misc.excCmd::(<lambda>) FAILED: <err> = '  Volume group "9b0af9c6-96a2-43c5-857a-61e9f45b6cdf" not found\n  Skipping volume group 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf\n'; <rc> = 5
>Thread-18::WARNING::2014-01-22 20:52:31,492::lvm::379::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "9b0af9c6-96a2-43c5-857a-61e9f45b6cdf" not found', '  Skipping volume group 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf']
>Thread-18::DEBUG::2014-01-22 20:52:31,492::lvm::403::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-18::ERROR::2014-01-22 20:52:31,493::sdc::143::Storage.StorageDomainCache::(_findDomain) domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf not found
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain
>    dom = findMethod(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 170, in _findUnfetchedDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'9b0af9c6-96a2-43c5-857a-61e9f45b6cdf',)
>Thread-18::ERROR::2014-01-22 20:52:31,493::domainMonitor::225::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 190, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 98, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain
>    dom = findMethod(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 170, in _findUnfetchedDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'9b0af9c6-96a2-43c5-857a-61e9f45b6cdf',)
>Thread-231::DEBUG::2014-01-22 20:52:33,218::task::579::TaskManager.Task::(_updateState) Task=`859770ed-f171-4460-990e-8829c94a95c0`::moving from state init -> state preparing
>Thread-231::INFO::2014-01-22 20:52:33,218::logUtils::40::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-231::INFO::2014-01-22 20:52:33,218::logUtils::42::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9b0af9c6-96a2-43c5-857a-61e9f45b6cdf': {'delay': '0', 'lastCheck': '1.7', 'code': 358, 'valid': False, 'version': -1}, u'7af59253-11d4-43e4-8d3c-a68a4d2650c2': {'delay': '0.00324796', 'lastCheck': '2.5', 'code': 0, 'valid': True, 'version': 3}}
>Thread-231::DEBUG::2014-01-22 20:52:33,218::task::1168::TaskManager.Task::(prepare) Task=`859770ed-f171-4460-990e-8829c94a95c0`::finished: {u'9b0af9c6-96a2-43c5-857a-61e9f45b6cdf': {'delay': '0', 'lastCheck': '1.7', 'code': 358, 'valid': False, 'version': -1}, u'7af59253-11d4-43e4-8d3c-a68a4d2650c2': {'delay': '0.00324796', 'lastCheck': '2.5', 'code': 0, 'valid': True, 'version': 3}}
>Thread-231::DEBUG::2014-01-22 20:52:33,219::task::579::TaskManager.Task::(_updateState) Task=`859770ed-f171-4460-990e-8829c94a95c0`::moving from state preparing -> state finished
>Thread-231::DEBUG::2014-01-22 20:52:33,219::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-231::DEBUG::2014-01-22 20:52:33,219::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-231::DEBUG::2014-01-22 20:52:33,219::task::974::TaskManager.Task::(_decref) Task=`859770ed-f171-4460-990e-8829c94a95c0`::ref 0 aborting False
>Thread-19::DEBUG::2014-01-22 20:52:40,693::misc::83::Storage.Misc.excCmd::(<lambda>) '/bin/dd iflag=direct if=/dev/7af59253-11d4-43e4-8d3c-a68a4d2650c2/metadata bs=4096 count=1' (cwd None)
>Thread-19::DEBUG::2014-01-22 20:52:40,701::misc::83::Storage.Misc.excCmd::(<lambda>) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000663136 s, 6.2 MB/s\n'; <rc> = 0
>Thread-18::ERROR::2014-01-22 20:52:41,494::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf
>Thread-18::ERROR::2014-01-22 20:52:41,494::sdc::153::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf
>Thread-18::DEBUG::2014-01-22 20:52:41,494::lvm::374::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-18::DEBUG::2014-01-22 20:52:41,495::misc::83::Storage.Misc.excCmd::(<lambda>) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \'a|/dev/mapper/36090a038d0f751994bf9045fb249cf34|/dev/mapper/36090a038d0f7b16e4df9245fb249cf30|\', \'r|.*|\' ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf' (cwd None)
>Thread-18::DEBUG::2014-01-22 20:52:41,677::misc::83::Storage.Misc.excCmd::(<lambda>) FAILED: <err> = '  Volume group "9b0af9c6-96a2-43c5-857a-61e9f45b6cdf" not found\n  Skipping volume group 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf\n'; <rc> = 5
>Thread-18::WARNING::2014-01-22 20:52:41,679::lvm::379::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "9b0af9c6-96a2-43c5-857a-61e9f45b6cdf" not found', '  Skipping volume group 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf']
>Thread-18::DEBUG::2014-01-22 20:52:41,679::lvm::403::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-18::ERROR::2014-01-22 20:52:41,680::sdc::143::Storage.StorageDomainCache::(_findDomain) domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf not found
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain
>    dom = findMethod(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 170, in _findUnfetchedDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'9b0af9c6-96a2-43c5-857a-61e9f45b6cdf',)
>Thread-18::ERROR::2014-01-22 20:52:41,680::domainMonitor::225::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 190, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 98, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain
>    dom = findMethod(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 170, in _findUnfetchedDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'9b0af9c6-96a2-43c5-857a-61e9f45b6cdf',)
>Thread-100::DEBUG::2014-01-22 20:52:43,005::task::579::TaskManager.Task::(_updateState) Task=`548426bb-a8be-48fd-9a34-5b3f1014ebcc`::moving from state init -> state preparing
>Thread-100::INFO::2014-01-22 20:52:43,005::logUtils::40::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='7af59253-11d4-43e4-8d3c-a68a4d2650c2', spUUID='c4777d10-e010-4a3a-8fe1-9149263e5bf4', imgUUID='47ae4d42-7fda-4e86-ad21-942f802a3e7d', volUUID='bbd21e13-3b14-4592-b723-19032750a639', options=None)
>Thread-100::INFO::2014-01-22 20:52:43,033::logUtils::42::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1073741824', 'apparentsize': '1073741824'}
>Thread-100::DEBUG::2014-01-22 20:52:43,033::task::1168::TaskManager.Task::(prepare) Task=`548426bb-a8be-48fd-9a34-5b3f1014ebcc`::finished: {'truesize': '1073741824', 'apparentsize': '1073741824'}
>Thread-100::DEBUG::2014-01-22 20:52:43,034::task::579::TaskManager.Task::(_updateState) Task=`548426bb-a8be-48fd-9a34-5b3f1014ebcc`::moving from state preparing -> state finished
>Thread-100::DEBUG::2014-01-22 20:52:43,034::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-100::DEBUG::2014-01-22 20:52:43,034::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-100::DEBUG::2014-01-22 20:52:43,034::task::974::TaskManager.Task::(_decref) Task=`548426bb-a8be-48fd-9a34-5b3f1014ebcc`::ref 0 aborting False
>VM Channels Listener::DEBUG::2014-01-22 20:52:43,444::vmChannels::90::vds::(_handle_timeouts) Timeout on fileno 16.
>Thread-237::DEBUG::2014-01-22 20:52:48,342::task::579::TaskManager.Task::(_updateState) Task=`9629a13d-8ecc-4dc1-967e-03992e26f2c0`::moving from state init -> state preparing
>Thread-237::INFO::2014-01-22 20:52:48,342::logUtils::40::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-237::INFO::2014-01-22 20:52:48,342::logUtils::42::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9b0af9c6-96a2-43c5-857a-61e9f45b6cdf': {'delay': '0', 'lastCheck': '6.7', 'code': 358, 'valid': False, 'version': -1}, u'7af59253-11d4-43e4-8d3c-a68a4d2650c2': {'delay': '0.000663136', 'lastCheck': '7.6', 'code': 0, 'valid': True, 'version': 3}}
>Thread-237::DEBUG::2014-01-22 20:52:48,342::task::1168::TaskManager.Task::(prepare) Task=`9629a13d-8ecc-4dc1-967e-03992e26f2c0`::finished: {u'9b0af9c6-96a2-43c5-857a-61e9f45b6cdf': {'delay': '0', 'lastCheck': '6.7', 'code': 358, 'valid': False, 'version': -1}, u'7af59253-11d4-43e4-8d3c-a68a4d2650c2': {'delay': '0.000663136', 'lastCheck': '7.6', 'code': 0, 'valid': True, 'version': 3}}
>Thread-237::DEBUG::2014-01-22 20:52:48,342::task::579::TaskManager.Task::(_updateState) Task=`9629a13d-8ecc-4dc1-967e-03992e26f2c0`::moving from state preparing -> state finished
>Thread-237::DEBUG::2014-01-22 20:52:48,343::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-237::DEBUG::2014-01-22 20:52:48,343::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-237::DEBUG::2014-01-22 20:52:48,343::task::974::TaskManager.Task::(_decref) Task=`9629a13d-8ecc-4dc1-967e-03992e26f2c0`::ref 0 aborting False
>Thread-19::DEBUG::2014-01-22 20:52:50,703::misc::83::Storage.Misc.excCmd::(<lambda>) '/bin/dd iflag=direct if=/dev/7af59253-11d4-43e4-8d3c-a68a4d2650c2/metadata bs=4096 count=1' (cwd None)
>Thread-19::DEBUG::2014-01-22 20:52:50,711::misc::83::Storage.Misc.excCmd::(<lambda>) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000873191 s, 4.7 MB/s\n'; <rc> = 0
>Thread-18::ERROR::2014-01-22 20:52:51,681::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf
>Thread-18::ERROR::2014-01-22 20:52:51,681::sdc::153::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf
>Thread-18::DEBUG::2014-01-22 20:52:51,681::lvm::374::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-18::DEBUG::2014-01-22 20:52:51,682::misc::83::Storage.Misc.excCmd::(<lambda>) u'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \'a|/dev/mapper/36090a038d0f751994bf9045fb249cf34|/dev/mapper/36090a038d0f7b16e4df9245fb249cf30|\', \'r|.*|\' ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf' (cwd None)
>Thread-18::DEBUG::2014-01-22 20:52:51,848::misc::83::Storage.Misc.excCmd::(<lambda>) FAILED: <err> = '  Volume group "9b0af9c6-96a2-43c5-857a-61e9f45b6cdf" not found\n  Skipping volume group 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf\n'; <rc> = 5
>Thread-18::WARNING::2014-01-22 20:52:51,850::lvm::379::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['  Volume group "9b0af9c6-96a2-43c5-857a-61e9f45b6cdf" not found', '  Skipping volume group 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf']
>Thread-18::DEBUG::2014-01-22 20:52:51,850::lvm::403::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-18::ERROR::2014-01-22 20:52:51,851::sdc::143::Storage.StorageDomainCache::(_findDomain) domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf not found
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain
>    dom = findMethod(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 170, in _findUnfetchedDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'9b0af9c6-96a2-43c5-857a-61e9f45b6cdf',)
>Thread-18::ERROR::2014-01-22 20:52:51,851::domainMonitor::225::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain 9b0af9c6-96a2-43c5-857a-61e9f45b6cdf monitoring information
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/domainMonitor.py", line 190, in _monitorDomain
>    self.domain = sdCache.produce(self.sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 98, in produce
>    domain.getRealDomain()
>  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
>    return self._cache._realProduce(self._sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce
>    domain = self._findDomain(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain
>    dom = findMethod(sdUUID)
>  File "/usr/share/vdsm/storage/sdc.py", line 170, in _findUnfetchedDomain
>    raise se.StorageDomainDoesNotExist(sdUUID)
>StorageDomainDoesNotExist: Storage domain does not exist: (u'9b0af9c6-96a2-43c5-857a-61e9f45b6cdf',)

Actions: View

Attachments on bug 1056544: 853858